KR20190115119A - Sample region merging - Google Patents

Abstract

According to one embodiment the arrangement of information samples is coded with a reduced amount of data in which preferred merging or grouping of subdivided simple connection regions is reduced. To this end, for each of the simple connection regions, for a predetermined simple connection region, the identification and activation of the simple connection regions in the plurality of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is defined. do. That is, if the number is zero, there may be no merge indicator for the predetermined simple connection area in the data stream. According to still further embodiments, by using multi-tree subdivision, samples representing spatial sampling of a two-dimensional information signal by recursively multi-partitioning such that simple connection regions have different sizes. Spatial subdivision into simple connection regions is performed according to a first subset of syntax elements included in the data stream, followed by a second subset of syntax elements in the data stream, separated from the first subset. Accordingly, the consolidation of the arrangement of information samples is a combination of spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision into separate subsets of simple connection regions, the plurality of simple connection regions. This is done. Intermediate sub-division is used when recovering the arrangement of samples from the data stream.

Description

Sample region merging

The present invention relates to coding techniques for two-dimensionally sampled information signals, such as video or still pictures.

In image and video coding, certain sets of pictures or sample arrangements for pictures are usually broken down into blocks associated with specific coding parameters. Pictures are usually composed of multiple sample arrays. In addition, the picture is also associated with additional auxiliary sample arrangements, which may specify transparency information or depth maps, for example. The sample arrangements (including secondary sample arrangements) of the picture can be grouped into at least one so-called plane groups, where each plane group consists of at least one sample arrangement. Plane groups of a picture may be coded independently or with prediction from other plane groups of the same picture, if the picture is associated with one or more plane groups. Each plane group is usually broken up into blocks. Blocks (or corresponding blocks of sample arrangements) are predicted by inter-picture prediction or intra-picture prediction. The blocks may have different sizes and may be rectangular or rectangular. The division of the picture into blocks is fixed by syntax, or it can be signaled in the bitstream. Syntax elements, which are signals that subdivide into blocks of predefined sizes, are often transmitted. Such syntax elements indicate how the block is subdivided into smaller blocks and associated with coding parameters, ie for the purpose of prediction. For all samples of a block (or blocks corresponding to sample arrangements), the decoding of the associated coding parameters is specified in some way. For example, all samples in a block may include reference indices (which identify a reference picture in a set of already coded pictures), motion parameters (specifying a measure for the motion of the block between the reference picture and the current picture), It can be predicted using the same set of prediction parameters, such as parameters specifying interpolation filter, intra prediction modes, and the like. The motion parameters may be represented by displacement vectors with horizontal and vertical components or by higher order motion parameters such as affine motion parameters consisting of six elements. In addition, one or more sets of specific prediction parameters (such as reference indices and motion parameters) are associated with a single block. In such a case, for each set of these specific prediction parameters, a single intermediate prediction signal for the block (or corresponding blocks of sample arrangements) is generated and the final prediction signal overlaps the intermediate signals. It is made by a combination that includes. Corresponding weight parameters and potentially also a constant offset (in addition to the weighted sum) are fixed for the picture, or reference picture, or sets of reference pictures, or they are a set of prediction parameters for the corresponding block. Can be included. The difference between the original blocks (or corresponding blocks of sample arrays) and their prediction signals is also referred to as a residual signal and is usually transformed and quantized. Often, two-dimensional transform is applied to the residual signal (or corresponding sample arrangements for the residual block). For transform coding, the blocks (or blocks corresponding to the sample arrangements) can be further divided before the transform is applied, such that a specific set of prediction parameters is used. The transform blocks can be the same or smaller than the block used for prediction. The transform block may comprise one or more of the blocks used for prediction. Different transform blocks may have different sizes, and the transform blocks may be represented by a square or a rectangle. After the transform, the resulting transform coefficients are quantized and so-called transform coefficient levels are obtained. Transform coefficient levels as well as prediction parameters, and, if present, subdivision information are entropy coded.

In image and video coding standards, the possibilities provided by the syntax for a picture (or plane group) to subdivide into blocks are very limited. Usually, it can only specify how (and potentially how) a block of predefined size can be subdivided into smaller blocks. For example, the largest block size in H.264 is 16x16. 16 × 16 blocks also appear as macroblocks and each picture can be divided into macroblocks in the first step. For each 16 × 16 macroblock, it can be signaled whether it is coded as a 16 × 16 block, or as two 16 × 8 blocks, or as two 8 × 16 blocks, or as four 8 × 8 blocks. have. If a 16x16 block is subdivided into four 8x8 blocks, each of these 8x8 blocks is as one 8x8 block, or as two 8x4 blocks, or two 4x8 blocks. Or as four 4 × 4 blocks. A small set of possibilities for specifying division into blocks in the latest image and video coding standards has the advantage of keeping the auxiliary information for signaling subdivision information small, but as described below for The disadvantage is that the bit rate required for transmitting the prediction parameters can be important. The auxiliary information rate for signaling the prediction information can usually represent an important total of the overall bit rate for the block. And the coding efficiency can increase when the auxiliary information decreases, for example, can be achieved by using larger block sizes. The actual images or pictures of the video sequence are composed of objects that are arbitrarily shaped with specific characteristics. For example, such objects or portions of objects may be characterized by unique textures or distinctive weaves. And in general, the same set of prediction parameters can be applied to such an object or part of an object. However, the boundary of the object usually does not coincide with the possible block boundary for large prediction blocks (ie 16 × 16 macroblocks in H.264). The encoder usually determines the subdivision (between a limited set of possibilities) that results at the minimum of a particular rate-distortion cost measure. For arbitrarily shaped objects this can result in many small blocks. And, since each of these small blocks is associated with a set of prediction parameters that need to be transmitted, the auxiliary information rate can be an important part of the overall bit rate. However, because some of the small blocks still represent regions or parts of the same object, the prediction parameters for the number of blocks obtained are the same or very similar.

That is, tiling a subdivision or picture into smaller portions or tiles or blocks has a significant impact on coding efficiency and coding complexity. As outlined above. By enabling better adaptation of these coding parameters to the picture / video medium, subdivision of the picture with a higher number of smaller blocks allows for spatially better setting of coding parameters. On the other hand, setting coding parameters at better granularity places a high burden on the total amount of assistance information necessary to inform the decoder on the necessary settings. Even the freedom of the encoder to subdivide the picture / video into blocks spatially greatly increases the total number of possible coding parameter settings, thereby making the search for coding parameter settings generally leading to the best rate / distortion. It can make it harder.

The present invention aims to provide a coding technique for coding an array of information samples representing a spatially sampled two-dimensional information signal, such as, but not limited to, pictures of video or still pictures. For the rate distortion rate and / or a better compromise between encoding complexity and the rate distortion rate achievable can be achieved.

This object is achieved by a decoder according to claims 1, 8 or 9, an encoder according to claim 20, 22, 23 or 24, methods according to any one of claims 19, 20, 25 and 26, a computer program according to claim 27 and claim 28. Or by a data stream according to 30.

According to an embodiment, the preferred merging or grouping of simple connection regions into which the arrangement of information samples is subdivided is coded with reduced data totals. For this purpose, for a predetermined simple connection area, a predetermined relative regional relationship is defined to enable identification of simple connection areas within a plurality of simple connection areas having a predetermined relative local relationship to the predetermined simple connection area. . That is, if the number is zero, a merge indicator for the predetermined simple connection area may be absent in the data stream. Furthermore, if the number of simple connection regions having a predetermined relative locality in the predetermined simple connection region is one, the coding parameters of the simple connection region are coding parameters for the predetermined simple connection region without the need for any additional syntax elements. It may be applied or used for prediction for the purpose. Otherwise, i.e., if the number of simple connection areas with a predetermined relative local relation to the predetermined simple connection areas is greater than one, even if the coding parameters associated with these identified simple connection areas are the same, Introduction of the urea can be suppressed.

According to one embodiment, if the coding parameters of adjacent simple connection regions do not coincide with each other, the reference neighbor identifier checks a true subset of the number of simple connection regions having a predetermined relative local relationship to the predetermined simple connection region. As can be seen, this true subset is used to apply coding parameters or to predict coding parameters of a predetermined simple connection region.

According to another embodiment, a spatial subdivision of an area of samples representing a spatial sampling of a two-dimensional information signal into a plurality of simple connected regions of different sizes by recursive multi-division is included in the data stream. Separated from the first subset to obtain an intermediate level of subdivision of the arrangement of samples into separate sets of simple connection regions, performed according to a first subset of syntax elements, which is a combination of a plurality of simple connection regions. Followed by a combination of spatially adjacent simple connection regions according to a second subset of syntax elements in the data stream. Intermediate subdivision is used when restoring the arrangement of samples from the data stream. This is due to the fact that too precise subdivision can be compensated by subsequent merging, so optimization for subdivision may be less important. Moreover, the combination of subdivision and merging is such that the concatenation of subdivision and merging by the use of separate sets of syntax elements is effective in the actual context of the two-dimensional information signal or better at intermediate subdivisions. It is possible to achieve intermediate subdivision which is impossible by means of a recursive multi-division to enable the application. In comparison with the advantages, the additional overhead resulting from the additional subset of syntax elements to detail the merge can be ignored.

According to an embodiment, the arrangement of information samples representing the spatially sampled information signal is into a tree into smaller simply connected regions of different sizes by recursive multi-division of tree root regions. In at least one subset of root regions, according to the multi-tree-sub-division information extracted from the data-stream, spatially the first in the tree root region with the following subdivision: have. In order to find a good compromise between too precise subdivision and too coarse subdivision in the rate-distortion sense, at a reasonable coding complexity, the maximum region size of the tree root regions where the array of information samples is spatially subdivided is included in the data stream. And extracted from the data stream on the decoding side. Thus, the decoder extracts a maximum region size and information to multi-tree-detail from the data stream, a subset of the tree root regions into a tree root into smaller simple concatenated regions of different sizes by recursive multi-division. In at least one subset of regions, according to the information to be multi-tree-detailed, a detail division that spatially divides the array of information samples represented by the spatially sampled information signal into tree root regions of maximum region size and subdivision Sub-divider; And a reconstruction unit for reconstructing the arrangement of information samples from the data stream using subdivision into smaller simple connection regions.

According to an embodiment, the data stream also includes a maximum hierarchy that is adjusted upward by causing a subset of the tree root regions to be recursively multidivided. By this measure, the signaling of the information to be multi-tree-detailed is easier and requires less bits for coding.

In addition, the reconstruction unit performs one or more of the following measures in granularity according to intermediate subdivision: at least, determining the prediction mode between the intra and inter prediction modes to be used; Transform from spectral to spatial domain, performing and / or setting parameters for inter-prediction; Performing and / or performing parameters for intra prediction.

In addition, the extractor may extract syntax elements associated with the leaf regions of the treeblocks partitioned in a depth-first traversal order from the data stream. By this measure, the extractor can use the statistics of the syntax elements of the already coded adjacent leaf regions with a higher probability than using the breadth-first traversal order.

According to another embodiment, the further subdivision is used to subdivide according to the further multi-tree subdivision information in at least one subset of the smaller simple connection regions even in smaller simple connection regions. do. The second-stage subdivision can be used by the reconstruction section to perform reconversion from spectral to the spatial domain, while the first-stage subdivision can be used by the reconstruction section to perform prediction of the region of information samples. Defining residual subdivisions that are relatively dependent on prediction subdivision provides coding of the overall subdivision, which consumes less bits, while on the other hand, restrictions or freedom for residual subdivisions resulting from dependencies are generally coded. It only has a slight negative impact on efficiency, and parts of the picture with similar motion compensation parameters are larger than parts with similar spectral characteristics.

According to another embodiment, an additional maximum region size is included in the data stream, and the additional maximum region size is in at least one subset of the tree root subregions according to the additional multi-tree subdivision information in even smaller simple concatenated regions. Prior to subdivision, the smaller simple connection regions define the size of the first tree root subdivisions. This, in turn, can independently set the maximum area sizes of the prediction subdivision on the one hand and the residual subdivision on the other hand, thus finding a better rate / distortion compromise.

According to another embodiment of the invention, the data stream comprises a first subset of syntax elements separated from a second subset of syntax elements forming a multi-tree subdivision information, wherein the middle of the arrangement of samples According to the first subset of syntax elements, a merger may combine at the decoding side, spatially adjacent to the smaller simple concatenated regions of the multi-tree subdivision to obtain subdivision. The reconstruction unit may reconstruct the array of samples using intermediate subdivision. By this measure, the encoder is easier to subdivide effectively the spatial distribution of the arrangement characteristics of the information samples while finding the optimum rate / distortion compromise. For example, if the maximum region size is high, the multi-tree subdivision information may be more complicated due to the further expansion of the tree root regions. On the other hand, however, if the maximum region size is small, adjacent tree root regions are associated with information content with similar characteristics such that tree root regions can also be processed together. Merging fills in these gaps between the extremes described above, thereby enabling near optimal subdivision of granularity. From the encoder's point of view, merging of syntax elements may allow for a more slack or computationally less complex encoding procedure because the encoder incorrectly uses too precise subdivision, and the syntax elements set up prior to setting merge syntax elements. This error can be compensated for later by setting the syntax elements to merge later, with or without matching only a small portion of them.

According to another embodiment, the maximum region size and the multi-tree-detail information may be used for residual subdivision rather than prediction subdivision.

The depth-first traversal order for dealing with simple connection regions of quadtree subdivision of an array of information samples representing spatially sampled information signals is used according to an embodiment rather than the width-first traversal order. By using the depth-first traversal order, each simple connection region has a higher probability of having adjacent simple connection regions that have already been traversed so that information about these adjacent simple connection regions is positive when restoring each current simple connection region. Used as

The arrangement of information samples is a regular arrangement of tree root regions of zero-order hierarchical size with the following subdivision in at least one subset of tree root regions within smaller simple concatenated regions of different sizes. When divided into the tree roots, the restorer handles the tree root regions so that each tree root region is split, dealing with simple-connected leaf regions in a depth-first traversal sequence before proceeding further to the next tree root region in the zigzag scan order. A zigzag scan can be used to scan areas. In addition, consistent with the depth-first traversal order, simple connected leaf regions of the same hierarchical level may be traversed in a zigzag scan order. Thus, the increased likelihood of having adjacent simple connected leaf regions is maintained.

According to an embodiment, even if the flags associated with the nodes of the multi-tree structure are arranged in a depth-first traversal sequence in succession, the continuous coding of the flags is multiplied within the same hierarchical level of the multi-tree structure. Use and, in doing so, different, probability estimation contexts for the nodes of the same, but multi-tree structure that lies at different hierarchical levels of the multi-tree structure for the flags associated with the nodes of the tree structure. On the other hand it may allow a good compromise between the number of contexts made and the adaptation to the actual symbol statistics of the flags.

According to an embodiment, the probability estimation contexts for the predetermined flag to be used also process a predetermined flag that matches the depth-first traversal order, and have a tree having a predetermined relative positional relationship to a region corresponding to the predetermined flag. It depends on the flags corresponding to the regions of the root area. Similar to the idea underlying the progress that has been made, the use of the depth-first traversal order is a context in which this knowledge is used for predetermined flags, including flags corresponding to adjacent regions where already coded flags correspond to predetermined flags. Better adaptation to this guarantees a high probability of being used.

The flags used to set the context for the predetermined flag may correspond to the area lying on top and / or left of the area corresponding to the predetermined flag. In addition, the flags used to select the context may be limited to flags belonging to the same hierarchical level as the node with which the predetermined flag is associated.

According to an embodiment, the coded signaling comprises a sequence of flags associated with nodes of a multi-tree structure that do not match the indication of the highest hierarchical level and the highest hierarchical level, wherein each flag is associated with the associated nodes. Specify whether it is an intermediate node or a child node, and successively decoded, in a depth-first or width-first first traversal sequence, a sequence of flags occurs from the data stream, skipping the nodes of the highest hierarchical level, and leaving the same leaf node Setting them automatically, thereby reducing the coding rate.

According to another embodiment, the coded signaling of the multi-tree structure may comprise an indication of the highest hierarchical level. This measure can limit the presence of flags at hierarchical levels other than the highest hierarchical level as further division of blocks of the highest hierarchical level is still excluded.

In the case of the non-partitioned tree root regions of the spatial multi-tree subdivision and primary multi-tree subdivision, which are part of the secondary subdivision of leaf nodes, the coding flags of the secondary subdivision The context used for may be selected such that the context is the same for flags associated with regions of the same size.

According to another embodiment, the preferred merging or grouping of simple connection regions into which the arrangement of information samples is subdivided is coded with a reduced total amount of data. For this purpose, a predetermined relative positional relationship, for a predetermined simple connection area, of the simple connection areas, in a plurality of simple connection areas with a predetermined relative local relationship to the predetermined simple connection area, for the simple connection areas. Is defined to enable some verification. That is, if the number is zero, a merge indicator for the predetermined simple connection area may be absent in the data stream. Furthermore, if the number of simple connection regions with a predetermined relative regional relationship is one, the coding parameters of the simple connection region are used for prediction for the coding parameters for the predetermined simple connection region without the need for any additional syntax elements. May be applied or used. Otherwise, i.e., if the number of simple connection areas with a predetermined relative locality relative to the predetermined simple connection areas is greater than one, even if the coding parameters associated with these identified simple connection areas are each identical to each other, The introduction of syntax elements can be suppressed.

According to an embodiment, if the coding parameters of adjacent simple connection regions do not coincide with each other, then the reference neighbor identifier identifies a true subset of the number of simple connection regions having a predetermined relative local relationship to the predetermined simple connection region. This true subset is used to apply coding parameters or to predict coding parameters of a predetermined simple connection region.

According to another embodiment, the spatial subdivision of an area of samples representing a spatial sampling of a two-dimensional information signal into a plurality of simple connected regions of different sizes by regressive multi-segmentation is achieved by the syntax elements of the data stream. In a data stream separate from the first subset, to obtain a medium level of subdivision of the arrangement of samples into separate sets of simple connection regions, performed according to one subset and combining a plurality of simple connection regions. This is followed by a combination of spatially adjacent simple connection regions according to the second subset of syntax elements. Intermediate subdivision is used when restoring the arrangement of samples from the data stream. This is due to the fact that too precise subdivision can be compensated by subsequent merging, so optimization for subdivision may be less important. Moreover, the combination of subdivision and merging allows the continuation of subdivision and merging by the use of separate sets of syntax elements to enable better application of intermediate or subdivided subdivision in the actual context of the two-dimensional information signal. It is possible to achieve the intermediate subdivision impossible by a recursive multi-division method. Comparing the advantages, the additional overhead resulting from the additional subset of syntax elements to detail the merge can be ignored.

When using the method and apparatus according to the embodiment of the present invention, redundancy can be reduced through the association between coding parameters of planes.

In addition, better rate distortion ratio can be achieved through inter-plane prediction according to the present invention.

Preferred embodiments of the present invention will be described with reference to the following drawings.
1 is a block diagram of an encoder according to an embodiment of the present invention.
2 is a block diagram of a decoder according to an embodiment of the present invention.
3A-3C schematically illustrate an embodiment for quadtree subdivision, where FIG. 3A shows a first hierarchical level, FIG. 3B shows a second hierarchical level, and FIG. 3C shows a third hierarchical level.
4 schematically illustrates a tree structure for quadtree subdivision according to the embodiments of FIGS. 3A-3B in accordance with an embodiment of the present invention.
5A and 5B diagrammatically illustrate a tree structure with indices indexing the quadtree subdivision and leaf blocks of FIGS. 3A-3C.
6A and 6B schematically illustrate the binary tree series or sequences of flags representing the tree structure of FIG. 4 and the quadtree subdivision of FIGS. 3A-3C, in accordance with each of the other embodiments.
7 is a flowchart illustrating steps performed by a data stream extracting unit according to an embodiment.
8 is a flowchart illustrating a function of a data stream extracting unit according to another embodiment.
9A and 9B are diagrams illustrating quadtree subdivisions with adjacent candidate blocks for a predetermined block highlighted in accordance with an embodiment.
10 is a flowchart illustrating a function of a data stream extracting unit according to another embodiment.
11 is a diagram illustrating a configuration of a picture from planes and plane groups according to an embodiment, and illustrating coding using interplanar application / prediction.
12A and 12B diagrammatically illustrate a subtree structure and corresponding subdivision for describing an inheritance scheme, in accordance with an embodiment.
12C and 12D schematically illustrate a subtree structure for explaining the inheritance technique with adaptation and prediction, respectively, according to an embodiment.
13 is a flowchart illustrating a step of realizing an inheritance technique by an encoder according to an embodiment.
14A and 14B illustrate primary subdivisions and secondary subdivisions to illustrate the possibility of performing inheritance techniques related to inter-prediction, in accordance with an embodiment.
15 is a block diagram illustrating a decoding procedure associated with an inheritance technique, according to an embodiment.
16 schematically illustrates a scanning order among sub-regions of a multitree subdivision according to an embodiment, with sub-regions for intra prediction.
17 is a block diagram of a decoder according to an embodiment.
18A-18C illustrate different possibilities for subdivisions in accordance with another embodiment.
19 is a block diagram of an encoder according to an embodiment of the present invention.
20 is a block diagram of a decoder according to another embodiment of the present invention.
21 is a block diagram of an encoder according to another embodiment of the present invention.

According to the description of the figures, elements occurring in some of these figures are indicated by general reference numerals, and repeated descriptions of these elements are avoided. Rather, the description of the elements indicated in one drawing also applies to the other drawings. Deviations from them are indicated so that each element occurs only in the descriptions presented in these other figures.

In addition, the following description begins with the elements of the encoder and decoder described with respect to FIGS. Embodiments describe these figures, combine with many aspects of the present application, but are also advantageous if performed separately in a coding scheme, and therefore, for the following figures, the embodiments have different meanings. And briefly discussed using the exactly mentioned aspects with each of these embodiments illustrating the idea of an embodiment described with respect to FIGS. 1 and 11.

1 shows an encoder according to an embodiment of the invention. The encoder 10 of FIG. 1 includes a predictor 12, a residual precoder 14, a residual reconstructor 16, a data stream inserter 18, and a block divider 20. Encoder 10 is for coding a spatiotemporally sampled information signal into data stream 22. The space-time sampled information signal may be, for example, video (ie, a sequence of pictures). Each picture represents an array of image samples. For example, other examples of spatiotemporal information signal may include depth images captured by, for example, time-of-light cameras. In addition, the spatially sampled information signal may be one or more arrays per frame, such as, for example, in the case of color video comprising an array of luma samples with two arrays of chroma samples per frame. Or include a time stamp. In other words, the spatial sampling rates for the other components of the information signal such as luma and chroma may be different. The same applies to spatial resolution. In addition, the video may involve additionally spatially sampled information such as depth or transparency information. However, the following description will focus on the processing of one of these arrangements for the purpose of better understanding the primary issues of the present application which utilizes the transition to processing of one or more planes.

The encoder 10 of FIG. 1 is configured to generate the data stream 22 such that the syntax elements of the data stream 22 describe the pictures in granularity that lies between the entire pictures and the individual image samples. . For this purpose, the divider 20 is configured to sub-divide each picture 24 into simple connecting regions of different sizes 26. In the following, these regions may simply be called blocks or sub regions 26.

As will be described in more detail below, the divider 20 may use multi-tree sub-division to subdivide the picture 24 into blocks 26 of different sizes. Can be. To be even more accurate, the detailed embodiments described below with respect to FIGS. 1 through 11 mostly use quadtree subdivision. As will be explained in more detail below, the divider 20 divides the pictures 24 into subdivisions 28 for subdividing the pictures 24 into the just mentioned blocks 26 followed by the merger 30. Merge section 30 includes an effective subdivision or granularity located between the subdivision defined by subdivision section 28 and the non-sub-division of pictures 24. It is possible to combine the groups of these blocks 26 to obtain.

As indicated by dotted lines in FIG. 1, the predictor 12, the residual precoder 14, the residual reconstructor 16, and the data stream inserter 18 are on the picture subdivisions defined by the divider 20. It works. For example, as will be described in more detail below, whether or not an individual subregion should be intra picture predicted using the setting of corresponding prediction parameters for the individual subregion according to the selected prediction mode or inter. As to whether the picture should be predicted, the prediction unit 12 uses the prediction subdivision defined by the division unit 20 to make decisions for the individual subregions of the prediction subdivision.

Next, the residual precoder 14 may use the residual subdivision of the pictures 24 to encode the residual of the prediction of the pictures 24 provided by the predictor 12. The residual restoring unit 16 restores the residual from the syntax elements output by the residual precoder 14, and the residual restoring unit 16 also operates on the above-mentioned residual subdivision. The data stream inserter 18 may, for example, use means such as entropy encoding among syntax elements for insertion of the output of syntax elements by the residual precoder 14 and the predictor 12. The above subdivisions, i.e. subdivision and residual subdivision, can be used for the determination of the insertion order and neighbor relationship into the data stream 22 by means of this.

As shown in FIG. 1, the encoder 10 includes an input 32 through which an original information signal enters the encoder 10. The subtractor 34, the residual precoder 14 and the data stream inserter 18 are within the order mentioned between the output and input 32 of the data stream inserter 18 from which the coded data stream 22 is output. Are connected sequentially. The subtractor 34 and the residual precoder 14 are part of a prediction loop closed by the residual restoring unit 16, the adding unit 36, and the predictor 12, and the residual restoring unit 16. The adder 36 and the predictor 12 are sequentially connected within the stated order between the output of the residual precoder 14 and the transform output of the subtractor 34. The output of predictor 12 is also connected to an additional output of adder 36. In addition, the predictor 12 includes an input connected directly to the output 32 and further connected to the output of the adder 36 via an optional in-loop filter 38. It may include additional input. In addition, the predictor 12 generates auxiliary information during operation, so that the output of the predictor 12 is also coupled to the data stream inserter 18. Similarly, divider 20 includes an output coupled to the other input of data stream inserter 18.

The mode of operation that describes the structure of the encoder 10 is described in more detail below.

As described above, the divider 20 determines, for each picture 24, how to subdivide equally into sub-areas 26. According to the subdivision of the picture 24 used for prediction, the prediction unit 12 determines a method for predicting the individual subregions for each subregion corresponding to the subdivision. The predictor 12 outputs the prediction of the subregion for the inverting input of the subtractor 34 or the additional input of the adder 36 to the data stream inserter 18, and the predictor 12 outputs the video. The prediction information reflecting a method of obtaining such a prediction from the pre-encoded parts is output to the data stream inserter 18.

At the output of the subtractor 34, the prediction residual is thus obtained, where the residual precoder 14 processes this prediction residual in accordance with the residual subdivision further defined by the divider 20. As described below in more detail with respect to FIGS. 3 to 10, the picture 24 used by the residual precoder 14 such that each prediction subregion is further subdivided into small residual subregions or adopted as a residual subregion. The residual subdivision for) may be related to the prediction subdivision used by the prediction unit 12. However, overall independent prediction or residual subdivisions may also be possible.

Residual precoder 14 transforms each residual subregion from a space for the spectral domain by the following two-dimensional transform or quantization of the generated transform coefficients of the generated transform blocks, which are essentially related. This results in distortion from the quantization noise. For example, the data stream inserter 18 may, for example, entropy encode syntax elements describing the aforementioned transform coefficients. Using # will allow lossless encoding to the data stream 22.

Next, the residual reconstruction unit 16 reconstructs the transform coefficients into a residual signal by using requantization according to the reconversion, where the residual signal is used to estimate the prediction used by the subtractor 34 for obtaining the prediction residual. Combined in the adder 36 to be used, thereby obtaining the detail area or the reconstructed portion of the current picture at the output of the adder 36. The prediction unit 12 may directly use the reconstructed picture detail region for intra prediction, which is for predicting a specific prediction detail region by extrapolation from adjacent already reconstructed prediction detail regions. However, intra prediction is performed within the spectral domain by prediction of the spectrum of the current subregion from an adjacent subject, and may also be directly theoretically possible.

For inter prediction, the prediction unit 12 may use pictures that have already been encoded and reconstructed in a version according to the same object that has been filtered by the optional in-loop filter 38. For example, in-loop filter 38 may comprise an adaptive filter and / or a de-blocking filter having a transfer function which is advantageously applied in the form of the quantization noise mentioned above. will be.

The prediction unit 12 selects prediction parameters that are revealed as a method of predicting a specific prediction subregion by a method of comparing original samples in the picture 24. As described in more detail below, the prediction parameters may include an indication of a prediction mode for each prediction subregion, such as intra picture prediction and inter picture prediction. In the case of intra picture prediction, the prediction parameters may also include an indication of the angle according to the edges within the prediction detail region mainly intended to extend to intra prediction, and inter picture prediction, motion vectors In the case of motion picture indices and, finally, in the case of high-dimensional motion transform parameters and in the case of intra and / or inter picture prediction, selective filter information for filtering the reconstructed image samples based on the current prediction detail region is provided. It is predicted.

As will be described in more detail below, the above-mentioned subdivisions defined by the divider 20 are maximized by the residual precoder 14, the predictor 12 and the data stream inserter 18. It significantly affects the possible rate / distortion ratio. In the case of very precise subdivision, the prediction parameters 40 output by the divider 12 inserted into the data stream 22 may yield very high coding rates, even if the prediction obtained by the predictor 12 is better. And the residual signal coded by the residual precoder 14 may be less to be coded by the same few bits. In the case of very coarse subdivisions, the opposite applies. In addition, the notion just mentioned also applies for residual subdivision in a similar manner, where the transformation of a picture using good granularity of individual transform blocks results in low complexity for the computation of the transforms and increased spatial resolution of the resulting transform. do. That is, the small residual subregions activate spectral distortion of the content within the individual residual subregions for further matching. However, the spectral resolution is reduced, and the ratio coefficients between significant or insignificant (i.e. quantized to '0') can be bad. In other words, the granularity of the transformation should be applied locally to the picture content. In addition, independent of the positive effects of finer granularity, finer granularity regularly increases the amount of assistance information needed to represent the subdivision selected by the decoder. As will be described in the detailed description below, the embodiments described below signal and encode subdivisions for use at the decoding side by instructing a data stream inserter to insert subdivision information into the coded data stream 22. An encoder 10 having the ability to adapt subdivisions very efficiently into the content of an information signal to be provided is provided. Details are given below.

However, before defining the subdivision of the divider 20 in the detailed description, the decoder corresponding to the embodiment of the present invention is described in more detail with reference to FIG.

The decoder of FIG. 2 is denoted by the reference numeral 100, and the extractor 102, the divider 104, the residual reconstructor 106, the adder 108, the predictor 110, the selective in-loop filter 112 ) And an optional post filter 114. The extractor 102 receives the data stream coded at the input 116 of the decoder 100, and extracts the coded data stream subdivision information 118, the prediction parameters 120, and the residual data 122. The 102 outputs the output to the picture division unit 104, the prediction unit 110, and the residual reconstruction unit 106, respectively. The residual restoring unit 106 has an output connected with the first output of the adding unit 108. The other input of the adder 108 and its output are connected in a prediction loop, with the optional in-loop filter 112 and the predictor 110 connected to the predictor 110 from the output of the adder 108. The by-pass path is sequentially connected to the prediction loop in the order mentioned, which is directly similar to the above-mentioned connections between the adder 36 and the predictor 12 in FIG. It is a target for intra picture prediction and another target for inter picture prediction. Either the output of the adder 108 or the output of the in-loop filter 112 may be connected to the output 124 of the decoder 100, where the reconstruction information signal is, for example, regenerated. Output to device. The optional post filter 114 may be connected to a path that causes the output 124 to enhance the visual quality of the visual impression of the reconstruction signal at the output 124.

Generally speaking, the residual reconstruction unit 106, the adder 108, and the predictor 110 behave like elements 16, 36, and 12 of FIG. 1. That is, they mimic the operation of the elements of FIG. 1 mentioned above. For this purpose, the prediction subregions are determined in the same manner as the determination by the prediction unit 12 to perform or to perform, and the reconstruction of the transform coefficients received at the same granularity as the residual precoder 14 performs. In order to predict the residuals, the residual reconstructing unit 106 and the predicting unit 110 divide the subdivision and prediction parameters 120 defined by the picture dividing unit 104 according to the subdividing information 118 from the extracting unit 102. Respectively). Next, the picture division unit 104 reconstructs the subdivisions selected by the division unit 20 of FIG. 1 in a synchronized manner by relying on the subdivision information 118. Next, the extractor may use the subdivision information to control data extraction, for example with respect to context selection, neighbor region determination, probability estimation, parsing the data stream, and the like.

Some deviations may be made on the above embodiments. Some are mentioned within the following detailed description of the subdivision performed by subdivision section 28 and the merging performed by merger 30, and others are described with respect to the following FIGS. 12-16. In the absence of any obstacles, all their deviations may be applied to the above mentioned description of FIGS. 1 and 2, respectively, individually or in subsets. For example, the partitions 20 and 104 may not determine prediction subdivision and residual subdivision only per picture. More precisely, they may also determine filter subdivisions for the selective in-loop filters 38 and 112, respectively, and may determine either independence or dependency, or residual coding, respectively, from the other subdivisions for prediction. . In addition, the determination of subdivision or subdivisions by their elements may not be performed on a frame by frame reference. More precisely, the subdivision or subdivisions determined for a particular frame may only be reused or applied for a particular number of subsequent frames with the subsequent transformation of the new subdivision.

In a more detailed description of the division of pictures into sub-regions, the following description focuses first on subdivisions, with subdivisions 28 and 104a being responsible for subdivisions. Therefore, the merging process is described, and the merging section 30 and the merging section 104b assume responsibility for the merging process. Finally, inter plane adoption / prediction is described.

In the method the subdivisions 28 and 104a divide the pictures so that the picture is divided into a number of blocks, perhaps of different sizes, for the purpose of residual coding and prediction of the image or video data. As mentioned previously, picture 24 may be used as one or more arrays of image sample values. For example, in the case of the YUV / YCbCr color space, the first arrangement may represent the luma channels, while the other two arrangements represent the channel channels. Their arrangements may have other dimensions. All arrangements may form a group into at least one plane group with each plane group in which there are at least one contiguous plane, so that each plane may be included in one or only one plane group. This applies for each group of planes below. The first arrangement of a particular plane group may be called the primary arrangement of that plane group. Perhaps the following arrays are dependent arrays. The block division of the primary array is best done below It may be performed based on the quadtree technique described. The block division of the dependent arrays may be derived based on the partitioning of the primary array.

According to the embodiments described below, the subdivisions 28 and 104a are configured for dividing the primary arrangement into a plurality of rectangular blocks of the same size, hereinafter called so-called treeblocks. When quadtrees are used, the edge lengths of the treeblocks are usually doubled, such as 16, 32 or 64. However, for the purpose of completeness, it may be possible to use several tree types, such as trees or binary trees with a large number of leaves. In addition, the number of children of the tree can vary depending on the level of the tree and the signal it represents.

In contrast to this, as mentioned above, the arrangement of samples may also represent video sequences and other information, such as depth maps or lightfields, respectively. For simplicity, the following description focuses on a quadtree as a representative example for multi-trees. Quadtrees are trees with exactly 4 children in each internal node. Each treeblock constitutes a primary quadtree with dependent quadtrees in each leaf of the primary quadtree. The primary quadtree determines the subdivision of a given treeblock for prediction, while the dependent quadtree determines the subdivision of a given prediction block for this purpose of residual coding.

The root node of the primary quadtree corresponds to a complete treeblock. For example, FIG. 3A shows treeblock 150. It should be recalled that each picture is divided into regular grids of columns and lines of these treeblocks 150, so equally, for example, an array of samples without gaps. It includes. However, attention should be paid to all block subdivisions shown below, and seamless subdivision without overlap is not important. Rather, neighboring blocks may overlap each other, while at the same time a block without a leaf is a suitable detail of the neighboring leaf block.

According to the quadtree structure of treeblock 150, each node can be further divided into four child nodes, which means that in the case of primary quadtree each treeblock 150 is half the width of treeblock 150. And four detailed blocks having a height of half. In Fig. 3A, their detail blocks are indicated by reference numerals 152a through 152d. In the same way, each of these detail blocks can be further divided into four smaller detail blocks having half the width and half the height of the original detail blocks. In FIG. 3D, this is shown as an example for detail block 152c, which is subdivided into four small detail blocks 154a through 154d. 3A-3C show an example of how the treeblock 150 is first divided into its four sub-blocks 152a-152d, followed by a lower-left. Detail block 152c may be further divided into four small detail blocks 154a through 154d, and finally the upper-right block 154b of these smaller detail blocks is the original treeblock 150. Is divided once more into four blocks of 1/8 width and 1/8 height, and their very small blocks are designated 156a through 156d.

FIG. 4 illustrates the underlying tree structure for quadtree based splitting of the example as shown in FIGS. 3A-3C. The numbers next to the tree nodes are values called subdivision flags, which subdivision flags will be described in more detail later, then the signaling of the quadtree structure will be discussed. The root node of the quadtree is represented above the figure (denoted "level 0"). Four branches at level 1 of this root node correspond to four sub-blocks as shown in FIG. 3A. The detailed blocks of these three quadrants are further subdivided into its four detailed blocks in FIG. 3B, and the third node of level 1 in FIG. 4 has four branches. Again, corresponding to the subdivision of the second (top-right) child node in FIG. 3, there are four detail branches connected to the second node at level 2 of the quadtree hierarchy. The nodes at level 3 are not subdivided further.

Each leaf of the primary quadtree corresponds to a block of various sizes, and individual prediction parameters can be specified for blocks of various sizes (ie, intra or inter, prediction mode, motion parameters, etc.). In the following, these blocks are called prediction blocks. In particular, these leaf blocks are the blocks shown in FIG. 3C. Referring briefly to the description of FIGS. 1 and 2, the divider 20 or subdivision 28 determines the quadtree subdivision just described. The subdivisions 152a through 152d are treeblock 150, subdivision blocks, with the purpose of finding the best tradeoff between the very precise prediction subdivision and the very coarse prediction subdivision as already shown above. Determination of subdivision or further division for 152a through 152d, small detail blocks 154a through 154d, and so forth. Next, the prediction unit 12 may determine, for example, each prediction detail region represented by the aforementioned prediction parameters or blocks shown in FIG. 3 in a granularity according to the prediction subdivision. For this purpose, the specified prediction subdivision is used.

The prediction blocks shown in FIG. 3C may be further divided into smaller blocks for the purpose of residual coding. For each prediction block, ie, for each leaf node of the primary quadtree, the corresponding subdivision is determined by at least one dependent quadtree (s) for residual coding. For example, if a maximum residual block size of 16 × 16 is allowed, a given 32 × 32 prediction block can be divided into four 16 × 16 blocks, with each four 16 × 16 blocks dependent on the residual coding. Determined by the quadtree. In this example, each 16x16 block corresponds to the root node of the dependent quadtree.

As just described for the case of subdividing a given treeblock into prediction blocks, each prediction block can be divided into a number of residual blocks by use of dependent quadtree decomposition (s). Each leaf of the dependent quadtree corresponds to a residual block, and individual residual coding parameters may be specified for the residual block by the residual precoder 14 (ie, transform mode, transform coefficients, etc.), The residual coding parameters control the residual restoring units 16 and 106, respectively.

In other words, the detail divider 28 can be configured to determine the dependent residual subdivision and prediction subdivision by first separating the pictures for each picture or for each group of pictures into a regular arrangement of the treeblock 150. And inductively partition the subset of these treeblocks by quadtree subdivision to obtain prediction subdivision into predictive blocks in a similar manner with subsequent subdivisions of the subset of these prediction blocks. Can be treeblocks if no partitioning is performed in each treeblock, or can be leaf blocks of quadtree subdivision, and if the prediction block is larger than the maximum size of the dependent residual subdivision, obtaining residual blocks. Subsequent subdivision of a subset of these subtree blocks according to the quadtree subdivision procedure for Together, each prediction block is first divided into a regular array of detailed treeblocks, and the residual blocks may be prediction blocks when no partitioning is performed in each prediction block into detailed treeblocks, and the residual blocks are the respective detail blocks. It may be detailed treeblocks when splitting is not performed into smaller regions in the treeblock, or may be leaf blocks of residual quadtree subdivision.

As briefly described above, it may be mapped onto subdivision dependent arrays selected for the primary arrangement. This is easy when considering dependent arrays of the same dimension as the primary array. However, special measurements should be taken when the dimensions of the dependent arrays and the dimensions of the primary array are different. Generally speaking, mapping of primary array subdivisions onto dependent arrays for different dimensions may be performed by spatial mapping (ie, spatial mapping of block boundaries of primary array subdivisions onto dependent arrays). will be. In particular, for each dependent arrangement, there may be scaling factors in the horizontal and vertical directions, the scaling factors determining the ratio of the dimensions of the primary array for the dependent arrangement. The partitioning of the dependent array into sub-blocks for prediction and residual coding consists of the dependent quadtrees of each combined treeblocks of the primary array, together with the generated treeblocks of the dependent array scaled by relative scaling factors. S) and primary quadtree. If the scaling factors in the horizontal and vertical directions are different (eg, 4: 2: 2 chroma detailed sampling), the residual blocks of the dependent arrangement and the generated prediction may no longer be squares. In this case, adaptively choose whether to split the non-square residual block into rectangular blocks (one for the whole sequence, one for the picture or one for each prediction) one) of the picture or the residual block), or in advance. In the first case, for example, the encoder and decoder may grant subdivision into rectangular blocks, and the mapped block is not always rectangular. In the second case, the subdivision unit 28 may signal the subdivision unit 104a for selection via the data stream inserter 18 and the data stream 22. For example, for 4: 2: 2 chroma detailed sampling, dependent arrays may have the same height as the primary array and half the width of the primary array, and the residual blocks may have twice the width. By dividing the block vertically, it can obtain two square blocks again.

As mentioned above, each subdivision 28 or division 20 sends a signal for quadtree based separation to subdivision 104a via data stream 22. Finally, subdivision section 28 informs data stream inserter 18 about the subdivisions selected for pictures 24. Next, the data stream inserter has a structure of primary and secondary quadtrees, and thus, the picture arrangement into blocks of various sizes for residual coding or prediction within the data stream or bit stream 22. The division is sent to the decoding side, respectively.

The minimum and maximum allowed block sizes are transmitted as auxiliary information and may be exchanged between the picture and the picture. Alternatively, the minimum and maximum allowed block sizes may be fixed at the encoder and the decoder. These minimum and maximum block sizes can be different for prediction and residual blocks. The quadtree for signaling of the quadtree structure is traversed, and the quadtree for each node indicates whether the particular node is a leaf node of the quadtree (ie, the corresponding block is no longer subdivided) Specifies whether to branch to four child nodes of (ie, the corresponding block is split into four sub-blocks in half size).

Signaling within one picture performs a treeblock by a treeblock in raster scan order, such as from left to right, from top to bottom, as shown at 140 in FIG. 5A. In addition, this scanning scheme may be different from bottom-right to top-left or as a checkerboard sense. In a preferred embodiment, each treeblock and thus each quadtree is traversed in a depth-first order for signaling of subdivision information.

In a preferred embodiment, subdivision information (i.e. the structure of the tree) as well as predictive data and the like (i.e. the payload associated with the leaf nodes of the tree) are transmitted / processed in depth-first order. Because depth-first traversal has an advantage over breadth-first order, it is performed. In Fig. 5B, the quadtree structure is a, b,... Shown with leaf nodes marked with j. 5A shows the generated block division. If the blocks / leaf nodes are traversed in width-first order, we obtain the following order abjchidefg. However, in the depth-first order, the order is abc... ij. As can be seen from FIG. 5A, in the depth-first order, the left neighboring block and the upper neighboring block are always transmitted / processed before the current block. Therefore, motion vector prediction and context modeling can always use the parameters specified for the left and top neighboring blocks to achieve improved coding performance. For the width-first order, this cannot be the case, for example, because block j is transmitted before blocks e, g and i.

As a result, signaling for each treeblock is performed inductively along the quadtree structure of the primary quadtree, so that a flag for each node is sent, and the flag indicates whether the corresponding block is divided into four sub-blocks. Specifies. If the flag has the value "1" (for "True") then the signaling process is inductively repeated for all four child nodes, i.e., the raster scan order (top left) to the leaf nodes of the primary quadtree. sub-blocks are reached: left, top-right, bottom-left, bottom-right. Note that a leaf node is represented by having a subdivision flag with a value of "0". The node is located on the lowest hierarchical level of the primary quadtree and therefore the subdivision flag is not sent, for the case corresponding to the smallest grant prediction block size. For the example of FIGS. 3A-3C, it first transmits “1” as shown at 190 of FIG. 6A, and the treeblock 150 is split into its four sub-blocks 152a-152d. Specify it. It may then inductively encode the subdivision information of all four sub-blocks 152a through 152d in raster scan order 200. For the first and second sub-blocks 152a, 152b it may transmit “0” and may specify that there is no subdivision (see 202 of FIG. 6A). For the third detail block 152c (bottom left), it can send “1”, specifying that the block has been subdivided (see 204 of FIG. 6A). Now, according to an inductive approach, the four sub-blocks 154a through 154d of the block can be processed. Here, it may transmit “0” for the first 206 and “1” for the second (top right) detail block 208. Now, four blocks of the smallest block size 156a through 156d in FIG. 3 can be processed. In this example, if we have already reached the smallest allowable block size, no more data can be transmitted because no further subdivision is possible. Otherwise, as shown at 210 in FIG. 6A, “0000” may be sent specifying that there are no additional blocks to separate. Thereafter, it may transmit “00” (see 212 in FIG. 6A) for the two lower blocks in FIG. 3B, and finally transmit “0” (see 214) for the lower right block in FIG. 3A. Can be. Thus, the complete binary string representing the quadtree structure may be any one shown in FIG. 6A.

Different background contrasts in the binary string representation of FIG. 6A correspond to different levels in the hierarchy of quadtree based subdivision. Shading 216 represents level 0 (corresponding to the same block size as the original treeblock size), and shading 218 represents level 1 (corresponding to the same block size as half the original treeblock size), contrast 220 represents level 2 (corresponding to a block size equal to 1/4 of the original treeblock size), and contrast 222 represents level 3 (corresponding to block size equal to 1/8 of the original treeblock size). Indicates. All subdivision flags of the same hierarchical level (corresponding to the same block size and same color in the binary string representation example) may be entropy coded using the same probability model and the same, for example, by inserter 18. .

Note that the subdivision information for the case of width-first crossing, shown in FIG. 6B, may be transmitted in different orders.

Similar to the subdivision of each treeblock for the purposes of prediction, the partitioning of each generated prediction block into residual blocks allows to be transmitted in the bitstream. In addition, there may be a maximum and minimum block size for residual coding, which may be transmitted as auxiliary information, which may be exchanged from picture to picture. Alternatively, the maximum and minimum block sizes for residual coding can be fixed at the encoder and decoder. As shown in FIG. 3C, at each leaf node of the primary quadtree, the corresponding prediction block may be divided into residual blocks of maximum allowable size. These blocks are the component root nodes of the dependent quadtree structure for residual coding. For example, if the maximum residual block size for a picture is 64 × 64 and the prediction block is 32 × 32 size, then the entire prediction block will correspond to one dependent (residual) quadtree root node of size 32 × 32. On the other hand, if the maximum residual block size for a picture is 16 × 16, then the 32 × 32 prediction block will consist of four residual quadtree root nodes of size 16 × 16. Within each prediction block, the signaling of the dependent quadtree structure performs the root node by the root node in raster scan order (left to right, top to bottom). Similar to the case of the primary (prediction) quadtree structure, a flag for each node is coded, and the flag specifies whether the particular node is split into its four child nodes. Then, if the flag has the value “1”, the procedure is inductively repeated for all four corresponding child nodes, and the corresponding sub-blocks are in raster scan order (top left, top right, bottom left, bottom right). The leaf nodes of the dependent quadtree are reached. As in the case of the primary quadtree, signaling for nodes on the lowest hierarchical level of the dependent quadtree is not required, because their nodes are blocks of the smallest possible residual block size that can no longer be split. Because they correspond to

For entropy coding, residual block subdivision flags belonging to residual blocks of the same block size may be encoded using it and the same probability model.

Therefore, according to the embodiment presented above with respect to FIGS. 3A-6A, subdivision section 28 is used for dependent subdivision and prediction purposes for blocks of different sizes of primary subdivision for residual coding purposes. Define primary subdivisions. The data stream inserter 18 codes the primary subdivision by signaling for each treeblock in a zigzag scan order, with the bit sequence coding the maximum hierarchical level and the maximum primary block size of the primary subdivision Along with FIG. 6a. As mentioned above, for each defined prediction block, the associated prediction parameters are included in the data stream. In addition, the similar information coding, i.e. the maximum size, the maximum hierarchical level and the bit sequence according to FIG. 6A, is the size for each prediction block that is less than or equal to the maximum size for the residual detail block and each residual tree root block. And the predictive blocks are pre-divided into the size above the maximum size defined for the residual blocks. As mentioned above, for each defined residual block, the residual data is inserted into the data stream.

The extractor 102 extracts each bit sequence from the data stream at the input 116 and informs the divider 104 about the subdivision information obtained as described above. In addition, data stream inserter 18 and extractor 102 may predict block to transmit additional syntax elements such as residual data output by residual precoder 14 and prediction parameters output by predictor 12. And the above mentioned order of residual blocks may be used. Using this order has advantages in that the syntax elements of neighboring blocks already facilitate coding / decoding so that sufficient contexts for encoding of individual syntax elements for a particular block can be selected. In addition, similarly, residual precoder 14 and predictor 12 as well as residual reconstructor 106 and precoder 110 may process individual prediction and residual blocks in the order described above.

7 shows a flow chart of the steps, which may be performed by the extractor 102 to extract the subdivision information from the data stream 22, which is encoded in this manner as described above. . In a first step, the extractor 102 divides the picture 24 into tree root blocks 150. This step is designated as step 300 of FIG. Step 300 may involve an extractor 102 that extracts the maximum prediction block size from the data stream 22. In addition or instead, step 300 may involve an extractor 102 that extracts the maximum hierarchical level from the data stream 22.

Next, in step 302, the extractor 102 decodes a flag or bit from the data stream. The first step 302 is performed, and the extraction unit 102 recognizes that each flag is the first flag of the bit sequence belonging to the first tree root block 150 in the tree root block scan order 140. As this flag is a hierarchical level 0 flag, the extraction unit 102 may use context modeling associated with layer 0 in step 302 to determine the context. Each context may have a respective probability estimate for entropy decoding of the flag associated with it. Probability estimates of contexts may be contexts and are individually adapted to each context symbol statistic. For example, to determine an appropriate context for decoding of the hierarchical level 0 flag in step 302, the extractor 102 may select one context of a set of contexts, which is a neighboring tree. Depends on the hierarchical level 0 flag of the blocks, or, in addition, information located within the bit strings that define the quadtree subdivision of neighboring treeblocks of the currently processed treeblock, such as the top and left neighboring treeblocks. Associated with hierarchical level 0.

In a next step, namely step 304, the extractor 102 checks as to whether the recently decoded flag implies partitioning. If this is the case, the extractor 102 splits the current block-now the treeblock-or informs the subdividing unit 104a of the splitting in step 306 and the current hierarchical level is 'maximum hierarchy' in step 308. Check for equality to enemy level minus-1 '. For example, the extractor 102 may also have a maximum hierarchical level extracted from the data stream, for example, at 300. If the current hierarchical level is different from the 'maximum hierarchical level minus (minus) -1', the extractor 120 increases the current hierarchical level by 1 in step 310, and for decoding the next flag from the data stream. Steps are returned to step 302. These times, the flags are decoded in step 302 and belong to different hierarchical levels, and accordingly, according to an embodiment, the extractor 102 may select one of the other sets of contexts, the set at the current hierarchical level. Belongs. The selection may also be the basis of subdivided bit sequences according to FIG. 6 of neighboring treeblocks that have already been decoded.

If the flag is decoded, and if the confirmation in step 304 indicates that the flag does not imply partitioning of the current block, then the extraction section 102 performs step 312 to ascertain whether the current hierarchical level is zero. Continue. In this case, the extraction unit 102 proceeds the processing on the next tree root block in the scanning order 140 in step 314, or stops processing to extract the subdivided information when the processed tree root block is not on the left side. do.

It should be noted that the description of FIG. 7 focuses only on the decoding of subdivision indication flags of the prediction subdivision, so, in fact, step 314 may be for example of additional bins or syntax elements related to the current treeblock. Involves decoding. In any case, if there is an additional or next tree block, the extraction section 102 starts from step 314 for decoding the next flag from the first flag of the subdivision information, ie, the flag sequence associated with the new tree root block. Proceed to step 302.

If it is found in step 312 that the hierarchical level is not equal to zero, the operation proceeds to step 316 with confirmation as to whether there are additional child nodes associated with the current node. That is, when the extraction unit 102 performs confirmation in step 316, it has already been confirmed in step 312 that the current hierarchical level is a hierarchical level other than the 0 hierarchical level. Next, it means that there is a parent node, which belongs to the tree root block 150 or one of the smaller blocks 152a through 152d, the smaller blocks 152a through 152d, and so on. The recently decoded flag belongs to a node of the tree structure, the node of the tree structure having a parent node, which is common with the three additional nodes of the current tree structure. The scan order of child nodes having a common parent node has been illustrated by way of example in FIG. 3A for hierarchical level 0 with reference numeral 200. Therefore, in step 316 the extractor 102 checks whether all four of the child nodes have already stayed in the process of FIG. If this is not the case, i.e., if there are additional child nodes with the current parent node, the process of FIG. 7 proceeds to step 318, where the next according to the zigzag scan order 200 within the current hierarchical level. The child node will stay, so corresponding to the detail block it now represents the current block of the process 7, after which the flag is decoded from the data block for the current block or the current node in step 302. However, if there are no additional child nodes for the current parent node in step 316, the process of FIG. 7 proceeds to step 320, where the current hierarchical level is reduced by one and post-process step 312. Continue

By performing the steps shown in FIG. 7, the extractor 102 and subdivision partition 104a cooperate to retrieve the subdivision selected at the encoder side from the data stream. The process of FIG. 7 concentrates on the case of prediction subdivision described above. 8 together with the flowchart of FIG. 7 illustrates how the extractor 102 and subdivision 104a cooperate to retrieve the residual subdivision from the data stream.

In particular, FIG. 8 shows the above steps performed by each of the extraction section 102 and the subdivision section 104a for each prediction block generated from the prediction subdivision. As mentioned above, these prediction blocks have recently been in accordance with the zigzag scan order 140 of the treeblocks 150 of the prediction subdivision and, for example, to traverse the leaf blocks as shown in FIG. 3C. (currently) within each stayed treeblock 150 can be traversed using a depth-first traversal order. According to the depth-first traversal order, the leaf blocks of the divided primary treeblocks are in zigzag scan order 200 with the zigzag scan order 200 and with visiting specific blocks of a particular hierarchical level having a common current node. The first sub-blocks stay in depth-first traversal order with primary scanning of the subdivisions of each of these sub-blocks.

For example, in FIG. 3C, the scan order generated among the leaf nodes of treeblock 150 is shown at 350.

For the currently-visited prediction block, the process of FIG. 8 begins at 400. In step 400, an internal parameter indicating the current size of the current block is set equal to the size of the hierarchical level 0 of the residual subdivision, that is, the maximum block size of the residual subdivision. It should be recalled that the maximum residual block size may be smaller than the smallest block size of the prediction subdivision, or may be greater than or equal to the latter. In other words, according to the embodiment, the encoder is free to choose any of the possibilities just mentioned.

In a next step, step 402, the check is performed as to whether the predicted block size of the currently staying block is greater than an internal parameter indicating the current size. In this case, the currently remaining prediction block may be a leaf block of the prediction subdivision or a tree block of the prediction subdivision, wherein the current remaining prediction block is no longer split, and the current remaining prediction block is larger than the maximum residual block size, In this case, the process of FIG. 8 continues to step 300 of FIG. 7. That is, the currently remaining prediction block is divided into residual tree root blocks, and within the currently remaining prediction block, the first flag of the flag sequence of the first residual tree block is decoded at step 302 or the like.

However, if the currently remaining prediction block is less than or equal to an internal parameter representing the current size, the process of FIG. 8 proceeds to step 404, where the prediction block size is equal to an internal parameter representing the current size. Confirmed to determine whether or not. In such a case, separation step 300 may be skipped and the process continues directly to step 302 of FIG. 7.

However, if the prediction block size of the currently staying prediction block is smaller than an internal parameter representing the current size, the process of FIG. 8 continues to step 406, where the hierarchical level is increased by one, and the current The size is set to the size of the new hierarchical level as separated by two (in the directions of both axes in the case of quadtree subdivision). Thereafter, verification of step 404 is performed again. The effect of the loop formed by steps 404 and 406 is that the hierarchical level always corresponds to a size corresponding to blocks partitioned independently from each prediction block that is less than or equal to / greater than the maximum residual block size. Therefore, when decoding the flag in step 302, context modeling is performed depending on the hierarchical level and the size of the block, which the flag simultaneously represents. The use of different contexts for flags of different hierarchical levels or block sizes has the advantage that the probability estimation can fit well with the actual probability distribution during flag value generation, on the other hand, the relative ordinary of the contexts to be managed. Having the number, thereby not only reduces the context of managing overhead, but also increases context adaptation with actual symbol statistics.

As already mentioned above, there may be more than one arrangement of samples, and these arrangements of samples may be classified into one or more plane groups. The input signal, which is encoded and enters the input 32, may be, for example, one picture of a video sequence or a still image. Therefore, the picture may be given in the form of one or more sample arrangements. In the context of coding a picture of a video sequence or still image, the sample arrangements may refer to luma and chroma planes or three color planes such as red, green and blue in the color representations of YUV or YCbCr. In addition, sample arrangements representing alpha, i.e. transparency and / or depth information for 3-D video material, may also be represented. Many of these sample arrangements can be grouped together as what is called a planar group. For example, luma (Y) can be one plane group with only one sample array, and like CbCr, chroma can be another plane group with two sample arrays, or in another example. , YUV can be one plane group with three matrices, and depth information for 3-D video material can be another plane group with only one sample array. For every plane group, one primary quadtree structure is within the data stream 22 to indicate the division into prediction blocks and the next quadtree structure for each prediction block to indicate the division into residual blocks. May be coded. Therefore, according to the first example just mentioned, the luma component there is one plane group, while the chroma component forms another plane group, one quadtree structure for the prediction blocks of the luma plane, the luma plane There may be one quadtree structure for the residual blocks of, one quadtree structure for the prediction block of the chroma plane, and one quadtree structure for the residual blocks of the chroma plane. However, in the second example mentioned above, one quadtree structure for prediction blocks of both luma and chroma (YUV), one quadtree structure, 3-D video for residual blocks of both luma and chroma (YUV) There will be one quadtree structure for the prediction blocks of depth information for material and one quadtree structure for the residual blocks of depth information for 3-D video material.

Additionally, in the foregoing description, a method is described in which the input signal is separated into prediction blocks using a primary quadtree structure, and these prediction blocks are further subdivided into residual blocks using a dependent quadtree structure. According to an alternative embodiment, subdivision may not be terminated in the dependent quadtree phase. That is, blocks obtained from partitioning using the dependent quadtree structure may be further subdivided using the third quadtree structure. In the following, this partitioning may be used for the purpose of using additional coding tools, which may enable encoding of the residual signal.

The foregoing description focuses on the subdivision performed by each of the subdividing unit 28 and the subdividing unit 104a. As mentioned above, the subdivision defined by each of the subdivisions 28 and 104a may control the granularity processing of the modules for the encoder 10 and decoder 100 mentioned above. . However, according to the embodiment described below, each of the subdividing sections 28 and 104a is followed by each of the merging section 30 and the merging section 104a. However, merging portions 30 and 104b are optional and may be omitted.

In practice, however, and as will be described in more detail below, the merger provides an encoder for the opportunity to combine several prediction blocks or residual blocks into groups or clusters, so that other Or at least some other modules may process with these groups of blocks. For example, the prediction unit 12 may indicate that the signaling of the grouping of the prediction blocks together with the common parameter transmission for all the blocks belonging to the group has a rate / distortion ratio sense rather than the individual signaling of the prediction parameters for all these prediction blocks. If more promising, then use the prediction parameters in place of all these prediction blocks in common, and between the prediction parameters of some prediction blocks as determined by optimizing using the subdivision of subdivision 28 You can sacrifice small deviations. Based on these common prediction parameters, processing for recovery of the prediction in the prediction units 12 and 110 may still perform the prediction block. However, it is also possible for the predictors 12 and 110 to even perform the prediction process once for every group of prediction blocks.

As will be described in more detail below, grouping of prediction blocks not only uses common prediction parameters or the same for a group of prediction blocks, but alternatively or in addition, prediction residuals for prediction blocks belonging to that group. Is to activate the encoder 10 to send one prediction parameter for that group, so the signaling overhead for signaling of the prediction parameters for that group may be reduced. In the latter case, the merging process may only affect the data stream inserter 18 rather than the determination made by the residual precoder 14 and the predictor 12. However, a more detailed description is given below. However, for completeness, it should be noted that the aspect just mentioned also applies to other subdivisions, such as the filter subdivision or residual subdivision mentioned above.

First, the merging of sets of samples, such as the prediction and residual blocks mentioned above, will result from a more general sense, i.e., not limiting the multi-tree subdivision mentioned above. However, the following description focuses on the merging of blocks resulting from multi-tree subdivision, and embodiments for multi-tree subdivision have already been described above.

Generally speaking, the merging of the distinguishing elements associated with specific sets of samples for the purpose of transmission associated with the coding parameters enables the reduction of the auxiliary information rate in image and video coding applications. For example, the sample arrangements of signals to be encoded are usually divided into specific sets or sample sets of samples, which are rectangular or square blocks, or of randomly formed regions, triangles or other shapes. It could represent some other vowel. In the embodiments described above, the simple connection regions are residual blocks and prediction blocks resulting from multi-tree subdivision. The subdivision of the sample arrangements may be fixed by the syntax as described above, or the subdivision may be signaled at least in part within the bit stream. In order to maintain an auxiliary information rate for small subdivision information signaling, the usual syntax only allows a limited number of choices that arise within a simple partition, such as subdividing blocks into smaller blocks. Sample sets are associated with specific coding parameters, which may specify prediction information or residual coding modes, and the like. Descriptions of these issues have been described above. For each sample set, separate coding parameters, such as to specify prediction and / or residual coding, may be sent. In order to achieve improved coding efficiency, the aspect of merging described below, that is, merging two or more sample sets into groups of sample sets will enable several advantages, which are described further below. . For example, the sample sets may be merged such that all sample sets of a group share the same coding parameters, which may be sent with one of the sample sets in the group. By doing so, the coding parameters are not transmitted individually for each sample set of the group of sample sets, but instead, the coding parameters are sent only once for the entire group of sample sets. As a result, the auxiliary information rate for transmission of coding parameters can be reduced, and overall coding efficiency can be improved. As an alternative approach, further refinements for one or more coding parameters may be sent for the sample sets of the group for one or more sample sets. The improvement may be applied to all sample sets of the group, or only to the sample set for which it is to be sent.

The merging aspect, described further below, also provides the encoder with more freedom in the generation of the bit stream 22, because the merging technique significantly increases the number of possibilities for selecting a split for sample arrangements of a picture. Because it is. The encoder can choose between additional options such as specific rate / distortion measurements for minimization, thereby improving coding efficiency. There are several possibilities of encoder operation. In a simple approach, the encoder can preferentially determine the best subdivision of the sample arrays. Referring briefly to FIG. 1, subdivision unit 28 may determine an optional subdivision in the first step. Later, for each sample set, it may be identified whether the merging of another group of sample sets or another sample set reduces a particular rate / distortion cost measure. Thus, prediction parameters associated with the merged group of sample sets may be reestimated as by performing new motion search or prediction parameters, which It may have already been determined for a common sample set, and the group of sample sets or candidate sample set for merging may be evaluated for the considered group of sample sets. In a broader approach, specific rate / distortion cost measurements can be evaluated for additional candidate groups of sample sets.

It should be noted that the merging technique described below does not change the processing order of sample sets. That is, the merging concept can be implemented in such a way that the delay does not increase, i.e., each sample set remains decodable at the same time as not using the merging technique.

For example, if the bit rate stored by reducing the number of coded prediction parameters is greater than the bit rate additionally consumed for coded merge information to indicate that it is merged to the decoding side, the merge technique described further below Resulting in increased coding efficiency. The syntax extension described for merging gives the encoder additional freedom in the choice of dividing a picture or plane group into blocks, which will be discussed further below. In other words, the encoder does not limit the subdivision first and then it checks to see if any result blocks are going to the same or the same set of prediction parameters. As one simple alternative, the encoder may first determine the subdivision according to the rate-distortion cost measure, and then the encoder may combine one of its neighboring blocks for each block or an associated already determined group of blocks. Determine whether to reduce distortion cost measurements. Thus, prediction parameters associated with a new group of blocks can be reestimated as by performing new motion search or prediction parameters, which has already been determined for the current block and neighboring blocks, or groups of blocks Can be evaluated for new groups. The merging information may be signaled on the base of the block. Effectively, merging can also be described as inference of prediction parameters for the current block, where the inferred prediction parameters are the same set as the prediction parameters of one of the neighboring blocks. Alternatively, the residuals may be sent for blocks within a group of blocks.

Therefore, the fundamental basic idea of the merging concept described further below is to reduce the bit rate, which is required to transmit other coding parameters or prediction parameters to the group of blocks by merging of neighboring blocks, Each group of blocks there is associated with a unique set of coding parameters, such as prediction parameters or residual coding parameters. The merge information, if present, is signaled in the subdivision information as well as the bit stream. An advantage of the merging concept is the increased coding efficiency resulting from the reduced auxiliary information rate for the coding parameters. It should be noted that the merging processes described herein can also be extended to other dimensions besides spatial dimensions. For example, a group of sets for blocks or samples, each located within several different video pictures, may be merged into one group of blocks. Merging can also be applied to 4-D compression or light-field coding.

Therefore, briefly returning to the previous description of FIGS. 1 to 8, the merging process following subdivision has the advantage of being independent from the particular method of subdivisions 28 and 104a for subdividing pictures respectively. More precisely, the latter is provided, for example, for macroblocks, as definition for partitioning for residual and corresponding residual transform granularity, as well as granularity for prediction and corresponding prediction parameters in the data stream. Signaling in a data block, i.e., within a data stream, with a macroblock with certain coding parameters associated with including partitioning parameters defining partitioning into 1, 2, 4 or some other number of regular sub-grids. By dividing each picture into a regular array of rectangular or square macroblocks of a predetermined size, such as a predetermined size or 16 × 16 luma samples, it is possible to subdivide the pictures in a manner similar to H.264.

In some cases, merging provides the advantages discussed above and briefly discussed, such as a reduction in auxiliary information rate in image and video coding applications. Certain sets of samples may represent rectangular or square blocks or arbitrary formed regions or some other set of samples, such as any simply connected region or samples, which particular sets of samples are usually a specific set of coding parameters and angles. Linked for sample sets, the coding parameters are included in the bit stream, for example, the coding parameters indicative of the prediction parameters specify how the corresponding set of samples is predicted using already coded samples. The division of the sample arrays of a picture into a sample set may be fixed by syntax or may be signaled by corresponding subdivision information in the bit stream. Coding parameters for a sample set may be sent in a predefined order, which is given by syntax. Depending on the functionality of the merge, the merger 30 may signal for a common set of current blocks or samples, such as a predictive block or residual block, so that it is merged into one or more other sample sets and a group of sample sets. . Thus, the coding parameters for a group of sample sets only need to be transmitted once. In a particular embodiment, if the current sample set is combined with an existing group or sample set of sample sets for which the coding parameter has already been transmitted, the coding parameters of the current sample set are not transmitted. Instead, the coding parameters for the current set of samples are the same set as the group of sample sets or coding parameters of the sample set, and the current set of samples is merged with the sample set or group of sample sets. As an alternative approach, further refinements for one or more coding parameters may be sent for the current sample set. The improvement may be applied to all sample sets in the group, or only to the sample set, which is sent for the sample set.

According to an embodiment, for each set of samples, such as the prediction block as mentioned above, the residual block as mentioned above, or the leaf blocks of the multitree subdivision as mentioned above, the pre-coded The set of decoded sample sets is called the "set of causal sample sets". For example, referring to FIG. 3C, all blocks shown in the figure are the result of a particular subdivision, such as prediction subdivision or residual subdivision or some multi-tree subdivision, or the like, of which The coding / decoding order that is defined is defined by arrow 350. Considering a particular block of these blocks as the current sample set or currently simply concatenated region, its set of causal sample sets is created by all blocks preceding the current block in order 350. However, it is recalled that another subdivision that does not use multi-tree subdivision is possible as well as interested in further discussion of the merging principles above.

The sets of samples that can be used for merging with the current set of samples are referred to below as the "set of candidate sample sets", and the set of samples is always a subset of the "set of causal sample sets". How the subset is formed is known to the decoder, or it can be specified in the data stream or bit stream from the encoder to the decoder. If a particular current set of samples is coded / decoded and its set of candidate sample sets is not empty, then whether a common set of samples is merged with one sample set from that set of candidate sample sets, if they With any of, it is signaled to the data stream at the encoder or extracted from the data stream at the decoder. Otherwise, merging will not be available for that block, because the set of candidate sample sets is empty anyway.

There are different ways to determine a subset of a set of causal sample sets, which can represent a set of candidate sample sets. For example, the determination of candidate sample sets may be based on a sample in the current set of samples, which is geometrically defined in a unique form, such as an upper-left image sample of a rectangular or square block. Starting from a sample that is geometrically defined in its distinctive form, it is defined as a specific nonzero number of samples, which represents the direct spatial neighbors of the sample that are geometrically defined in that distinctive form. For example, that particular non-zero number of samples includes the upper or left neighbor of the sample that is geometrically defined in the distinctive form of the current set of samples, so that the non-zero number of neighboring samples is at most Low 2, 1 if one of the top or left neighbors is not available or located outside of the picture, or 0 if both neighbors are absent.

The set of candidate sample sets can then determine to include those sample sets, the sample sets comprising at least one of the non-zero numbers of neighboring samples just mentioned. See, eg, FIG. 9A. The set of samples currently considered as a merge object will be block X, and the sample defined in its geometrically distinct form will be an example of the top-left sample, denoted by 400. The upper left neighboring samples of sample 400 are indicated by 402 and 404. The set of causal sample sets or the set of causal blocks is highlighted in the manner formed. Of these blocks, blocks A and B include one of neighboring samples 402 and 404, and thus these blocks form a set of candidate blocks or a set of candidate sample sets.

According to another embodiment, the set of candidate sample sets determined for the purpose of merging may additionally or exclusively comprise a set of samples, the set of samples comprising a particular nonzero number of samples, which is the same spatial It may be 1 or 2 with position, but it is included in different pictures, ie, in a previous coded / decoded picture. For example, in addition to blocks A and B of FIG. 9A, a block of a previously coded picture can be used, which includes a sample at the same location as sample 400. By this method, it is noted that only the top neighboring sample 404 or only the left neighboring sample 402 can be used to define the non-zero number of neighboring samples mentioned above. In general, the set of candidate sample sets may be derived from data previously processed within the current picture or other pictures. The derivation may include spatial direction information, such as image gradient of the current picture and transform coefficients associated with a particular direction, or it may include temporal direction information, such as neighboring motion representations. If present, the set of candidate sample sets may be derived from any data available at the receiver / decoder and auxiliary information and other data in the data stream.

Note that the derivation of candidate sample sets is performed in parallel by both the merger 30 on the encoder side and the merger 104b on the decoder side. As just mentioned, the merger 30, 104b may determine a set of candidate sample sets independent from each other based on a predefined method known by both, or the encoder may signal for implicit in the bit stream. In this case, this gives the merger 104b a position for performing derivation of these candidate sample sets in the same way as the merger 30 at the encoder side where the subset of candidate sample sets is determined.

As will be described in more detail below, merger 30 and data stream inserter 18 cooperate to transmit one or more syntax elements for each set of samples, which merges the set of samples with another sample set. In the following, it may be part of an already merged group of sample sets, and any of the set of candidate sample sets is used for merging. In the following, the extractor 102 extracts these syntax elements and informs the merger 104b accordingly. In particular, according to a specific embodiment described later, one or two syntax elements are sent to specify merge information for a particular set of samples. The first syntax element specifies whether the current set of samples is merged with another sample set. If the first syntax element specifies that the current set of samples is to be merged with another set of samples, the second syntax element is transmitted only, and the second syntax element is any of the sets of candidate sample sets used for merging. Specify that. The transmission of the first syntax element may be suppressed if the derived set of candidate sample sets is empty. That is, the first syntax element may only be sent if the derived set of candidate sample sets is not empty. If the derived set of candidate sample sets contains more than one sample set, then the second syntax element may only be transmitted, because if only one sample set is included in the set of candidate sample sets, In addition, no additional choices are possible. In addition, the transmission of the second syntax element will be suppressed if the set of candidate sample sets contains more than one sample set, with the exception that all sample sets for the set of candidate sample sets are associated with the same coding parameter. Will be. That is, the second syntax element may only be transmitted if at least two sample sets for the derived set of candidate sample sets are associated with different coding parameters.

Within the bit stream, the merge information for the set of samples may be coded before other specific coding parameters or prediction parameters associated with its sample set. If the merge information signals that the current set of samples is not merged with any other set of samples, the prediction or coding parameters may only be transmitted.

For example, the merge information for a particular set of samples, ie, a block, may be coded after a true subset of prediction parameters, or in a more general sense, and associated with each sample set. The coding parameters that have been transmitted have been transmitted. The subset of prediction / coding parameters may consist of one or more components for one or more reference picture indexes or motion parameter vector or one or more components for reference index and motion parameter vector, and the like. The already transmitted subset of prediction or coding parameters will be used to obtain a set of candidate sample sets that deviates from the large temporary set of candidate sample sets, which may have been obtained as just mentioned above. By way of example, a different size or distance may be calculated according to a predetermined distance measurement between coding parameters of a current set of prediction and samples already coded and coding parameters of a primary set of corresponding prediction or candidate sample sets. Then only their sample sets for the different size, or distance, calculated may be equal to or less than a predefined or obtained threshold, which is finally included, i.e., as a reduced set of candidate sample sets. See, eg, FIG. 9A. The current set of samples will be block X. The subset of coding parameters for that block has already been inserted into the data stream 22. For example, think of block X as a predictive block, and in the case of a suitable subset of coding parameters, such as a subset outside of a set that includes picture reference indices and motion-mapping information such as motion vectors, It may be a subset of the prediction parameters for that block X in the prediction block. If block X was a residual block, the subset of coding parameters is a subset of residual information, such as a map or transform coefficients, indicating the positions of significant transform coefficients within block X. Based on this information, both data stream inserter 18 and extractor 102 enable the use of this information to determine a subset that deviates from blocks A and B, which is a specific embodiment. In, form a primary set of previously mentioned candidate sample sets. In particular, because of blocks A and B belonging to the set of causal sample sets, its coding parameters are available at both the encoder and the decoder at that time, and the coding parameters of block X are currently coded / decoded. Thus, the previously mentioned comparisons using different sizes may be used to exclude any number of blocks of the primary set of candidate sample sets A and B. The resulting-reduced set of candidate sample sets may then be used as described above, i.e. whether a merge indication indicative of merging is transmitted within the reduced portion of candidate sample portions or candidate sample. As to whether it is extracted from the data stream depending on the number of sample sets in the reduced portion of the portions, and whether a second syntax element should be transmitted in the reduced portion of the candidate sample portions or reduced of the candidate sample portions. It may be used to determine whether the sample parts within the part should be extracted from the data stream having a second syntax element indicating that it is to be a partner block for merging. That is, the merging determination or transmission of the individual merge syntax elements for the predetermined simple connection area has a predetermined relative positional relationship to the predetermined simple connection area and at the same time, the first of the coding parameters for the predetermined simple connection area. Can depend on the number of simple connection regions having coding parameters associated therewith that satisfy a predetermined relationship to the subset, and the adoption or prediction using prediction residual extraction is based on the second subset of coding parameters for the predetermined simple connection region. It can be performed in the. That is, a simple having a predetermined relative positional relationship with respect to the predetermined simple connection area and at the same time having coding parameters associated therewith that satisfy the predetermined relationship to the first subset of coding parameters for the predetermined simple connection area. One or a subset of the identified one coding parameters of the number of connection regions is only adopted from the second subset of the predetermined simple connection regions, respectively, or used to predict the second subset of the predetermined simple connection regions. Can be.

The aforementioned distances are compared against the aforementioned threshold, which may be fixed, and known to both the encoder and decoder, or in the middle of different values, or some other central tendency. Or based on the calculated distances as such. In this case, the reduced set of candidate sample sets will inevitably be a true subset of the primary set of candidate sample sets. Alternatively, only those sets for the samples are chosen out of the primary set of candidate sample sets, for which the distance according to the distance measurement is minimized. Alternatively, the exact one set of samples is selected outside of the primary set of candidate sample sets using the aforementioned distance measure. In the latter case, the merge information will only need to specify whether or not the current set of samples should be merged with one candidate set for the samples.

Therefore, the set of candidate blocks may be formed or may be obtained, as described below with respect to FIG. 9A. Starting from the top-left sample position 400 for the current block X of FIG. 9A, its left neighboring sample 402 and its upper neighboring sample 404 are at its encoder and decoder side. Obtained. Therefore, the set of candidate blocks can have only two elements, that is, those blocks that deviate from the shaded portion of the causal blocks of FIG. 9A include one of two sample positions, and in the case of FIG. 9A, Their block is blocks A and B. Therefore, the set of candidate blocks may only have two directly neighboring blocks for its top-left sample position as its elements. According to another embodiment, the set of candidate blocks may be given by all blocks that have been coded before the current block, and include one or more samples that represent neighbors of direct space for any sample of the current block. Neighbors of direct space may be limited to direct left neighbors and / or direct upper neighbors and / or direct right neighbors and / or direct lower neighbors for any sample of the current block. For example, reference is made to FIG. 9B showing another block subdivision. In this case, the candidate blocks include four blocks, that is, blocks A, B, C and D.

Alternatively, the set of candidate blocks may, in addition or exclusively, include blocks comprising one or more samples located at the same location, such as some samples of the current block, but in other ways (ie, already Coded / decoded pictures).

Further alternatively, the candidate set of blocks represents a subset of the sets of blocks described above, which are determined by neighbors in the spatial or temporal direction. The subset of candidate blocks may be fixed, signaled or obtained. Derivation for a subset of candidate blocks may take into account decisions made for other blocks within the picture or other pictures. By way of example, blocks that are associated with the same or very similar coding parameters than other candidate blocks may not be included in the candidate set of blocks.

The following description of an embodiment applies for the case where only two blocks containing the left and top neighboring samples for the top-left sample of the current block are considered as the maximum potential candidates.

If the set of candidate blocks is not empty, one flag called the merge flag is signaled to specify which candidate blocks the current block is merged with. If the merge flag is equal to zero (for "false"), the block is not merged with one of its candidate blocks, and all coding parameters are transmitted normally. If the merge flag is equal to 1 (for "true"), the following applies. If the set of candidate blocks includes one and only one block, the candidate block is used for merging. Otherwise, the set of candidate blocks contains exactly two blocks. If the prediction parameters for these two blocks are the same, these prediction parameters are used for the current block. Otherwise (if the two blocks have different prediction parameters), a flag called merge_left_flag is signaled. If merge_left_flag is equal to 1 (for “true”), the block containing the left neighboring sample position relative to the top-left sample position of the current block is selected out of the set of candidate blocks. If merge_left_flag is equal to 0 (for "false"), the other (ie, upper neighboring) block is selected out of the set of candidate blocks. The prediction parameters of the selected block are used for the current block.

Summarizing some embodiments described above with respect to merging, reference is made to FIG. 10 showing the steps performed by extractor 102 to extract merging information from data stream 22 entering input 116. Is made.

The process begins at 450 with identifying sample sets or candidate blocks for the current sample set or block. The coding parameters for the blocks should be recalled to be transmitted within the data stream 22 in a particular one dimensional order, so that FIG. 10 refers to the process of retrieving the merge information for a sample set or block that currently resides.

As mentioned previously, the verification and step 450 may include verification of previously decoded blocks, ie, the causal set of blocks, based on neighboring aspects. For example, their neighboring blocks may be designated as candidates, which includes certain neighboring samples neighboring one or more geometrically predetermined samples for the current block X in space or time. Additionally, the step for verification may comprise two steps, i.e., as just mentioned, i.e., the data stream before the first step and step 450 related to the verification, based on neighbors, which result in a primary set of candidate blocks. Only those blocks are designated as candidates according to the already transmitted coding parameters for processing a particular relationship for the true subset of coding parameters for the current block X that have already been decoded from A second step is included.

Next, in step 452 of the process steps, it is determined as to whether the number of candidate blocks is greater than zero. If this is the case, merge_flag is extracted from the data stream at step 454. Step 454 for extraction may involve entropy decoding. In step 454 the context for entropy decoding of merge_flag may be determined based on, for example, syntax elements belonging to the set of candidate blocks or to the primary set of candidate blocks, where the dependencies of the syntax elements are of interest. The blocks belonging to the set are limited to information about whether they have been merged or not. Probability estimation for the selected context may be adapted.

However, if the number of candidate blocks is determined to be zero instead of 452, the process of FIG. 10 continues to step 456, where the coding parameters of the current block are extracted from the bit stream, or as mentioned above. As an alternative in the two step verification case, the coding parameters left after the extractor 102 continue to process for the next block in block scan order, such as the order 350 shown in FIG. 3C.

Returning to step 454, the process proceeds to step 458 after the extraction in step 454 with confirmation as to whether the extracted merge_flag implies the occurrence or absence of a merge for the current block. If no merge has occurred, the process proceeds to step 456 mentioned above. Otherwise, proceed to step 460, including checking as to whether the number of candidate blocks is equal to it. In this case, transmission of an indication for a particular candidate block of the candidate blocks is not necessary, so the process of FIG. 10 proceeds to step 462 as the merging partner of the current block is set to only candidate blocks, where The coding parameters of the merged partner block after step 464 are used for adaptation or prediction of the remaining coding parameters or coding parameters of the current block. For adaptation, the absence of coding parameters for the current block is only copied from the merge partner block. In other cases, i.e. for prediction, step 464 may additionally involve the extraction of residual data from the data stream, combining their missing coding parameters obtained from the merge partner block with the residual data and lacking the current block. Residual data relating to prediction residuals of the specified coding parameters.

However, if the number of candidate blocks in step 460 If determined to be greater than that, the process of FIG. 10 proceeds to step 466 where it relates to the portion of the coding parameters or to the coding parameters-i.e. the portion that has not yet been migrated within the data stream for the current block. Its subparts are checked to see if they are equal to each other. If this is the case, their common coding parameters are set up like a merge reference, or candidate blocks are set up like merge partners in step 468, and each interesting coding parameters are set up for adaptation or prediction in step 464. Used.

It may be noted that the merging partners may have been the block itself, and that the merge was signaled for the block. In this case, the merge parameter for the coding parameters adapted or predicted in step 464 is used.

Otherwise, ie, if the coding parameters are not the same, the process of FIG. 10 proceeds to step 470 where additional syntax elements are extracted from the data stream (ie, its merge_left_flag). A separate set of contexts may be used for entropy-decoding of that flag. The set of contexts used for entropy decoding of merge_left_flag may also include only one context. After step 480, the candidate block indicated by merge_left_flag is set as a merge partner in step 472, and used for adaptation or prediction in step 464. After step 464, the extraction unit 102 proceeds to the processing for the next block in block order.

Of course, there are many alternatives. For example, a combined syntax element may be sent in the data stream instead of the merge_left_flag and split syntax elements merge_flag described before the combined syntax elements signaling the merge process. In addition, the aforementioned merge_left_flag may be transmitted in the data stream with or without two rear blocks having the same prediction parameters, thereby reducing the computational overhead for performing the process of FIG. 10.

For example, as already mentioned with respect to FIG. 9B, more than two blocks may be included as a set of candidate blocks. Additionally the merge information (ie, information signaling whether the block is merged, and if the block is merged, it is merged with the candidate block) may be signaled by one or more syntax elements. The first syntax element may specify whether the block is merged with which candidate blocks, such as merge_flag described above. If the set of candidate blocks is not empty, the flag may only be sent. The second syntax element may signal for any of the candidate blocks used for merging, such as merge_left_flag mentioned above, but generally indicates a selection from at least two or more than two candidate blocks. If the first distinguishing element signals that the current block is merged with one of the candidate blocks, the second syntax element may only be transmitted. If the set of candidate blocks includes one or more candidate blocks and / or if any of the candidate blocks have prediction parameters different from any of the other candidate blocks, the second syntax element may additionally be transmitted only. . The syntax may depend on how many candidate blocks are given and / or how other prediction parameters are associated with the candidate blocks.

The syntax for signaling of any of the candidate blocks used may be set simultaneously and / or in parallel at the encoder and decoder side. For example, if there are three choices for the candidate blocks identified in step 450, for example, in step 470, the syntax is selected such that only these three choices are considered and available for entropy coding. That is, the syntax element is selected so that its symbol alphabet only has as many elements as the selections of the candidate blocks present. The possibilities of all other choices may be considered zero, and entropy-coding / decoding may be adjusted at the encoder and decoder at the same time.

Additionally, as already mentioned with respect to step 464, the prediction parameters inferred as a result of the merging process may represent a complete set of prediction parameters associated with the current blocks, or they may be for one hypothesis of the block. It may be possible to represent a subset of these prediction parameters, such as prediction parameters, and multi-home prediction is used for this.

As mentioned above, syntax elements related to merge information may be entropy-coded using contextual modeling. The context elements may be composed of similar syntax elements or merge_left_flag and merge_flag described above. In a specific example, deviating from the contexts or the three context models can be used for coding / decoding of merge_flag, for example, at step 454. The context model index merge_flag_ctx used may be obtained as follows. If the set of candidate blocks includes two elements, the value of merge_flag_ctx is equal to the sum of the values of merge_flag for the two candidate blocks. However, if the set of candidate blocks contains one element, the value of merge_flag_ctx may be equal to twice the value of merge_flag for that one candidate block. As each merge_flag of neighboring candidate blocks can be 1 or 0, three contexts are available for merge_flag. merge_left_flag may be coded using only one probability model.

However, in accordance with alternative embodiments, other context models may be used. For example, non-binary syntax elements may be mapped on a sequence of binary symbols called bins. While other syntax elements or bins of syntax elements may be coded in a fixed context model, a context model for a syntax or binaries of syntax elements that define merging information may be a neighboring block. May be obtained based on the number or other measures of syntax elements or candidate blocks already transmitted for the data.

According to the description of the merging of blocks described above, the set of candidate blocks is the same as one of the embodiments described above with the following correction (candidate blocks are limited to blocks using motion-compensation or inter prediction, respectively). Can be obtained. Only they can be elements of a collection of candidate blocks. Contextual modeling of the signaling and merging information may be as described above.

In view of the switch to the combination of the multitree subdivision embodiments described above and the merging described above now, if a picture is divided into square blocks of various sizes by use of a quadtree-based subdivision structure, an example is given. For example, merge_flag and merge_left_flag or other syntax elements specifying merge may be interleaved with the prediction parameters sent for each leaf node of the quadtree structure. For example, consider FIG. 9A again. 9A shows an example of quadtree-based subdivision of a picture in prediction blocks of various sizes. The upper two blocks of the largest size are the so-called treeblocks, ie they are the predictable blocks of the largest possible size. The other blocks in FIG. 9A get the same subdivision of their corresponding treeblock. The current block is marked with an "X". All shaded blocks are encoded / decoded before the current block, so that all the shaded blocks form a collection of causal blocks. It is analyzed within the specification of the acquisition of the set of candidate blocks for one of the embodiments and includes a direction (i.e., top or left) that is directly neighboring the samples of the top-left sample position of the current block. Only blocks can be members of a set of candidate blocks. Thus, the current block can be merged with block "A" or block "B". If merge_flag is equal to 0 (for "false"), the current block "X" is not merged with the other two blocks. If the "A" and "B" blocks have the same prediction parameters, the merge of the two blocks will lead to the same result, so there is no need to distinguish. In this case, therefore, merge_left_flag is not transmitted. Conversely, if block "A" and block "B" have different prediction parameters, merge_left_flag such as 1 (for "true") will merge block "X" and "B", as compared to 0 ("false"). Merge_left_flag (for ") will merge the blocks" X "and" A ". In another embodiment, further neighboring (already transmitted) blocks indicate candidates for merging.

Another embodiment is shown in FIG. 9B. Here, the current block "X" and the left neighboring block "B" are treeblocks, that is, they have a maximum allowed block size. The size of the upper neighboring block "A" is 1/4 of the treeblock size. Blocks that are elements of a set of causal blocks are shaded. According to one embodiment of the present invention, the current block "X" cannot merge with other upper neighboring blocks, only with "A" or "B" two blocks. In another embodiment, further neighboring (already transmitted) blocks indicate candidates for merging.

Before continuing with the description of a method aspect of handling different sample arrangements of a picture according to an embodiment of the present application, multitree subdivision, signaling in one hand, merging in the other hand The above discussion with regards to clarify the aspects that provide the advantage that can be used independently from each other. That is, as already described above, the combination of multitree subdivision and merging has specific advantages, but the advantages also arise from alternatives in which the merging feature is embodied, for example. However, the subdivision performed by subdivisions 30 and 104a is not based on quadtree or multitree subdivision, but rather corresponds to macroblock subdivision into regular subdivisions of macroblocks. Conversely, in order, the combination of multitree subdivisions according to the transmission of the maximum treeblock size indication in the bitstream and the multitree subdivisions according to the use of depth-first traversal order carrying the corresponding coding parameters of the blocks. The use of is independent from the merging feature used in concurrent or non-concurrent states. In general, the advantages of merging are that coding efficiency may be increased when the syntax of sample array codings is extended in a manner that allows not only the subdivision of blocks but also the merging of two or more blocks obtained after subdivision. Given that it can be understood intuitively. As a result, the merge yields a group of blocks that are coded with the same prediction parameters. Prediction parameters for a group of blocks need only be coded once. Furthermore, with regard to the merging of sets of samples, it should be noted that the sets of samples considered can be rectangular or square blocks, and in the case of merged sets of samples, it represents a collection of rectangular and / or square blocks. . However, instead, the considered sets of samples are arbitrarily specified picture regions, and the merged sets of samples represent a collection of arbitrarily specified picture regions.

The following discussion focuses on handling different sample arrays of a picture when there is at least one sample array per picture, and an overview of some aspects in the following description is independent of the type of subdivision used, i.e. multitree detail. There are advantages that are independent from subdivision based on partitioning or not multitree subdivision, and from merging used or unused. Before beginning to describe specific embodiments regarding the handling of different sample arrangements of a picture, the main issue of these embodiments is synchronized by the method of short introduction in the field of handling of different samples per picture.

The following discussion focuses on coding parameters between blocks of other sample arrangements of a picture in an image or video coding application, and in particular, the following discussion may, for example but not exclusively, be used with the encoder and decoder of FIGS. It focuses on the method of adaptive predictive coding parameters between different sample arrangements of a picture in another image or video coding environment. As described above, the sample arrangements may represent sample arrangements associated with a picture or sample arrangements associated with other color elements along with additional information such as transparency data or depth maps. Sample arrangements associated with the color elements of a picture are also referred to as color planes. The techniques described in the following relate to inter-plane adoption / prediction, which may be used for block-based images, video encoders, decoders, and because of inter-plane adoption / prediction The processing order of the blocks of sample arrays for may be arbitrary.

Image and video coders are typically designed for coding color pictures (either of still images or pictures of a video sequence). The color picture consists of multiple color planes representing a sample arrangement for other color elements. Primarily, color pictures are coded into a set of sample arrangements consisting of a luma plane and two chroma planes that specify color difference elements. In some application areas, it is common for a set of coded sample arrays to consist of three color planes representing sample arrays for the three primary colors red, green and blue. Additionally, for improved color representation, a color picture may consist of more than three color planes. Moreover, the picture may be associated with preliminary sample arrangements detailing additional information for the picture. For example, preliminary sample arrays may be used to determine sample arrays or depth maps (e.g. suitable for rendering multiple views, eg 3-D displays) detailing the transparency (suitable for detailed display purposes) for the associated color sample arrays. It may be sample arrays to refine.

In conventional image and video coding standards (e.g., H.264), the color planes are coded together, thereby allowing specialty such as macro block and sub-macroblock prediction modes, reference indices and motion vectors. Coding parameters are used for all color elements of the block. The luma plane is considered the primary color plane for which special coding parameters are detailed in the bitstream, and the chroma plane is the secondary planes for which the corresponding coding parameters are inferred from the primary luma plane. May be considered. Each luma block is associated with two chroma blocks that represent the same region in the picture. Depending on the chroma sampling format used, the chroma sample arrangements may be smaller than the luma sample arrangement for the block. For each macroblock consisting of one luma and two chroma elements, the same partition in smaller blocks is used (when the macroblock is subdivided). For each block consisting of a block of luma samples and two blocks of chroma samples (which can be a macroblock or a subblock of a macroblock), reference indices, motion parameters And, sometimes, prediction parameter sets, such as intra prediction modes. In the detailed profiles of the conventional video coding standard (4: 4: 4 profiles in H.264), it is possible to code different color planes of a picture independently. In such a configuration, macroblock partitions, prediction modes, reference indices, and motion parameters may be selected separately for the color component of the macroblock or subblock. In conventional coding standards, all color panels are coded together using the same set of special coding parameters (such as information and prediction parameters to detail), or all color panels are coded completely independently of each other.

If the color planes are coded together, one set of subdivisions and prediction parameters should be used for all color elements of one block. This ensures that the auxiliary information is kept small, but it is comparable to independent coding because the use of different block decompositions and prediction parameters for different color elements can lead to lower rate-distortion costs. This can cause a reduction in coding efficiency. As an example, the use of reference frames for other motion vectors or chroma elements can significantly reduce the energy of residual signals for chroma elements and can significantly increase their overall coding efficiency. If the color planes are coded independently, coding parameters such as block partitions, reference indices and motion parameters can be selected individually for each color element to optimize coding efficiency for each color element. However, it is not possible to use repetition between color elements. Multiple transmission of special coding parameters can lead to an increased auxiliary information rate (compared to merged coding), which can have a negative effect on the overall coding efficiency. In addition, support for preliminary sample arrangements in the latest video coding standard (eg H.264) is limited when preliminary sample arrangements are coded when using their own set of coding parameters.

Thus, in all the embodiments described so far, the picture planes can be handled as described above, but if it can be determined in one block, for example, all samples for one block with the same coding parameters If the arrays may or may not be coded or other coding parameters may or may not be used, coding of multiple sample arrays (which may be associated with other color planes and / or preliminary sample arrays) as described above. Overall coding efficiency can be increased. The basic idea of the following inter-plane prediction is, for example, to allow adaptive decisions on a single block basis. The encoder may for example be based on a rate-distortion criterion, whether all or some of the sample arrays for a particular block are coded using the same coding parameters or are not coded or other coding parameters for other sample arrays. You can choose whether it is used or not. This selection may be performed by signaling for a particular block of the sample array if the detailed coding parameters are inferred from a block that has already been coded and co-located in another sample array or not. It is possible to adjust other sample arrangements for the picture in the groups also referred to as sample arrangement groups or plane groups. Each plane group may include at least one sample arrangement of a picture. And blocks of sample arrays inside a plane group are subdivided into the same selected coding parameters, e.g., while other coding parameters, e.g., transmission coefficient levels, are individually transmitted for each sample array in the plane group. Share information, prediction modes, and residual coding modes. One plane group is coded into the primary plane group, ie no coding parameters are inferred or predicted from the other plane groups. For each block of the second plane group, adaptively select whether a new set of selected coding parameters is transmitted or not transmitted or whether the selected coding parameters are inferred or predicted from the primary or other secondary plane group. Can be. Decisions about whether selected coding parameters for a particular block are inferred or predicted are included in the bitstream. Inter-plane prediction allows greater freedom in choosing between tradeoffs between the auxiliary information rate and the prediction quality associated with the latest coding of a picture consisting of multiple sample arrays. An advantage is the improved coding efficiency associated with conventional coding of pictures consisting of multiple sample arrangements.

Intra-plane adoption / prediction can extend an image or video coder, such as, for example, with a selected set of coding parameters already coded together of other sample arrangements within the same picture, as in the embodiments described above. Whether a block or preliminary sample of a color sample array is inferred or predicted from the placed blocks or whether the selected set of coding parameters for the block is independently coded without reference to the co-located blocks of other sample arrays within the same picture It may be adaptively selected by a set of arrangements or color sample arrangements and / or preliminary sample arrangements. Decisions about whether a selected set of coding parameters are inferred or predicted for a block of sample arrays or a block of multiple sample arrays may be included in the bitstream. Other sample arrays associated with the picture do not have to have the same size.

As discussed above, sample arrangements associated with a picture (sample arrangements represent color elements and / or preliminary sample arrangements) may be adjusted to two or more so-called plane groups, each plane group being at least one sample arrangement. It consists of. The sample arrays contained within a particular plane group do not have to have the same size. Adjustments to plane groups include cases where each sample array is coded separately.

More specifically, according to one embodiment, whether the coding parameters specifying how the block is predicted are inferred or predicted from already coded and co-located blocks of different plane groups for the same picture or these coding parameters are individually for the block. Is coded adaptively for each block of one plane group. The coding parameters that specify how the block is predicted include at least one of the following coding parameters: block prediction modes that specify what prediction is used for the block (intra prediction, single motion vector and inter prediction using a reference picture, two Inter prediction using motion vectors and reference pictures, inter prediction using higher-mode, that is, non-translational motion model and one single reference picture, multiple motion models and reference pictures Inter prediction modes), intra prediction modes specifying how one intra prediction signal is generated, an identifier specifying how many prediction signals are merged to produce the last prediction signal for the block, which reference picture (s) Reference indices, reference picture (s) that specify whether to use for motion-compensated prediction Motion parameters (e.g., displacement vectors or affine motion parameters) that specify how the prediction signal (s) are generated when used, how to generate motion-compensated prediction signals Includes an identifier that specifies whether the (s) are to be filtered. In general, one block may only be associated with a subset of the aforementioned coding parameters. For example, if the block prediction mode specifies that one block is intra predicted, the coding parameters for the block may additionally include intra prediction models, but the coding parameters such as the reference index and how the inter prediction signal may be Motion parameters specifying whether they are created are not specified; Or if the block prediction mode specifies inter prediction, the related coding parameters may additionally include reference indices and motion parameters, but intra prediction modes are not specified.

One of the two or more plane groups may be coded or indicated in the bitstream, such as a primary plane group. For all blocks of this primary plane group, coding parameters specifying how the prediction signal is generated are transmitted without reference to another plane group of the same picture. The remaining plane groups are coded into second plane groups. For each block of secondary plane groups, at least one syntax element is transmitted to infer or predict whether coding parameters to specify how the block is predicted are inferred or predicted from the co-located block of other plane groups. Or a new set of such coding parameters is sent for the block or not. One of the one or more syntax elements may be referred to as an inter-plane prediction flag or an inter-plane prediction parameter. If the syntax elements signal that the corresponding coding parameters are not inferred or predicted, then a new set of corresponding coding parameters for the block is sent in the bitstream. If the syntax elements signal that the corresponding coding parameters are inferred or predicted, then the assignment of the reference plane group for the block may be configured in a multiple way. In one embodiment, a particular reference plane group is assigned to each second plane group; This assignment may be fixed or signaled in high-level syntax structures, eg, parameter sets, access unit header, picture header, or slice header.

In a second embodiment, the assignment of the reference plane group can be coded inside the bitstream, and the assignment of the reference plane group is coded for the block to specify whether the selected coding parameters are inferred, predicted or individually coded. The signal may be signaled by at least one syntax element.

To facilitate the possibilities mentioned with respect to inter-plane prediction and the following detailed embodiments, the reference is represented by FIG. 11 which clearly shows a picture 500 consisting of three sample arrangements 502, 504, 506. . For easier understanding, only sub-portions of sample arrangements 502-506 are shown in FIG. 11. The sample arrays are shown as if they are spatially registered with respect to each other so that the sample arrays 502-506 are placed next to each other along the 508 direction and the samples of the sample arrays 502-506 along the 508 direction. Projection causes the samples of all such sample arrangements 502-506 to be accurately and spatially positioned to each other. That is, planes 502 and 506 are laid out along the horizontal and vertical directions to adapt their spatial resolutions to each other and register each other.

According to one embodiment, all the sample arrangements of the picture belong to the same part of the spatial scene, where the resolution along the vertical and horizontal directions may differ between the individual sample arrangements 502-506. Furthermore, in FIG. 11, the sample arrangements 502, 504 are considered to belong to one plane group 510, whereas the sample arrangements 506 are considered to belong to another plane group 512. Moreover, FIG. 11 illustrates a typical case where the spatial resolution along the horizontal axis of the sample array 504 is doubled for decomposition in the horizontal direction of the sample array 502. Moreover, sample arrangement 504 is contemplated to construct a primary arrangement associated with sample arrangement 502, which constitutes a dependent arrangement associated with primary arrangement 504. In this case, as described above, according to the example of FIG. 11, subdividing the sample arrangement 504 into blocks as determined by the subdivision section 30 of FIG. 1 is adopted by the subordinate arrangement 502. Because of the vertical resolution of the sample array 502, which is half resolution in the vertical direction of the primary array 504, each block is bisected into two horizontal juxtaposition blocks, and of the sample positions in the sample array 502. When measured in units, the blocks are squared again.

As currently shown in FIG. 11, the subdivision selected for sample arrangement 506 is different from the subdivision of other planar groups 510. As described above, the subdivision section 30 may individually select subdivisions of the pixel arrangement 506 or independently select subdivisions for the plane group 510. Of course, the resolution of the sample arrangement 506 may be different from the resolution of the planes 502, 504 of the plane group 510.

When encoding the individual sample arrays 502-506, the encoder 10 may begin with coding of the primary array 504 in a plane group, for example, in the manner described above. The blocks of FIG. 11 may be the prediction blocks mentioned above, for example. Instead, the blocks are residual blocks or other blocks that define granularity for defining certain coding parameters. Although the inter-plane prediction is shown in FIG. 11, it is not limited to quadtree or multitree subdivision.

After transmission of the syntax element for primary array 504, encoder 10 determines to declare primary array 504 as a reference plane for subordinate plane 502. The association may be apparent from the fact that the sample arrangement 504 forms a primary arrangement of plane groups 510, in which information may also be in order and part of the bitstream 22, and the encoder 10 and extraction Each of the sections 30 may signal for this determination via the bitstream 22. In this case, for each block in the sample array 502, which is the insert 18, or with the insert 18, another module of the encoder 10 can suppress the movement of the coding parameters of this block in the bitstream. Determine to send a signal in the bitstream to replace the block used in place of the coding parameters of the block placed together in primary array 504, or placed together in primary array 504 The coding parameters of the given block may be used together with prediction for the coding parameters of the current block of the sample array 502 with only transmitting residual data for the current block of the sample array 502 in the bitstream. In the case of a negative decision, the coding parameters are usually transmitted in the data stream. The decision is signaled in the data stream 22 for each block. On the decoder side, the extractor 102 appropriately uses this interplanar prediction information for each block to obtain the coding parameters of each block of the sample array 502, i.e., placing the primary array 504 together. By inferring or instead of coding parameters of a given block, extracting residual data for that block from a data stream and if the interplanar adoption / prediction information suggests interplanar adoption / prediction, This residual data can be merged with the prediction obtained from the coding parameters of the co-located blocks, or the coding parameters of the current block of the independent sample array 502 can usually be extracted from the primary array 504.

As mentioned above, the reference planes are not limited to being in the same plane group as the block in which interplanar prediction is currently interested. Thus, as described above, plane group 510 may represent a reference plane group for primary plane group or secondary plane group 510. In this case, the bitstream consists of a sample arrangement as to whether the already mentioned adoption / prediction of the coding parameters of the co-located macroblocks of the planes 502 and 504 of the reference plane group 510 or the primary plane group is performed. Syntax elements indicating for each block of 506, and the coding parameters of the current block of the sample array 506 can usually be transmitted if no adoption / prediction is performed.

The subdivision and / or prediction parameters for the planes inside the plane group may be the same, i.e. they are subdivided information and / or from one group (all the second planes of one plane are from the primary plane inside the same plane group And coded only once for the prediction parameters), and the interference of the adaptive prediction or subdivision information and / or prediction parameters is performed between plane groups.

The reference plane group may be a primary plane group or may be a secondary plane group.

Co-location together between blocks of different planes in one plane group can be easily understood, while subdivision of the primary sample array 504 can be spatially adopted by the dependent sample array 502. And sub-division of the blocks just described to make the adopted leaf blocks into square blocks. In the case of interplanar adoption / prediction between different plane groups, co-loaction can be defined in a way related to allowing greater freedom between subdivision of such plane groups. When a reference plane group is defined, blocks arranged together inside the reference plane group are determined. Acquisition of blocks and reference plane groups arranged together may be performed by a process similar to the following. The particular sample 514 in the current block 516 of one of the sample arrays 506 of the second plane group 512 is selected. The same may be the upper left sample of the current block 516, indicated by 514 of FIG. 11, and may be a sample in the current block 516 that is close to another sample in the middle of the current block 516 or inside the current block. The placement of the selected sample 515 inside the sample array 502 and 502 and 504 of the reference plane group 510 are calculated. The positions of the sample in the sample arrangements 502, 504 are represented by 518 and 520 of FIG. 11, respectively. Substantially used planes 502 and 504 in the reference plane group 510 may be predetermined or signaled within the bitstream. Samples in the corresponding sample arrays 502 or 504 of the reference plane group 510 closest to positions 518 and 520 are determined, respectively, and a block comprising such samples is in each sample array 502 and 504. Each is selected as a co-located block. In the case of FIG. 11, these are 522 and 524 blocks, respectively. Alternative techniques for the determination of blocks placed together in different planes are described later.

In one embodiment, the coding parameters specifying the prediction for the current block 516 may correspond to blocks 522/524 that are co-located within another plane group 510 of the same picture 500 without additional auxiliary information transmitted. It can be completely inferred using the prediction parameters. The inference can consist of a simple copy of the corresponding coding parameters or the adaptation of the coding parameters taking into account the difference between the current 512 and the reference plane group 510. For example, this adaptation may be further configured with motion parameter correction (eg, substitution vector correction) to account for the phase difference between luma and chroma sample arrangements, or the adaptation may be It can be configured with precise modulation of motion parameters to take into account different resolutions of luma and chroma sample arrangements. In another embodiment, at least one inferred coding parameter to specify prediction signal generation is not used directly for current block 516, but as a prediction for corresponding coding parameters for current block 516. An improvement of these coding parameters for the current block 516 is transmitted in the bitstream 22. For example, inferred motion parameters are not used directly, but motion parameter differences (eg, substitutions) that specify a deviation between the motion parameters used for the current block 516 and the inferred motion parameters. Vector difference) is coded in the bitstream; On the decoder side, the motion parameters actually used are obtained by merging the inferred motion parameters and the transmitted motion parameter differences.

According to another embodiment, the treeblocks of the aforementioned prediction subdivision with subdivision of the block, e. It is adaptively inferred or predicted from already coded co-located blocks of another plane group for the bit sequence according to FIG. 6B. According to an embodiment, one of the plurality of plane groups is coded into a primary plane group. For all blocks of this primary plane group, subdivision information is transmitted without reference to other plane groups of the same picture. The remaining plane groups are coded into second plane groups. For blocks of the second plane groups, at least one syntax element is transmitted to signal whether subdivision information is inferred or predicted from the co-located block of other plane groups or whether subdivision information is transmitted in the bitstream. One of the one or more syntax elements may be referred to as an inter-plane prediction flag or an inter-plane prediction parameter. If syntax elements signal that subdivision information is not inferred or predicted, subdivision information for the block may be transmitted in the bitstream without reference to other plane groups of the same picture. When syntax elements signal that subdivision information is inferred or predicted, the co-located blocks in the so-called reference plane group are determined. The assignment of reference plane groups for blocks is organized in multiple ways. In one embodiment, a particular reference plane group is assigned to each second plane group; This assignment may be fixed or this assignment may be signaled in higher-level syntax structures such as parameter sets, access unit headers, picture headers or slice headers. In a second embodiment, the assignment of a reference plane group is signaled by at least one syntax element that is coded within the bitstream and coded for a block to specify whether subdivision information is inferred, predicted or individually coded. Can be. The reference plane group may be a primary plane group or another secondary plane group. Given a reference plane group, the co-located blocks inside the reference plane group are determined. The co-located blocks are blocks within the reference plane group that correspond to the same image region as the current block, or blocks representing inner blocks of the reference plane group that share the largest portion of the image region with the current block. Blocks placed together may be partitioned into smaller prediction blocks.

In another embodiment, the subdivision information for the current block, for example quadtree-based subdivision information according to FIG. 6A or 6B, is a block placed together in another plane group of the same picture without transmission of additional auxiliary information. It is fully inferred using the subdivision information of. In a particular example, if a block placed together is partitioned into two or four prediction blocks, the current block may be partitioned into two or four subblocks for prediction. In another particular example, if a co-located block is partitioned into four subblocks and one of these subblocks is further partitioned into four smaller subblocks, then the current block is also partitioned into four subblocks. One of these subblocks (one corresponding to the subblocks of the more co-located block) is also partitioned into four smaller subblocks. In another embodiment, the inferred subdivision information is not directly used as the current block, but the inferred subdivision information is used as a prediction for the actual subdivision information for the current block, and the corresponding improved information is in the bitstream. Is sent from. For example, the subdivision information deduced from the co-located blocks can be further improved. For each subblock that corresponds to a subblock in a co-located block that is not partitioned into smaller blocks, the syntax element may be coded within the bitstream specifying whether the subblock is further decomposed within the current plane group. . The transmission of syntax elements may be limited by the size of the subblock. Alternatively, the transmission of syntax elements may be signaled in the bitstream so that subblocks further partitioned within the reference plane group are no longer partitioned into smaller blocks within the current plane group.

In another embodiment, the coding parameters specifying the subdivision of a block in the prediction blocks and how the subblocks are predicted may be adaptively inferred or predicted from an already coded co-located block of another plane group for the same picture. Can be. In one embodiment of the invention, one of the plurality of plane groups is coded into a primary plane group. For all blocks of this primary plane group, subdivision information and prediction parameters are transmitted without reference to other plane groups of the same picture. The remaining plane groups are coded into secondary plane groups. For blocks of secondary plane groups, at least one syntax element is transmitted whether subdivision information and prediction parameters are inferred or predicted from co-located blocks of other plane groups or subdivision information and prediction parameters are transmitted in the bitstream. Signal whether or not One of the one or more syntax elements may be referred to as an inter-plane prediction flag or an inter-plane prediction parameter. If syntax elements signal that subdivision information and prediction parameters are not inferred or predicted, the subdivision information for the block and the prediction parameters for the resulting subblocks are transmitted in the bitstream without referring to other plane groups of the same picture. . If the syntax elements signal that subdivision information and prediction parameters are inferred or predicted, then the co-located blocks in the so-called reference plane group are determined. The assignment of the reference plane group for the block may consist of multiple methods. In one embodiment, a particular reference plane group is assigned to each second plane group; This assignment can be fixed or can be signaled within higher-level syntax structures such as, for example, parameter sets, access unit headers, or slice headers. In a second embodiment, the assignment of the reference plane group is coded inside the bitstream and at least one syntax elements coded for a block to specify whether subdivision information and prediction parameters are to be inferred or predicted or individually coded. Signals can be sent by The reference plane group may be a primary plane group or may be another secondary plane group. In a given reference plane group, the co-located blocks inside the reference plane group are determined. Blocks placed together may be blocks within a reference plane group that correspond to the same image region as the current block, or may be blocks representing blocks inside the reference plane group that share the largest portion of the image region with the current block. . Blocks placed together may be partitioned into smaller prediction blocks. In one embodiment, the subdivision information for the current block and the prediction parameters for the resulting subblocks are subdivided information of the co-located blocks and corresponding subblocks in different plane groups of the same picture without transmission of additional auxiliary information. Inferred when using the parameters. In a particular embodiment, if the co-located block is partitioned into two or four prediction blocks, the current block may also be partitioned into two or four subblocks for prediction purposes and the sub-blocks of the current block. Prediction parameters for the blocks are obtained as described above. In another particular embodiment, if the co-located block is partitioned into four subblocks, one of these subblocks is further partitioned into four smaller subblocks, and the current block is divided into four subblocks. Also partitioned, one of these subblocks (corresponding to the subblocks of the more co-located block) is also partitioned into four smaller subblocks, the prediction parameters for all of which are not partitioned further described above. As inferred. In another embodiment, the subdivision information is fully inferred based on the subdivision information of the co-located blocks in the reference plane group, but the inferred prediction parameters for the subblocks are used only as predictions for the actual prediction parameters of the subblocks. do. Deviations between the actual prediction parameters and the inferred prediction parameters are coded in the bitstream. In another embodiment, the inferred subdivision information is used as prediction for the actual subdivision information for the current block and the difference is transmitted in the bitstream (as described above), but the prediction parameters are fully inferred. In another embodiment, the inferred subdivision information and the inferred prediction parameters are used as predictions and the differences between the actual subdivision information and the prediction parameters and their inferred values are transmitted in the bitstream.

In another embodiment, the residual coding modes are not inferred or predicted from the already coded and co-located block of another plane group for the same picture or the residual coding modes are individually for the block. Whether coded or uncoded may be adaptively selected for the block of plane groups. This embodiment is similar to the embodiment for adaptive inference / prediction of prediction parameters as described above.

In one embodiment, subdivision of a block (ie, prediction block) into transform blocks (i.e., blocks of samples to which a secondary transform has been applied) is from an already coded and co-located block of another plane group for the same picture. Can be adaptively inferred or predicted. This embodiment is similar to the embodiment for adaptive inference / prediction of subdivision into prediction blocks as described above.

In another embodiment, subdivision of the block into transform blocks and residual coding modes (ie, transform forms) for the resulting transport blocks are adapted from an already coded co-located block of another plane group for the same picture. Can be inferred or predicted. This embodiment is similar to the embodiment for adaptive inference / prediction of the subdivision into prediction blocks described above and the prediction parameters for the resulting prediction blocks.

In another embodiment, subdivision of a block into prediction blocks, associated prediction parameters, subdivision information of prediction blocks, and residual coding modes for transform blocks are already coded and placed together in another plane group for the same picture. Adaptively inferred or predicted from the generated block. This embodiment represents a combination of the embodiments described above. It is also possible that only some of the coding parameters mentioned are inferred or predicted.

Thus, interplanar adoption / prediction can increase the coding efficiency previously described. However, the coding efficiency gains by the method of inter-planar adoption / prediction are available for other block subdivisions used in addition to multitree-based subdivisions, and can be separate from the block merging implemented or unimplemented.

The above-described embodiments regarding inter-plane adoption / prediction can apply to image and video encoders and decoders that divide the color planes of a picture, where applicable, preliminary sample arrangements are combined with the picture in the blocks and , Associating these blocks with coding parameters. For each block, a set of coding parameters can be included in the bitstream. For example, these coding parameters may be parameters describing how the block can be predicted or decoded at the decoder. In particular embodiments, the coding parameters may include macroblock or block prediction modes, sub-separation information, intra prediction modes, reference indices used for motion-compensated prediction, motion parameters such as substitution vectors, residual Coding modes, transform coefficients, and the like. Different sample arrangements associated with the picture may have different sizes.

In the following, for example, a technique for enhanced signaling of coding parameters within the tree-based partition scheme described above with respect to FIGS. 1 to 8 is described. The merging and inter-plane adoption / prediction, the effects and gains of enhanced signaling techniques, and other techniques, referred to below as layers, can be derived from the above embodiments, although the techniques described below may be combined with one or a combination of the above embodiments. Are described independently.

In general, an improved coding scheme for coding assistance information within the tree-based partitioning scheme described next called a layer enables the following advantages associated with conventional techniques of coding parameter processing.

In conventional image and video coding, particular sets of pictures or sample arrangements for pictures are usually broken down into blocks associated with particular coding parameters. Pictures are usually composed of multiple sample arrays. In addition, a picture specifying, for example, transparency information or depth maps may also be associated with additional preliminary sample arrangements. The sample arrangements (including the preliminary sample arrangements) of the picture may be grouped into one or more so-called plane groups, and each plane group of the one or more so-called plane groups may consist of at least one sample arrangement. Plane groups of a picture may be coded independently, or if a picture is associated with one or more plane groups, it may be coded with prediction from other plane groups of the same picture. Each plane group is usually broken up into blocks. Blocks (or corresponding blocks of sample arrangements) are predicted by inter-picture prediction or intra-picture prediction. The blocks can have different sizes and can be square or rectangular. The partitioning of the picture into blocks can be fixed by syntax, or a signal can be sent (in the smallest part) inside the bitstream. Often the syntax elements are sent subdivided signals for blocks of a predefined size. Syntax elements may specify how a block is subdivided into smaller blocks and related to coding parameters, i.e. for prediction purposes. All samples (or corresponding blocks of sample arrangements) of the decoding block of related coding parameters are specified in a certain way. For example, all samples of a block specify, for example, reference indices (which identify a reference picture within a set of already coded pictures), motion parameters (measuring the movement of the block between the reference picture and the current picture). Is predicted using the same set of prediction parameters, such as parameters for specifying an insertion filter, intra prediction modes and the like. The motion parameters can be represented by displacement vectors, such as horizontal and vertical elements or higher dimension motion parameters, for example transform motion parameters consisting of six elements. At least one set of special prediction parameters (eg, reference indices and motion parameters) is associated with a single block. In this case, for each set of these special prediction parameters, a single intermediate prediction signal for the block (or corresponding blocks of sample arrangements) is generated, and the last prediction signal is in a combination comprising the intermediate prediction signals superimposed. Is made by Corresponding weighting parameters and potential invariant offsets (added as weighted sums) may be fixed to the picture, or to the reference picture, or to the set of reference pictures, or to the set of prediction parameters for the corresponding block. It may be included within. The difference between the original blocks (or corresponding blocks of the sample arrangements) and their prediction signals are also referred to as the residual signal and are usually transformed and quantized. Often, quadratic transformation is applied to the residual signal (or the corresponding sample for the residual block).

For transform coding, the blocks used for a particular set of prediction parameters (or corresponding blocks of sample arrays) can be further split before applying the transform. The transform blocks can be the same as or smaller than the blocks used for prediction. It is also possible for the transform block to comprise at least one block used for prediction. Different transform blocks can have different sizes, and the transform blocks can represent square or rectangular blocks. After the transform, the resulting transform coefficients are quantized and so-called transform coefficient levels are obtained. Transform coefficient levels and prediction parameters and subdivided information, if indicated, are entropy coded.

In some image and video coding standards, the possibility of subdivision into blocks provided by the syntax of a picture (or plane group) is very limited. Usually, it can only be specified which side or potentially how a block of predefined size can be subdivided into smaller blocks. For example, the largest block size in H.264 is 16x16. The 16x16 blocks are also referred to as macro blocks, each picture being partitioned into macro blocks in the first step. For each 16x16 macroblock, a signal may be signaled whether each 16x16 macroblock is coded as a 16x16 block, or as two 16x8 blocks, or as two 8x16 blocks, or as four 8x8 blocks. have. If a 16x16 block is subdivided into four 8x8 blocks, each 8x8 block may be coded into one 8x8 block, or two 8x4 blocks, or two 4x8 blocks, or four 4x4 blocks. In the latest image and video coding standards, a small set of possibilities for the specification of partitions into blocks has the advantage that the auxiliary information rate for signaling subdivision information can be kept small, but the transmission of prediction parameters for blocks The disadvantage is that the bit rate required for this is important as described below. The auxiliary information rate for signaling the prediction information usually represents an important amount of the overall bit rate for the block. And, the coding efficiency can increase when this assistance information is reduced, for example, can be performed by using larger block sizes. Pictures of the actual images or video sequence may consist of objects of any shape with specific properties. For example, the objects or portions of objects may be characterized by a unique texture or unique movement. And usually, the same set of prediction parameters can be applied for an object or part of an object. However, object boundaries usually do not coincide with possible block boundaries for large prediction blocks (eg, 16x16 macroblocks in H.264). The encoder usually determines the subdivision (between the limited set of possibilities) that measures the results within the minimization of a particular ratio-distortion cost. For objects of any shape, the encoder can cause a large number of small blocks. Since each of these small blocks is associated with a set of prediction parameters that need to be transmitted, the auxiliary information rate can be an important part of the overall bit rate. However, some of the smaller blocks still represent the same object or part of the object, so the prediction parameters for a large number of obtained blocks are the same or very similar. Intuitively, the coding efficiency can be increased when syntax is extended by a method that allows subdivision of blocks and by sharing coding parameters between the blocks obtained after subdivision. In tree-based subdivision, sharing of coding parameters for a given set of blocks may be performed by allocation of coding parameters or parts thereof for at least one parent mode in the tree-based layer. As a result, the shared parameters or portions thereof can be used to reduce the auxiliary information needed to signal the actual selection of coding parameters for the blocks obtained after subdivision. Reduction may be performed by omitting signaling of parameters for subsequent blocks or using shared parameter (s) for prediction and / or context modeling of parameters for subsequent blocks.

The basic idea of the layer scheme described below is to reduce the bit rate required for transmitting coding parameters by sharing information according to the tree-based layer of blocks. The shared information is signaled inside the bitstream (in addition to the information to be subdivided). The advantage of the layer technique is the increased coding efficiency resulting from reduced assistance information for the coding parameters.

In order to reduce the auxiliary information ratio, according to the embodiment described below, multi-indicating the respective coding parameters for particular sets of samples, namely rectangular or square blocks or arbitrary shaped regions or collections of other samples Simple connection areas in the tree subdivision are signaled inside the data stream in an efficient manner. The hierarchical technique described below ensures that the coding parameters do not need to be explicitly included in the bitstream for each such set of samples. The coding parameters may indicate prediction parameters that specify how the corresponding set of samples is predicted using already coded samples. Many possibilities and examples have been described above and may be applied here. As indicated above, as will be described further below, if the following hierarchical technique is involved, the tree-based partition of the sample arrays of a picture into sample sets may be fixed by syntax, or may be inside a bitstream. Signals can be sent by corresponding subdivision information. Coding parameters for sample sets may be sent in a predetermined order given by the syntax as described above.

According to the hierarchical technique, the decoder or extractor 102 of the decoder may be configured to simply obtain information of the coding parameters of the concatenated region or sample sets in a specific method. In particular, the coding parameters or parts thereof, such as those which assist for the purpose of prediction, are blocks along the given tree-based partitioning technique along with a shared group according to the tree structure determined by the encoder or insert 18. Each is shared between them. In a particular embodiment, the sharing of coding parameters for all child nodes of a given inner node of the partition tree is represented by the use of a specific binary-value sharing flag. In another approach, improvements in coding parameters can be sent for each node such that accumulated improvements in parameters along the tree-based layer of blocks can be applied to all sample sets of the block at a given leaf node. In another embodiment, portions of coding parameters sent for internal nodes along a tree-based layer of blocks may be used for context-adaptive entropy encoding and decoding of coding parameters or portions of coding parameters for a block at a given leaf node. Can be.

12A and 12B illustrate the basic idea of a hierarchy for a specific case using quadtree-based partitions. However, as indicated several times above, other multitree subdivision techniques can also be used. Although the tree structure is shown in FIG. 12A, the corresponding spatial partitioning of the tree structure corresponding to FIG. 12A is shown in FIG. 12B. The compartment shown therein is similar to that shown with respect to FIGS. 3A-3C. Generally speaking, the hierarchical technique may allow auxiliary information to be given to nodes in several non-leaf layers in the tree structure. As a result of providing auxiliary information to nodes at various layers in the tree, for example internal nodes or their root nodes in the tree of FIG. Sharing can be obtained. For example, if in the case of Fig. 12A it is determined that all leaf nodes in Layer 4, all having the same parent node, share auxiliary information, then in fact this means that the smallest blocks in Fig. 12B, represented by 156a through 156d, have this auxiliary information. This means that it is no longer necessary to share and send auxiliary information for all these small blocks 156a through 156d completely, i.e. four times, which is left as an option for the encoder. However, the entire area of hierarchical level 1 (layer 2) of FIG. 12A, i.e., the upper right 1 of tree block 150, which includes subblocks 154a, 154b and 154d and the smaller subblocks 156a to 156d already mentioned. It is also possible to provide the / 4 part to the area where the coding parameters are shared. Thus, the area sharing auxiliary information increases. The next level of increase may be the sum of all subblocks of layer 1, ie the subblocks 152a, 152c and 152d and the smaller blocks mentioned previously. That is, in this case, the entire tree block will have auxiliary information assigned thereto along with all subblocks of this tree block 150 that share the auxiliary information.

In the following hierarchical description, the following notation is used to describe the embodiment:

a. Reconstructed Samples of Current Leaf Node: r

b. Reconstructed Samples of Neighbor Leaves: r '

c. Predictor of current leaf node: p

d. Residue of current leaf node: Re s

e. Restored Residues of Current Leaf Node: Re c Re s

f. Scaling and Inverse Transform: SIT

g. Share Flag: f

As a first embodiment of inheritance, it may be described as intra-prediction signaling at internal nodes. More precisely, a method of signaling intra-prediction modes at inner nodes of a tree-based block partition for the purpose of prediction is described. By traversing the tree from the root node to the leaf nodes, internal nodes (including the root node) can carry portions of auxiliary information to be used by corresponding child nodes. More specifically, the shared flag f is sent for internal nodes with the following meanings:

F has a value of ten thousand and one, is 1 ( "true"), all the child nodes of a given internal nodes such as intra-prediction mode share. In addition to the shared flag f with value 1, the inner node also signals the intra-prediction mode parameters used for all child nodes. As a result, all subsequent child nodes carry no shared mode information as well as any prediction mode information. For restoration of all related leaf nodes, the decoder applies intra-prediction mode from the corresponding inner node.

F has a value of ten thousand and one, is 0 ( "false"), the child node corresponding to the internal nodes, such as intra-individual child node is an internal node, instead of sharing a prediction mode carries a separate shared flag.

12C shows intra-prediction signaling at internal nodes as described above. Internal nodes in Layer 1 carry auxiliary information given by shared flags and intra-prediction mode information, and child nodes carry no auxiliary information.

As a second embodiment of inheritance, inter-prediction improvement can be described. More precisely, a method is described for signaling auxiliary information of inter-prediction modes in inner modes of tree-based block partitioning for the purpose of improving motion parameters such as given by motion vectors, for example. . By traversing the tree from the root node to the leaf nodes, internal nodes (including the root node) can carry supplemental information parts to be improved by corresponding child nodes. More specifically, the shared flag f is sent for internal nodes with the following meanings:

· F is ten thousand and one has the value 1 ( "true"), all of the children of a given mode, the internal node share the same motion vector is referred to. In addition to the shared flag f with value 1, the inner node also signals a motion vector and a reference index. In conclusion, all subsequent child nodes may carry no shared flags, but may carry an improvement in inherited motion vector reference. For reconstruction of all related leaf nodes, the decoder adds a motion vector improvement at a given leaf node to the inherited motion vector reference belonging to the corresponding inner parent node with the shared flag f with value 1. This means that the motion vector improvement at a given leaf node is the difference between the actual motion vector to be applied for motion-compensated prediction at this leaf node and the motion vector reference of its corresponding inner parent node.

Of motion parameters in the child nodes by using the motion parameters from the internal node corresponding not essential to share the prediction mode, ten thousand and one f is 0 ( "false") has a value, the corresponding child nodes are the same inter internal node No improvement is done, and each child node, which is an internal node, carries a separate shared flag.

12D illustrates the motion parameter improvement as described above. Internal nodes in Layer 1 carry shared flags and auxiliary information. Child nodes that are leaf nodes only carry motion parameter enhancements, for example, internal child nodes in Layer 2 do not carry assistance information.

Reference is now made to FIG. 13. FIG. 13 is the same as the decoder of FIG. A flowchart showing the mode of operation of the decoder is shown. As mentioned above, each leaf region is associated with one hierarchical level of a sequence of hierarchical levels of multi-tree subdivision. For example, all blocks shown in FIG. 12B are leaf regions. For example, leaf region 156c is associated with hierarchy level 4 (or level 3). Each leaf region is associated with coding parameters there. Examples of such coding parameters have been described above. Coding parameters for each leaf region are represented by each set of syntax elements.

Each syntax element is a syntax element of an individual syntax element type of one of a set of syntax element types. This syntax element type may be, for example, a prediction mode, a motion vector element, an indication of an intra-prediction mode, or the like. According to FIG. 13, the decoder performs the following steps.

In step 550, inheritance information is extracted from the data stream. In the case of FIG. 2, the extraction unit 102 is responsible for step 550. Inheritance information indicates whether or not inheritance is used for the current arrangement of information samples. The following description presents some possibilities for inheritance information such as, for example, inter-alias, shared flag f, and signaling of a multitree structure separated into primary and secondary parts. Will be exposed.

The arrangement of information samples may already be a subpart of the picture, for example a treeblock, ie the treeblock 150 of FIG. 12B. Thus, inheritance information indicates whether inheritance is used or not used for the specific treeblock 150. Such inheritance information may be inserted into the data stream for all tree blocks of the prediction subdivision, for example.

In addition, the inheritance information consists of a set of leaf regions, if inheritance is indicated to be used, and of the hierarchical levels of the multi-tree subdivision, which are lower than the respective hierarchical levels to which the set of leaf regions are associated. Represent at least one inheritance region of the array of information samples, corresponding to the hierarchical level of one of the sequences. That is, the inheritance information indicates whether inheritance is used or not used for the current sample arrangement, such as treeblock 150. If used, the inheritance information indicates at least one inheritance region or sub-region of this treeblock 150, which shares coding parameters therein. Thus, the inheritance region may not be a leaf region. In the example of FIG. 12B, this inheritance region may be, for example, an area formed by subblocks 156a, 156b. Alternatively, the inheritance region may be larger and may further include subblocks 154a, 154b and 154d, and even alternatively, a tree with all leaf blocks sharing coding parameters associated with the inheritance region. Block 150 may be itself.

However, it should be noted that two or more inheritance regions may each be defined within one sample array or tree block 150. For example, imagine that the lower left subblock 152c is partitioned into smaller blocks. In this case, the subblock 152c may also form a succession area.

In step 552, inheritance information is checked as to whether or not inheritance is used. If used, the process of FIG. 13 proceeds to 554 where, in step 554, a factorial subset including at least one predetermined syntax element is extracted from the data stream for each inter-inheritance region. In a subsequent step 556, this inheritance subset can be copied to the corresponding inheritance subset of syntax elements in the set of syntax elements, or used as a prediction, the syntax elements being the leaf on which each at least one inheritance region is constructed. Represents coding parameters associated with a set of regions. That is, for each inheritance region indicated in the inheritance information, the data stream includes a factorial subset of syntax elements. In other words, inheritance relates to at least one particular syntax element type or syntax element category available for inheritance. For example, prediction mode or inter-prediction mode or intra-prediction mode syntax elements may be inherited. For example, the inheritance subset included within the data stream for the inheritance region may include inter-prediction mode syntax elements. The inheritance subset can also include syntax elements, the syntax element types rely on the values of the fixed syntax element types mentioned above, and the fixed syntax element types are related to the inheritance technique. For example, in the case of inter-prediction mode, which is a fixed element of the inheritance subset, syntax elements defining motion compensation, for example motion-vector elements, may or may not be included in the inheritance subset by syntax. have. For example, the upper right quarter of treeblock 150, i.e. subblock 152b, is a factorial region and may be indicated for this factorial region only in inter-prediction mode or motion vectors and motion vector indexes. Can be directed for the inter-prediction mode.

All syntax elements included in the factorial subset are copied or used as prediction for the corresponding coding parameters of the leaf blocks in the factorial region, i.e., the leaf blocks 154a, 154b, 154d and 156a through 156d. If prediction is used, the residuals are sent for the individual leaf blocks.

One possibility of transmission of inheritance information for treeblock 150 is the transmission of the shared flag f already mentioned. In this case, the extraction of the inheritance information in step 550 includes the following. In particular, the decoder may include extraction and check for non-leaf regions corresponding to a predetermined factorial set of at least one hierarchical level of the multi-tree subdivision, and whether each factorial flag or shared flag indicates factorial. Use the shared flag f from the data stream, the hierarchical level order from the lower hierarchical level to the higher hierarchical level. For example, a factorial set of hierarchical levels may be formed by layers 1 through 3 of FIG. 12A. Thus, for certain nodes of a subtree structure that are not leaf nodes and that lie within certain layers 1 to 3, they may have a shared flag associated therewith in the data stream. The decoder may extract the shared flags in an order from layer 1 to layer 3, such as a depth-first or width-first traversal order. As soon as one of the shared flags is equal to 1, the decoder knows that the leaf blocks included in the corresponding inheritance region share the inherited subset that is subsequently extracted at step 554. For child nodes of the current node, checking of inheritance flags is no longer necessary. That is, inheritance flags for these child nodes are not transmitted in the data stream, since it is natural that the region of these nodes already belongs to the inheritance region where the inheritance subset of syntax elements is shared.

The shared flags f may be interleaved with the already mentioned bits that signaled the quadtree subdivision. For example, an interleaved bit sequence containing subdivision flags and shared flags may be as follows:

1 0 001 1 01 (0000) 000,

This is the same subdivision information as shown in FIG. 6A using two deployed shared flags, where the two deployed shared flags are all sub-blocks within one quarter of the lower left treeblock 150 of FIG. 3C. It is highlighted by an underline to indicate sharing coding parameters.

Another method for defining inheritance information indicative of the inheritance region may be the use of two subdivisions, the two subdivisions being defined dependent on each other as described above with respect to prediction and residual subdivision, respectively. Generally speaking, a leaf block of a primary subdivision can form an inheritance region, where the inheritance region defines the regions where the inheritance subsets of syntax elements are shared, while the dependent subdivision copies the inheritance subset of syntax elements. Or define blocks within these inheritance regions for use as prediction.

For example, consider the residual tree as an extension of the prediction tree. Also consider the case where the predictive blocks can be separated into smaller blocks for the purpose of residual coding. For each prediction block corresponding to the leaf node of the prediction-related quadtree, the corresponding subdivision for residual coding is determined by at least one dependent quadtree (s).

In this case, rather than using a certain prediction signal at the inner nodes, we predict in the sense of the use of the already described constant prediction mode (signaled by the corresponding leaf node of the prediction related tree) but with improved reference samples. Improvements in the tree also consider residual trees, such as how to materialize. The following example illustrates this case.

For example, FIGS. 14A and 14B show quadtree partitions for intra prediction using neighboring reference samples highlighted for one explicit leaf node of the primary subdivision, while FIG. 14B shows an improved reference. Residual quadtree subdivision for the same prediction leaf node with samples. All subblocks shown in FIG. 14B share the same inter prediction parameters included within the data stream for each leaf block highlighted in FIG. 14A. Thus, FIG. 14A shows an example for a conventional quadtree partition for intra prediction, where reference samples for one explicit leaf node are represented. However, according to one preferred embodiment of the invention, the individual intra prediction signal is residual, for example by using neighboring samples of leaf nodes already reconstructed in the residual tree, represented by gray shaded stripes in Fig. 4 (b). Calculated for each leaf node in the tree. Then, the reconstructed signal of a given residual leaf node is obtained in the usual way of adding the quantized residual signal to this prediction signal. This reconstructed signal is used as a reference signal for the next prediction process. The decoding order for prediction may be the same as the residual decoding order.

In the decoding process as shown in FIG. 15, for each residual leaf node, the prediction signal p is in accordance with the actual intra-prediction mode (as indicated by the prediction related Quartry leaf node) using reference samples r '. Is calculated.

After the SIT process,

The recovered signal r is calculated and stored for the following prediction calculation process:

The decoding order for prediction is the same as the residual decoding order shown in FIG.

Each residual leaf node is decoded as described in the previous paragraph. The recovered signal r is stored in a buffer as shown in FIG. From this buffer, reference samples r ' will be selected for the next prediction and decoding process.

Using a combination of different subsets of the aspects outlined above, detailed embodiments have been described with respect to FIGS. 1-16, while other embodiments of the present application have already been described in some particular aspects of focus, but which embodiments Some generalizations of the above-described embodiments are described.

In particular, the embodiments described above with respect to the framework of FIGS. 1 and 2 are primarily combined with many aspects of the present application, and the embodiments described above may also be advantageous when used in other applications or other coding fields. . As mentioned mainly during the discussion, for example, multitree subdivision can be used without merging and / or without interplanar adoption / prediction and / or without inheritance. For example, transmission of the largest block size, depth-first traversal order, contextual adaptation according to the hierarchical level of each subdivision flag, and transmission of the maximum hierarchical level in the bitstream to store the auxiliary information bit rate, all such The views are advantageous independently of each other. This is also true when considering the merging technique. Merging is advantageously independent of the exact way the picture is subdivided into simple connecting regions, and is advantageously independent of the use of interplanar adoption / prediction and / or inheritance or the presence of one or more sample arrangements. This applies equally to the advantages associated with inter-plane adoption / prediction and succession.

Thus, the embodiments described below generalize the embodiments already mentioned with respect to the aspects of merging. The following embodiments are generalizations of the above-described embodiments, and many of the details described above may be considered to be combined with the embodiments described below.

17 illustrates a decoder according to an embodiment of the present application. The decoder of FIG. 17 includes an extracting unit 600 and a restoring unit 602. The extractor 600 extracts for payloads from each data stream 604 of a plurality of simple connection regions in which an array of information samples representing a spatially sampled information signal is subdivided.

As described above, simple connection regions into which the array of information samples are subdivided may result from a multitree subdivision, and may be square or rectangular in shape. In addition, the embodiments described in detail for subdivision of the sample arrangement may be only detailed embodiments, and other subdivisions may also be used. Some possibilities are shown in FIGS. 18A-18C. For example, FIG. 18A illustrates non-overlapping treeblocks adjacent to each other with some of the sample arrays 606 subdivided according to subblocks 610 of different sizes. Subdivided into normal secondary arrays at (608). As discussed above, although quadtree subdivision is as shown in FIG. 18A, partitioning of each parent node within some other number of child nodes is also possible. 18B illustrates an embodiment in which the sample array is subdivided into subblocks of different sizes by direct multitree subdivision application over the entire pixel array 606. In other words, the entire pixel array 606 is treated as a treeblock. 18C shows another embodiment. According to this embodiment, the sample array may be structured as a regular secondary array of square or rectangular macroblocks adjacent to each other, each of which leaves a macroblock 612 unpartitioned. It is individually associated with the partition information according to whether it is partitioned into a regular secondary array of blocks of the indicated size or according to the partition information. As can be seen, all subdivisions of FIGS. 18A-18C illustrate subdividing the sample arrangement 606 into exemplary simple connection regions in accordance with the embodiment of FIGS. 13A-13C which are non-overlapping. However, some alternatives are possible. For example, blocks may overlap each other. However, overlapping may be limited in the range in which each block has a portion which is not overlapped by other neighboring blocks, or each sample of the blocks includes peripheral blocks arranged in juxtaposition of the current block along a predetermined direction. Maximum overlap by one block in between. The latter means that the current block can overlap with respect to the current block completely covered by the left and right neighboring blocks, but they do not overlay each other, which can equally apply for neighbors in the vertical and diagonal directions. .

As described above with respect to FIGS. 1-16, the arrangement of information samples does not necessarily represent a picture of a video or still picture. Sample arrangement 606 may also represent a depth map or transparency map of some scenes.

The payload data associated with each of the plurality of simple connection regions is, as already described above, an important map that identifies the residual domain or transform coefficients in the spatial domain and the locations of significant transform coefficients in the transform block corresponding to the residual block; Include the same translation domain. Generally speaking, the payload data extracted by the extractor 600 for each simple connection region from the data stream 604 is in the spatial domain or the spectral domain, and directly or for example, of its prediction. It is data that spatially describes its associated simple connection area as a residual to some sort.

The reconstruction unit 602 processes the payload data for the individual simple connection areas in a predetermined manner by coding parameters associated with the individual simple connection areas, for each simple connection area, thereby providing a simple connection area of the array of information samples. The array of information samples from the payload data for the user. Similar to the discussion above, the coding parameters may be prediction parameters, so that the simple connection regions shown in 18a and 18b may correspond to the aforementioned prediction blocks, i.e., the data stream 604 may predict the individual simple connection regions. And may correspond to blocks that are units that define prediction details. However, coding parameters are not limited to prediction parameters. The coding parameters may indicate a transformation used to transform the payload data or may be defined as a filter used for the restoration of simple connection regions individually when restoring the arrangement of information samples.

The extractor 600 identifies the simple connection regions within the plurality of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, for the predetermined simple connection region. Details of this step have been described above with respect to step 450. That is, in addition to the predetermined relative positional relationship, the confirmation may depend on a subset of the coding parameters associated with the predetermined simple connection area. After confirmation, the extractor 600 extracts a merge indicator for the predetermined simple connection area from the data stream 604. If the number of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area is greater than zero. This corresponds to the details of steps 452 and 454. If the merge indicator is proposed as a merge process of the predetermined block, the extractor 600 determines whether the number of simple connection regions having a predetermined relative positional relationship is predetermined to the predetermined simple connection region or whether the merge indicator is one of the predetermined simple connection regions. It is checked if the number of simple connection areas, with a predetermined relative positional relationship, is greater than one, wherein the coding parameters are identical to each other. If one of the two alternatives is applied, the extractor adopts or uses the coding parameters for the prediction of the predetermined simple connection region coding parameters or its remaining subset as described above in steps 458-468. As described above in FIG. 10, an additional indicator indicates that the latter checks are made when the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than one and have different coding parameters. Is extracted by the extractor 600 from the data stream 604.

By using the latter checks, the transmission of additional indicators indicating one or a subset of candidate simple connection regions can be suppressed, thereby reducing the auxiliary information overhead.

FIG. 19 shows a general structure of an encoder for generating a data stream decodable by the decoder of FIG. 17. The encoder of FIG. 19 includes a data generator 650 and an inserter 652. The data generation unit 650 may include a plurality of interconnect areas in which an array of information samples is subdivided according to associated coding parameters associated with the individual simple connection areas to indicate how payload data for the individual simple connection areas is restored. Code an array of information samples with payload data for each of the interconnected regions. The inserting unit 652 performs the above checking and checking as the extracting unit 600 of the decoder of FIG. 12, but performs the insertion of a merge indicator instead of the extracting thereof, and suppresses the insertion of coding parameters into the data stream. Or replace the insertion of coding parameters into the data stream entirely by the insertion of the individual prediction residuals in lieu of the adoption / prediction described above for each of FIGS. 12 and 10.

Furthermore, the structure of the encoder of FIG. 19 is somewhat schematic, and in fact, the determination of payload data, coding parameters and merging indicator may be an iterative procedure. For example, if the coding parameters of neighboring simple connection regions are similar but not identical to each other,

Abandoning the small differences between these coding parameters is iterative, given that the merge indicator completely suppresses one coding parameters of simple connection regions and allows all submissions of these parameters to be replaced only by the submission of a residual. It can be determined that the in procedure is preferable to signaling this difference to the decoder.

20 shows another embodiment for a decoder. The decoder of FIG. 20 includes a detail divider 700, a merger 702, and a reconstructor 704. The subdivision section includes a plurality of non-overlapping of different sizes by recursively multi-segmenting an array of samples representing spatial sampling of the two-dimensional information signal, depending on whether the syntax elements are included in the data stream. Subdivide into simple connection areas. As such, the multi-compartments may correspond to the embodiments described above with respect to FIGS. 1-16 or 18A, or with respect to FIG. 18B. Syntax elements included in the data stream for indicating subdivision may be defined as indicated above or in alternative ways with respect to FIGS. 6A and 6B.

The merger 702 separates a subset of the simple connection regions of the arrangement of information samples, the integration of which is a plurality of simple connection regions, according to a second subset of syntax elements in the data stream, separated from the first subset. And to combine spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision into the furnaces. The second subset of the just mentioned syntax elements, representing the merging information, may be defined in the same manner as previously described for FIGS. 19 and 10, respectively, or in other ways. However, the ability of an encoder to mark subdivisions by the use of subsets separated from the subsets to which the merge is directed may increase coding efficiency by increasing the freedom of the encoder to apply sample array subdivisions to the actual sample array contents. Can be. The reconstruction unit 704 is configured to reconstruct an array of samples from the data stream using intermediate subdivision. As mentioned above, the reconstruction section may utilize intermediate subdivision by the adoption / prediction of the coding parameters of the merging partners for the current simple connection area. Thus, the reconstruction unit 704 may apply a transform or prediction process to the combined region of the merged group of simple connection regions.

21 shows a possible encoder for generating a decodable data stream by the decoder of FIG. 15. The encoder includes a subdivision / merge stage 750 and a data stream generator 752. The subdivision / merge stage is configured to determine an intermediate subdivision of an array of information samples representing a spatial sampling of two separate subsets of simple connection regions, the two-dimensional information signal and its integration being a plurality of simple connection regions. The first subset of syntax elements, wherein the array of information samples is subdivided into a plurality of simple connection regions that are non-overlapping by recursive multi-compartment according to the first subset of syntax elements, and the first subset This intermediate detail by means of a second subset of syntax elements, to which the spatially neighboring simple connection regions of the plurality of simple connection regions are combined, to obtain an intermediate subdivision according to a second subset of syntax elements, separated from This is done by defining a partition. The data stream generator 752 uses intermediate subdivision to code the array of information samples into a data stream. Subdivision / merge stage 750 also inserts the first and second subsets into the data stream.

Again, as in the case of FIG. 14, the process of defining the first and second subsets of syntax elements generated by the data stream generator 752 may be a process that operates repeatedly. For example, the subdivision / merge stage 750 may predetermine the optimal subdivision, but the data stream generator 752 codes and merges the sample array by using the subdivision / merge stage using the sample subdivision. The auxiliary information overhead is reduced by determining the corresponding optimal set of syntax elements for setting the syntax element describing the. However, the process of encoding may not stop here. Rather, the subdivision / merge stage 750 together with the data stream generator 752 uses the data stream generator 752 to determine whether a better rate / distortion ratio can be obtained by utilizing the positive characteristics of the merge. Cooperate to deviate from the optimal setting of the subdivision and syntax elements previously determined.

As mentioned above, the generalization of the embodiments described with respect to FIGS. 1-16 above of the embodiments described with respect to FIGS. 17-21 is shown, and thus the elements of FIGS. 1-16 are shown in FIGS. 17-21. It is possible to uniquely associate with the described embodiments. For example, the extractor 102 assumes responsibility for the tasks to be performed by the extractor 600 of FIG. 17 together with the detail divider 104a and the merger 104b. The subdivision is responsible for subdividing and merging neighboring relationships between individual simple connection regions. In addition, the merger 104b manages merging of the simple connection regions into a group and in the case of a merge event represented by the merge information currently decoded, a suitable coding parameter to be used or copied as a prediction for the current simple connection regions Confirm them. Extractor 102 assumes responsibility for extracting the actual data from the data stream using an appropriate context when using entropy decoding for data extraction. The remaining elements of FIG. 2 are an embodiment of the restorer 602. Of course, the restoration unit 602 may be implemented differently from that shown in FIG. 2. For example, the reconstruction unit 602 may not use motion-compensated prediction and / or intra-prediction. Rather, other possibilities may apply as well.

Furthermore, the simple connection regions mentioned in connection with the description of FIG. 17 may be any of the prediction blocks already mentioned above, as previously indicated, or any of the other subdivisions mentioned above, such as residual subdivision or filter subdivision. Corresponds to

When comparing the encoder of FIG. 19 with the embodiment of FIG. 1, the data generator 650 will encompass all elements with the data stream inserter 18 and the latter will correspond to the inserter 652 of FIG. 19. . Again, the data generator 650 may use a coding approach different from the hybrid coding approach shown in FIG. 1.

When comparing the decoder of FIG. 20 with the embodiment of FIG. 2, subdivision 104a and merge 104b will correspond to subdivision 100 and merge 102 in FIG. 20, and the elements ( 106 and 114 will correspond to the restoration unit 704. The extraction unit 102 will be commonly involved in the functions of all the elements shown in FIG.

As long as the encoder of FIG. 21 is considered, the subdivision / merge stage 750 will correspond to the subdivision 104a and the merger 104b, and the data stream generator 752 will have all the other features shown in FIG. It will encompass the elements.

Although certain embodiments have been described in the context of apparatus, it is apparent that such aspects also represent corresponding methods, where a block or apparatus corresponds to a step of the method or a feature of the step of the method. Similarly, aspects described in the context of the steps of the method represent a description of the corresponding block or item or feature of the corresponding device. Some or all of the steps of the method may be performed by (or using) a hardware device, eg, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important steps may be performed by the device.

The encoded / compressed signals of the invention may be stored in a digital storage medium or transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Depending on the specific implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation has an electronically readable control signal therein, such as a digital storage medium, for example a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, in which each method is performed. It may be performed using a storage medium that cooperates (or can collaborate), such as a programmable computer system. Thus, the digital storage medium may be computer readable.

Some embodiments in accordance with the present invention include a date carrier having an electronically readable control signal, which may cooperate with a programmable computer system to allow the method described herein to be performed.

In general, embodiments of the present invention may be implemented as a computer program product having a program code, the program code may operate to perform one of the methods when the computer program product is performed on a computer. The program code is stored on a machine readable carrier, for example.

Other embodiments include a computer program that performs one of the methods described herein, which is stored on a machine readable carrier.

That is, an embodiment of the method of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein when the computer program is run on a computer.

Other embodiments of the method of the present invention are thus data carriers (or digital storage media, or computer-readable media), which record and include a computer program for performing one of the methods described herein.

Still other embodiments of the present invention are thus a series of signals representing a data stream or a computer program for performing one of the methods described herein. The data stream and the series of signals may be configured to be transported, for example, via a data communication connection, for example the Internet.

Another embodiment includes a programmable logic device, adapted to perform processing means, for example a computer, or one of the methods described herein.

Another embodiment includes a computer with a computer program installed that performs one of the methods described herein.

A decoder according to another embodiment of the present invention includes an extracting unit configured to extract payload data from a data stream for each of a plurality of simple connection regions in which an array of information samples representing a spatially sampled information signal is subdivided; For each simple connection area, the payload for the simple connection areas of the array of information samples by processing the payload data for the individual simple connection areas in a manner defined by the coding parameters associated with the individual simple connection area. A retrieval section configured to reconstruct an array of information samples from data, wherein the extracting section also includes a plurality of simpler having a predetermined relative positional relationship with respect to the predetermined simple connection region with respect to the predetermined simple connection region. Identify simple connection areas within connection areas And if the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, a merge indicator for a predetermined simple connection region can be extracted from the data stream, and If the merge indicator suggests the merged processing of the predetermined block, if the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is 1, the coding parameter for the predetermined simple connection region Coding of simple connection regions having the predetermined relative positional relationship with respect to the predetermined simple connection region by adopting coding parameters of the simple connection region as one of the embodiments, or extracting a prediction residual for a predetermined simple connection region from the data stream.Can be configured to predict coding parameters for the predetermined simple connection area from parameters.

Here, the extracting unit may also check if the coding parameters associated with the simple connection regions coincide with each other if the number of simple connection regions having the predetermined relative positional relationship with respect to the predetermined simple connection region is greater than one. And if so, adopt the coding parameters of the simple connection area as coding parameters for the predetermined simple connection area, or extract a prediction residual from the data stream to determine a predetermined relative positional relationship for the predetermined simple connection area. Can be configured to predict the coding parameters for the predetermined simple connection area from the coding parameters of the number of simple connection areas with a.

Here, the decoder spatially arranges the array of information samples into the simple connection regions by using multi-tree subdivision to recursively multi-partition such that the simple connection regions have different sizes. The apparatus may further include a sub-division configured to subdivide, and the extractor may be configured to process the simple connection regions in a depth-first traversal order.

Wherein the sub-division section spatially divides the arrangement of information samples into tree root regions of the largest region size, and recursively multi-compartments a subset of the tree root regions to thereby divide at least a subset of the tree root regions. Subdivide into smaller simple concatenated regions of different sizes, according to the multi-tree subdivision information, wherein the tree root regions are not partitioned according to the multi-tree subdivided information and the smaller simple concatenated regions are divided into a plurality of sub-concatenation information. Simple connection areas can be formed.

Here, the extracting unit, in identifying the simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, information samples adjacent to the information sample at the boundary of the predetermined simple connection region, if any It may be configured to identify such simple connection areas within it.

Here, the extracting unit may extract, from the data stream, a first subset of coding parameters for a predetermined simple connection region, and having a predetermined relative positional relationship with respect to the predetermined simple connection region, before the confirmation. In identifying simple connection regions, having a predetermined relative positional relationship with respect to the predetermined simple connection region, while at the same time meeting a predetermined relationship with respect to a first subset of coding parameters for the predetermined simple connection region; It is possible to identify such simple connection regions with associated coding parameters, and at the same time have a first subset of coding parameters for the predetermined simple connection region while having a predetermined relative positional relationship to the predetermined simple connection region. About Conditionally dependent on the number of simple connection regions having associated coding parameters that satisfy the determined relationship, the second subset of coding parameters for the predetermined simple connection region separated from the first subset of coding parameters. Can be configured to execute the adaptation or prediction by extracting the prediction residual.

Here, the extracting unit, from the data stream, for the predetermined simple connection region, if the coding parameters associated with the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region are different from each other, Identify a true subset of the number of simple connection regions having a predetermined relative positional relationship and adopt coding parameters of this appropriate subset as coding parameters for the predetermined simple connection region, or the data By extracting a prediction residual from the stream, it may be configured to extract a reference neighbor identifier that predicts the coding parameters of the predetermined simple concatenated region from the appropriate subset.

A decoder according to another embodiment of the present invention is configured to extract payload data from the data stream for each of a plurality of simple connection regions in which an array of information samples representing a spatially sampled information signal is subdivided into pieces. For each simple connection region, pay for simple connection regions of the arrangement of information samples by processing the payload data for the individual simple connection regions in a manner defined by the coding parameters associated with the respective simple connection region. A reconstruction section, configured to reconstruct an array of information samples from the load data, wherein the extracting section may also extract a first subset of coding parameters for the predetermined simple connection area from the data stream, and For a predetermined simple connection area, the above It is possible to identify simple connection regions in a plurality of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, and if the simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, If the number is greater than zero, a merge indicator for a predetermined simple connection region can be extracted from the data stream, and if the merge indicator presents merged processing of a predetermined block, according to a predetermined distance measure , For each of a plurality of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, the first subset of coding parameters of the predetermined simple connection region and the predetermined simple connection region for the predetermined simple connection region. Dogs with relative positional relationships Compute the distance between corresponding subsets of coding parameters of a simple connection area, and assign the corresponding subset of coding parameters of the simple connection area with the minimum distance to the predetermined simple connection area, separated from the first subset. The predetermined simple from the corresponding subset of coding parameters of the simple connected region having the minimum distance, either as a second subset of coding parameters for the simple connected region or by extracting a prediction residual for the predetermined simple connected region from the data stream. And predict a second subset of coding parameters for the connection region.

According to another embodiment of the present invention, a decoder for decoding a data stream having a two-dimensional information signal coded therein may be configured to perform recursive multi- according to a first subset of syntax elements included in the data stream. A sub-division configured to spatially subdivide the array of information samples representing spatial sampling of the two-dimensional information signal by the partition into a plurality of simple connection regions of different sizes, separated from the first subset According to the second subset of syntax elements in the data stream, the consolidation of the arrangement of information samples is a plurality of simple to obtain an intermediate subdivision into separate subsets of simple connection regions, the plurality of simple connection regions. The data using a merger and intermediate subdivision configured to combine the spatially neighboring simple connection regions of the connection regions. It may include a restoration that is configured to restore the arrangement of the information samples from the stream.

Wherein the sub-division section spatially partitions the array of information samples into tree root regions of maximum region size, and recursively multi-compartments a subset of the tree root regions, according to the first subset of syntax elements. Subdividing at least a subset of the tree root regions into smaller simple concatenated regions of different sizes, wherein the tree root regions are not partitioned according to the multi-tree subdivision information and the smaller simple concatenated regions are arranged in a plurality of simple concatenations. Connection areas can be formed.

Wherein the sub-division is a tree of the array of information samples such that the tree root regions are rectangular blocks arranged regularly to cover the array of information samples without gaps of a size determined by the maximum region size. It may be configured to perform division into root regions.

Here, the sub-division section may check, for each tree root region, the first subset of the syntax elements as to whether the individual tree root region is to be partitioned in subdividing the subset of tree root regions. If an individual tree root region is to be partitioned, the individual tree root region can be partitioned into regions of the first hierarchical level according to the partition rules associated with the first hierarchical level, and higher- Recursively repeat the check and partition over the areas of the first hierarchical level to obtain areas of the hierarchical levels of order, and there is no additional partition to be performed according to the first subset of syntax elements, or the maximum Stopping the recursive iteration upon reaching a hierarchical level and further tree not further partitioned according to the first subset of syntax elements Region of the subset of bit areas, the multi-leaf tree may represent respectively areas and smaller areas of the simple connection details division.

Here, the maximum hierarchical level may be indicated by a first subset of the syntax elements.

Wherein the sub-division performs partitioning into sub-regions of the same size, according to partition rules associated with the first and higher-order hierarchical levels, so that the number of sub-regions is all It can be obtained to be common to hierarchical levels.

Here, the reconstruction unit may be configured to extract first syntax elements associated with leaf regions of the subset of tree root regions in a depth-first traversal order from the data stream.

Wherein the first subset of syntax elements has a partition indication flag associated with each tree root region and the regions of the first and higher-order hierarchical levels each do not belong to regions of the maximum hierarchical level; Partition indication flags may indicate whether the associated tree root area and the area of the first and higher-order hierarchical level are partitioned, respectively.

Wherein the reconstruction unit determines which prediction mode to use below measures, at least intra and inter prediction modes, with granularity corresponding to the intermediate subdivision, conversion from spectral to spatial domain, One or more of setting and / or performing parameters for inter-prediction, setting and / or performing parameters for intra prediction.

Here, the reconstruction unit determines the following measures, at least one of intra and inter prediction modes, granularity dependent on the intermediate subdivision, conversion from spectral domain to spatial domain, and parameters for inter-prediction. And / or perform parameters, set and / or perform parameters for intra prediction, extract and set coding parameters, extract coding parameter residuals, extract coding parameter predictions, and detail the coding parameter prediction and small simple connected regions. It may be configured to perform one or more of setting coding parameters by combining the coding parameter predictions and coding parameter residuals with a more granular granularity determined by partitioning.

A decoding method according to another embodiment of the present invention comprises extracting payload data from a data stream for each of a plurality of simple connection regions in which an array of information samples representing a spatially sampled information signal is subdivided. For a simple connection area, the payload data for the simple connection areas of the array of information samples by processing the payload data for the individual simple connection areas in a manner defined by the coding parameters associated with the respective simple connection area. Recovering the arrangement of information samples from the plurality of simple connecting regions, with respect to the predetermined simple connecting region, having a predetermined relative positional relationship with respect to the predetermined simple connecting region. Identifying simple connection areas in the Extracting a merge indicator for a predetermined simple connection area from the data stream if the number of simple connection areas having a predetermined relative positional relationship with respect to the predetermined simple connection area is greater than zero; When presenting the merged processing of, if the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is 1, coding of the simple connection region as coding parameters for the predetermined simple connection region Adopt parameters or extract a prediction residual for a predetermined simple connection region from the data stream to advance from coding parameters of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region. Predicting the coding parameters for the determined simple connection area.

According to yet another embodiment of the present invention, a method of decoding a data stream having a two-dimensional information signal coded therein may be provided according to a first subset of syntax elements included in the data stream. Spatially subdividing an array of information samples representing spatial sampling of a two-dimensional information signal by partitioning into a plurality of simple connected regions of different sizes, separated from the first subset, the syntax in the data stream According to the second subset of elements, the integration of the array of information samples, the consolidation of the plurality of simple connection regions, to obtain an intermediate subdivision of the simple connection regions into the separate subsets, to obtain a spatial separation of the plurality of simple connection regions. Combining neighboring simple connection areas into the information stream from the data stream using intermediate subdivision; It may include the step of restoring of the array.

According to another embodiment of the present invention, an array of information samples representing a spatially sampled information signal is allocated to payload data and individual simple connection areas for each of the plurality of simple connection areas where the array of information samples is subdivided. Encoded with associated coding parameters, the encoder configured to define how the payload data for an individual simple connection region is recovered from the payload data for the individual simple connection region, for the predetermined simple connection region, the predetermined simple connection It is possible to identify simple connection regions in a plurality of simple connection regions having a predetermined relative positional relationship with respect to the region, and if the number of simple connection regions having a predetermined relative positional relation with respect to the predetermined simple connection region is less than zero. If large, the predetermined simple connection area Can insert a merge indicator for the into the data stream, and if the merge indicator suggests merged processing of a predetermined block, if the merge indicator of the simple connection region has a predetermined relative positional relationship to the predetermined simple connection region If the number is 1, the coding parameters of the predetermined simple connection area are not inserted into the data stream or a prediction residual for a predetermined simple connection area is inserted into the data stream to determine a predetermined simple connection area for the predetermined simple connection area. It may be configured to predict the coding parameters for the predetermined simple connection area from the coding parameters of the simple connection areas having a relative positional relationship.

Here, the encoder checks whether the coding parameters associated with the simple connection regions coincide with each other if the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than one, and if so , By inserting coding parameters of the predetermined simple connection area into the data stream, or inserting a prediction residual into the data stream, so that the simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area. It may be configured to predict coding parameters for a predetermined simple connection area from the number of coding parameters.

An encoder for generating a data stream coded with a two-dimensional information signal according to another embodiment of the present invention is an information sample representing a spatial sampling of the two-dimensional information signal by regression multi-compartment. Of the first subset of syntax elements and the arrangement of the information samples, the first subset of syntax elements defining spatially subdividing the arrangement of the data into a plurality of simple connection regions of different sizes; Determine a second subset of syntax elements separated from the first subset defining a combination of spatially neighboring simple connection regions of a plurality of simple connection regions to obtain an intermediate subdivision into separation subsets Inserting into the data stream a subdivision / merge stage configured to subdivide and the first and second subsets of the syntax elements The intermediate subdivision may include a data stream generator configured to code the array of information samples into a data stream.

According to another embodiment of the present invention, an array of information samples representing a spatially sampled information signal is allocated to payload data and individual simple connection areas for each of the plurality of simple connection areas where the array of information samples is subdivided. The encoder, configured to encode with associated coding parameters, to specify how the payload data for the individual simple connection area is recovered from the payload data for the individual simple connection area, comprises: a first of the coding parameters for the predetermined simple connection area; A subset can be inserted into the data stream, and with respect to a predetermined simple connection area, simple connection areas within a plurality of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area can be identified and , For a predetermined simple connection area If the number of simple connection regions having a predetermined relative positional relationship is greater than zero, a merge indicator for the predetermined simple connection region may be inserted into the data stream, and the merge indicator may perform merged processing of the predetermined block. If present, for each of the simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region according to a predetermined distance measure, a first portion of the coding parameters of the predetermined simple connection region Compute the distance between a corresponding subset of coding parameters of an individual simple connection region having a predetermined relative positional relationship with respect to the set and the predetermined simple connection region, and separate a predetermined simple connection region from the first subset. Image a second subset of coding parameters for Coding parameters for the predetermined simple connection area from the corresponding subset of coding parameters of the simple connection area having a minimum distance by not inserting into the data stream or inserting a prediction residual for the predetermined simple connection area into the data stream. Can be configured to predict a second subset of these.

According to another embodiment of the present invention, an array of information samples representing a spatially sampled information signal is applied to payload data and individual simple connection areas for each of the plurality of simple connection areas where the array of information samples is subdivided. An encoding method configured to encode with associated coding parameters to specify how payload data for an individual simple connection area is recovered from payload data for an individual simple connection area, for a predetermined simple connection area, the predetermined method. Identifying the simple connection regions in the plurality of simple connection regions having a predetermined relative positional relationship to the simple connection region, if the number of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region If greater than zero, the predetermined simple connection Inserting a merge indicator for a region into the data stream, where the merge indicator suggests a merged processing of a predetermined block, if the merge indicator has a predetermined relative positional relationship to a predetermined simple connected region; If the number is 1, the coding parameters of the predetermined simple connection area are not inserted into the data stream or a prediction residual for a predetermined simple connection area is inserted into the data stream to determine a predetermined simple connection area for the predetermined simple connection area. Predicting the coding parameters for the predetermined simple connection area from the coding parameters of the simple connection areas having a relative positional relationship.

According to another embodiment of the present invention, a method of generating a data stream coded with a two-dimensional information signal includes: an information sample representing spatial sampling of the two-dimensional information signal by recursive multi-compartment; Of the first subset of syntax elements and the arrangement of the information samples, the first subset of syntax elements defining spatially subdividing the arrangement of the data into a plurality of simple connection regions of different sizes; Determining a second subset of syntax elements separated from the first subset defining a combination of spatially neighboring simple connection regions of a plurality of simple connection regions to obtain an intermediate subdivision into separation subsets. And using the intermediate subdivision to insert the first and second subsets of syntax elements into a data stream. Wherein the array of samples may include the step of coding the data stream.

A computer-readable digital storage medium according to another embodiment of the present invention, when operating in a computer, a computer program having a program code for performing any one of the above-described decoding method, encoding method and data stream generating method Can be saved.

According to yet another embodiment of the present invention, a data stream having an array of encoded information samples representing a spatially sampled information signal is provided in each of a plurality of simple connection regions in which the array of information samples is subdivided. Payload data for each simple connection region, and coding parameters associated with the individual simple connection regions, to define how the payload data for the individual simple connection regions are recovered from the payload data for the individual simple connection regions, and the data The stream may also be a merge indicator for a predetermined simple connection area, wherein the number of simple connection areas in the plurality of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection areas is greater than zero, if the predetermined The merge directive for the determined simple connection region is merged. Presenting, and if the number of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region is 1, the absence of coding parameters of the respective predetermined simple connection region in the data stream, Or for an individual predetermined simple connection region for reconstruction by prediction of coding parameters for the individual predetermined simple connection region from the coding parameters of the simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region. The prediction residual may further include.

Here, the data stream is a separate preview if the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than 1 and if the coding parameters associated with the simple connection regions coincide with each other. Prediction of coding parameters for an individual predetermined simple connection area from coding parameters of the absence of coding parameters for the determined simple connection area, or the number of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area. The method may further include a prediction residual for an individual predetermined simple connection area for reconstruction.

According to another embodiment of the present invention, a data stream in which a two-dimensional information signal is coded may be arranged using different arrangements of information samples representing spatial sampling of the two-dimensional information signal by regression multi-compartment. The integration of the simple connection regions of the arrangement of information samples, separated from the first subset and a first subset of syntax elements, that defines spatially subdivided into a plurality of simple connection regions of sizes A second subset of syntax elements, defining a combination of spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision into a separate subset that is simple connection regions, the information An array of samples may be coded into the data stream according to the intermediate subdivision.

In some embodiments, a programmable logic device (eg, field programmable gate arrangement) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate arrangement may perform one of the methods described herein in cooperation with a microprocessor. In general, the method may be preferably performed by any hardware apparatus.

The above-described embodiments merely illustrate the spirit of the present invention. It should be understood that modifications and variations to the arrangement and details described herein will be apparent to those skilled in the art. It is intended that it be limited only by the scope of the following claims and not by the specific details expressed by the method of description or description of the above-described embodiments.

Claims (30)

An extracting unit configured to extract payload data from a data stream for each of the plurality of simple connection regions into which the array of information samples representing the spatially sampled information signal is subdivided; And
For each simple connection area, the payload for the simple connection areas of the array of information samples by processing the payload data for the individual simple connection areas in a manner defined by the coding parameters associated with the individual simple connection area. A reconstruction section, configured to reconstruct an array of information samples from the data,
The extraction unit also,
For a predetermined simple connection area, identify simple connection areas in a plurality of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, extracting a merge indicator for the predetermined simple connection region from the data stream,
When the merge indicator suggests merged processing of a predetermined block,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is 1, adopt coding parameters of the simple connection region as coding parameters for the predetermined simple connection region, or the data stream Extracts a prediction residual for a predetermined simple connection region from the to predict coding parameters for the predetermined simple connection region from coding parameters of the simple connection regions having the predetermined relative positional relationship with respect to the predetermined simple connection region. A decoder, configured to. The method according to claim 1,
The extraction unit also,
If the number of simple connection regions having the predetermined relative positional relationship with respect to the predetermined simple connection region is greater than 1, it is checked whether the coding parameters associated with the simple connection regions coincide with each other, and if so, the predetermined The number of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region by adopting the coding parameters of the simple connection region as coding parameters for the simple connection region or by extracting a prediction residual from the data stream. And predict the coding parameters for a predetermined simple connection region from the coding parameters of. The method according to claim 1,
A detail configured to spatially subdivide the arrangement of information samples into the simple connection regions by multi-partitioning such that the simple connection regions recursively have different sizes, using multi-tree subdivision. Further comprising a divider,
And the extractor is configured to process the simple connection regions in a depth-first traversal order. The method according to claim 1,
The sub-division may multiply at least a subset of the tree root regions by spatially dividing the array of information samples into tree root regions of maximum region size, and recursively multi-segmenting a subset of tree root regions. Subdivide into smaller simple concatenated regions of different sizes according to tree subdivision information, wherein the tree root regions are not partitioned according to the multi-tree subdivided information and the smaller simple concatenated regions are arranged in a plurality of simple concatenated regions; A decoder that forms regions. The method according to claim 1,
The extracting unit, in identifying simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, includes, if any, information samples adjacent to the information sample at the boundary of the predetermined simple connection region. And a decoder configured to identify such simple connection areas within. The method according to claim 1,
The extraction unit,
Before the verification, extract a first subset of coding parameters for a predetermined simple connection region from the data stream,
In identifying simple connecting regions having a predetermined relative positional relationship with respect to the predetermined simple connecting region, having a predetermined relative positional relation with respect to the predetermined simple connecting region and simultaneously with the predetermined simple connecting region Identify those simple connection regions having associated coding parameters that meet a predetermined relationship to a first subset of coding parameters for, and
A simple connection region having a predetermined relative positional relationship with respect to the predetermined simple connection region while simultaneously having associated coding parameters that satisfy a predetermined relationship with respect to a first subset of coding parameters for the predetermined simple connection region; Conditionally depending on the number of bits, to perform the adaptation or prediction by extracting a prediction residual for a second subset of coding parameters for the predetermined simple connected region, separated from the first subset of coding parameters. Configured, decoder. The method according to claim 1,
The extraction unit,
If the coding parameters associated with the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region are different from each other, from the data stream, a predetermined relative positional relationship with respect to the predetermined simple connection region is determined. Identify a true subset of the number of simple connection regions with and adopt the coding parameters of the appropriate subset as coding parameters for the predetermined simple connection region, or extract a prediction residual from the data stream Thereby extracting a reference neighbor identifier that predicts coding parameters of the predetermined simple concatenated region from the appropriate subset. An extraction unit configured to extract payload data from the data stream for each of a plurality of simple connection regions in which an array of information samples representing a spatially sampled information signal is subdivided;
For each simple connection area, the payload for the simple connection areas of the array of information samples by processing the payload data for the individual simple connection areas in a manner defined by the coding parameters associated with the individual simple connection area. A reconstruction section, configured to reconstruct an array of information samples from the data,
The extraction unit also,
Extract a first subset of coding parameters for the predetermined simple connection region from the data stream,
Identify, with respect to the predetermined simple connection region, simple connection regions within a plurality of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, extracting a merge indicator for the predetermined simple connection region from the data stream,
When the merge indicator suggests merged processing of a predetermined block,
According to a predetermined distance measure, for each of a plurality of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, a first subset of coding parameters of the predetermined simple connection region and Compute the distance between corresponding subsets of coding parameters of the individual simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, and calculate the corresponding subset of coding parameters of the simple connection region with the minimum distance, Has a minimum distance by adopting as a second subset of coding parameters for the predetermined simple connected area, separated from the first subset, or by extracting a prediction residual for the predetermined simple connected area from the data stream. Coding Parameters for Simple Connection Regions Configured to estimate the second subset of the coding parameters for the predetermined simply connected region from the emitters of the corresponding subset of the decoder. A decoder for decoding a data stream in which a two-dimensional information signal is coded,
According to a first subset of syntax elements included in the data stream, an array of information samples representing spatial sampling of a two-dimensional information signal by regression multi-compartment is spatially divided into a plurality of simple connection regions of different sizes. A sub-division configured to subdivide into;
According to a second subset of syntax elements in the data stream, separate from the first subset, the integration of the arrangement of information samples into separate subsets of simple connection regions, wherein the integration is a plurality of simple connection regions. A merger configured to combine spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision; And
And a reconstruction section configured to reconstruct the array of information samples from the data stream using intermediate subdivision. The method according to claim 8,
The sub-division section spatially partitions the arrangement of information samples into tree root regions of maximum region size, and recursively multi-compartments a subset of tree root regions, according to the first subset of syntax elements, the tree Subdividing at least a subset of root regions into smaller simple concatenated regions of different sizes, wherein the tree root regions are not partitioned according to the multi-tree subdivision information and the smaller simple concatenated regions are arranged in a plurality of simple concatenated regions. To form them. The method according to claim 10,
The sub-division is a tree root region of the array of information samples such that the tree root regions are rectangular blocks arranged regularly to cover the array of information samples without gaps of a size determined by the maximum region size. A decoder configured to perform splitting into channels. The method according to claim 10,
The sub-division may further include subdividing a subset of the tree root regions.
For each tree root region, check the first subset of the syntax elements as to whether the individual tree root region will be partitioned,
If individual tree root regions will be partitioned,
Partition the individual tree root region into regions of a first hierarchical level according to a partition rule associated with a first hierarchical level,
Iteratively repeating the check and partition for the regions of the first hierarchical level to obtain regions of higher-order hierarchical levels using the associated partition rule,
Stop the recursive iteration if there is no additional partition to be performed according to the first subset of syntax elements, or if the maximum hierarchical level is reached,
Regions of the subset of tree root regions that are not further partitioned according to the first subset of syntax elements, each representing leaf regions of a multi-tree subdivision and smaller simple concatenated regions, respectively. The method according to claim 12,
And the maximum hierarchical level is indicated by the first subset of syntax elements. The method according to claim 12,
The sub-division is,
According to the partition rule associated with the first and higher-order hierarchical levels, partition into sub-regions of the same size, so that the number of sub-regions is common for all hierarchical levels. A decoder, obtained such that. The method according to claim 12,
And the reconstruction portion is configured to extract first syntax elements associated with leaf regions of a subset of tree root regions in a depth-first traversal order from the data stream. The method according to claim 12,
The first subset of syntax elements has a partition indication flag associated with each tree root region and the region of the first and higher-order hierarchical levels does not belong to regions of the maximum hierarchical level, respectively; Flags indicate whether the associated tree root region and the region of the first and higher-order hierarchical level are each partitioned. The method according to claim 9,
The reconstruction section measures below with granularity corresponding to the intermediate subdivision:
Determining which prediction mode to use, at least in intra and inter prediction modes;
Conversion from spectral to spatial domain;
Setting and / or performing parameters for inter-prediction;
Set and / or perform parameters for intra prediction
A decoder configured to perform one or more of the. The method according to claim 9,
The restoring unit has the following measures as granularity depending on the intermediate subdivision:
Determining which prediction mode to use, at least in intra and inter prediction modes;
Conversion from spectral domain to spatial domain;
Setting and / or performing parameters for inter-prediction;
Setting and / or performing parameters for intra prediction;
Extraction and setting of coding parameters;
Extract coding parameter residuals to extract coding parameter predictions and set coding parameters by combining the coding parameter predictions and coding parameter residuals with a more granular granularity determined by the coding parameter prediction and subdivision of small simple connected regions. To do;
A decoder configured to perform one or more of the. Extracting payload data from the data stream for each of the plurality of simple connection regions into which the array of information samples representing the spatially sampled information signal is subdivided; And
For each simple connection area, the payload for the simple connection areas of the array of information samples by processing the payload data for the individual simple connection areas in a manner defined by the coding parameters associated with the individual simple connection area. Recovering an array of information samples from the data,
The extracting step,
Identifying, for a predetermined simple connection region, simple connection regions in a plurality of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, extracting a merge indicator for the predetermined simple connection region from the data stream,
When the merge indicator suggests merged processing of a predetermined block,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is 1, adopt coding parameters of the simple connection region as coding parameters for the predetermined simple connection region, or the data stream Extracting a prediction residual for a predetermined simple connection region from the predicting coding parameters for the predetermined simple connection region from the coding parameters of the simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region. Including, decoding method. A method of decoding a data stream in which a two-dimensional information signal is coded,
According to a first subset of syntax elements included in the data stream, an array of information samples representing spatial sampling of a two-dimensional information signal by regression multi-compartment is spatially divided into a plurality of simple connection regions of different sizes. Subdividing into;
According to a second subset of syntax elements in the data stream, separate from the first subset, the integration of the arrangement of information samples into separate subsets of simple connection regions, wherein the integration is a plurality of simple connection regions. Combining spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision; And
Restoring the arrangement of information samples from the data stream using intermediate subdivision. An array of information samples representing a spatially sampled information signal is encoded by encoding payload data for each of the plurality of simple connection regions into which the array of information samples is subdivided, and the coding parameters associated with the individual simple connection regions, An encoder configured to specify how payload data for a simple connection area is recovered from payload data for an individual simple connection area.
The encoder also,
For a predetermined simple connection area, identify simple connection areas in a plurality of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, inserting a merge indicator for the predetermined simple connection region into the data stream,
When the merge indicator suggests merged processing of a predetermined block,
If the number of simple connection areas having a predetermined relative positional relationship with respect to the predetermined simple connection area is 1, then the coding parameters of the predetermined simple connection area are not inserted into the data stream or a predetermined predetermined connection into the data stream. Insert a prediction residual for a simple connection area to predict coding parameters for a predetermined simple connection area from coding parameters of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area. The method according to claim 21,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than 1, it is checked whether the coding parameters associated with the simple connection regions coincide with each other, and if so, the predetermined simple Coding parameters of the number of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region by not inserting coding parameters of a connection region into the data stream or inserting a prediction residual into the data stream. An encoder configured to predict coding parameters for a predetermined simple connection area from a. An encoder for generating a data stream in which a two-dimensional information signal is coded,
A first subset of syntax elements defining spatially subdividing an array of information samples representing spatial sampling of a two-dimensional information signal by regressive multi-compartment into a plurality of simple connected regions of different sizes; and The consolidation of the arrangement of information samples allows a combination of the spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision into separate subsets of simple connection regions, the plurality of simple connection regions. A subdivision / merge stage configured to determine a second subset of syntax elements that are separated from the first subset; And
And a data stream generator configured to code the array of information samples into a data stream using the intermediate subdivision by inserting the first and second subsets of syntax elements into the data stream. An array of information samples representing a spatially sampled information signal is encoded by encoding payload data for each of the plurality of simple connection regions into which the array of information samples is subdivided, and the coding parameters associated with the individual simple connection regions, An encoder configured to specify how payload data for a simple connection area is recovered from payload data for an individual simple connection area.
The encoder also,
Inserting a first subset of coding parameters for a predetermined simple connection area into the data stream,
For a predetermined simple connection area, identify simple connection areas in a plurality of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, inserting a merge indicator for the predetermined simple connection region into the data stream,
When the merge indicator suggests merged processing of a predetermined block,
According to a predetermined distance measure, for each of the simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region, a first subset of the coding parameters of the predetermined simple connection region and a predetermined Compute the distance between corresponding subsets of coding parameters of individual simple connection regions having a predetermined relative positional relationship to the simple connection region, and separate the coding parameters for the predetermined simple connection region, separated from the first subset. The predetermined simple from the corresponding subset of coding parameters of a simple connected region having a minimum distance by not inserting a second subset into the data stream or by inserting a prediction residual for the predetermined simple connected region into the data stream For connection areas An encoder configured to predict a second subset of coding parameters. An array of information samples representing a spatially sampled information signal is encoded by encoding payload data for each of the plurality of simple connection regions into which the array of information samples is subdivided, and the coding parameters associated with the individual simple connection regions, An encoding method configured to specify how payload data for a simple connection area is to be recovered from payload data for an individual simple connection area.
The encoding method is
Identifying, for a predetermined simple connection region, simple connection regions in a plurality of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region,
If the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is greater than zero, inserting a merge indicator for the predetermined simple connection region into the data stream;
When the merge indicator suggests merged processing of a predetermined block,
If the number of simple connection areas having a predetermined relative positional relationship with respect to the predetermined simple connection area is 1, then the coding parameters of the predetermined simple connection area are not inserted into the data stream or a predetermined predetermined connection into the data stream. Inserting a prediction residual for a simple connection area to predict coding parameters for a predetermined simple connection area from coding parameters of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection area. , Encoding method. A method of generating a data stream having two-dimensional information signals coded thereon,
A first subset of syntax elements defining spatially subdividing an array of information samples representing spatial sampling of a two-dimensional information signal by regressive multi-compartment into a plurality of simple connected regions of different sizes; and The consolidation of the arrangement of information samples defines a combination of spatially neighboring simple connection regions of the plurality of simple connection regions to obtain an intermediate subdivision of the simple connection regions into separate subsets of the plurality of simple connection regions. Determining a second subset of syntax elements separated from the first subset; And
Coding the array of information samples into the data stream using the intermediate subdivision by inserting the first and second subsets of syntax elements into a data stream. A computer readable digital storage medium storing a computer program having a program code for performing a method according to any one of claims 19, 20, 25, and 26 when operating in a computer. A data stream having an array of encoded information samples representing a spatially sampled information signal, the data stream comprising payload data for each of a plurality of simple connection regions into which the array of information samples is subdivided, and the respective simple Including the coding parameters associated with the connection region, specifying how the payload data for the individual simple connection region is recovered from the payload data for the individual simple connection region,
The data stream may also be
A merge indicator for a predetermined simple connection area, wherein the number of simple connection areas in the plurality of simple connection areas having a predetermined relative positional relationship to the predetermined simple connection areas is greater than zero,
If the merge indicator for the predetermined simple connection region presents merged processing, and if the number of simple connection regions having a predetermined relative positional relationship with respect to the predetermined simple connection region is 1, in the data stream Absence of coding parameters of an individual predetermined simple connection region, or coding parameters for an individual predetermined simple connection region from coding parameters of simple connection regions having a predetermined relative positional relationship to the predetermined simple connection region. Further comprising prediction residuals for the respective predetermined simple concatenated regions for reconstruction by prediction. The method according to claim 28,
If the number of simple connection areas having a predetermined relative positional relationship with respect to the predetermined simple connection area is greater than 1, and the coding parameters associated with the simple connection areas match each other, coding for the individual predetermined simple connection area A separate preset for reconstruction by prediction of coding parameters for an individual predetermined simple connection region from coding parameters of the absence of parameters, or a number of simple connection regions having a predetermined relative positional relationship to said predetermined simple connection region. Further comprising a prediction residual for the determined simple connection area. A data stream into which a two-dimensional information signal is coded,
A first subset of syntax elements defining spatially subdividing an array of information samples representing spatial sampling of a two-dimensional information signal by regressive multi-compartment into a plurality of simple connection regions of different sizes, and
Of the simple connection regions of the arrangement of information samples, separated from the first subset, the consolidation is a spatial neighbor of the plurality of simple connection regions to obtain intermediate subdivision into a separate subset, which is a plurality of simple connection regions. A second subset of syntax elements, defining a combination of simple connection regions
The array of information samples is coded into the data stream according to the intermediate subdivision.

Images