JP5456907B2 - Texture pattern adaptive division block transform - Google Patents

Description

  The present invention is made in the technical field of image encoding. More particularly, the present invention relates to encoding an image block of an image using a partitioned block transform.

  To encode an image, it is well known in the art to transform image blocks of an image using discrete cosine transforms (DCT) or discrete wavelet transforms (DWT). .

  For example, DCT, commonly used in hybrid video coding, exploits redundancy in the spatial domain, exhibits excellent energy compression for highly correlated signals, is separable, symmetric and Indicates an attribute called orthogonal.

  Current image / video coding standards typically use a two-dimensional separable DCT or DWT for coding, whose basis functions can be generated by multiplying a horizontally oriented 1D basis function by the same set of vertically oriented functions.

  For example, for an exemplary N × N block, first, N N-point 1D DCT operations (first 1D transform) are performed on the original pixel in the vertical direction and N × N intermediate coefficients Second, N N-point 1D DCT operations (second 1D transform) are performed laterally on the intermediate coefficients to produce further transformed coefficients. While this type of approach tends to work well for image texture details that are strictly oriented in the horizontal or vertical direction, there is a tendency for ringing artifacts to appear around other edge orientations. This significantly degrades visual quality.

  This problem is identified and addressed by Non-Patent Document 1. In this paper, the authors propose a direction-adaptive partitioned block transform (DA-PBT). According to this, the image block is divided into directional partitions. The partition boundary is aligned with the direction of the first 1D transformation, and the length of the second 1D transformation is limited so that it does not extend across the partition boundary. This ensures that the required maximum length for 1D conversion is equal to N.

C. L. Chang and B. Girod, "Directional-Adaptive Partitioned Block Transform for Image Coding", ICIP 2008

  Although the maximum length required for 1D conversion is limited, DA-PBT proposed by Non-Patent Document 1 can perform more than N first 1D conversions or more than N second 1D conversions. You may need it. Furthermore, for application of DA-PBT, a direction needs to exist in an image block.

  The inventor of the present invention accumulates the pixels of the first partition of at least a portion of the image block by applying an invertible mapping associated with the texture pattern to the pixels of the first partition, A first partition is required by partitioning the image block according to a current texture pattern to which an invertible mapping associated with the texture pattern is associated. It was recognized that the maximum number of 1D conversions can be limited to N, and the maximum number of required second 1D conversions can also be limited to N.

  This maximum required 1D conversion limitation allows more efficient implementation on hardware and improved encoding performance, but pixel accumulation not only achieves this limitation, but also multiple strips Further partitioning based on a texture pattern having at least one of a texture pattern with a highly asymmetric pixel distribution and a non-directional texture pattern is also allowed.

  Accordingly, a method and apparatus for encoding an image block of an image is proposed. The method accumulates pixels of the first partition of at least a portion of an image block by applying a reversible mapping associated with a texture pattern to the pixels of the first partition, wherein the first partition is the Resulting from partitioning the image block according to the current texture pattern to which the invertible mapping associated with the texture pattern is associated, and for allowing data to be reconstructed or obtained from the current texture pattern Including entropy encoding.

  In an embodiment where the image block includes N times M pixels, where N is greater than or equal to M, the proposed encoding method uses the at least a portion of the image block as a current texture pattern adaptation associated with a current texture pattern. Partitioning the first subset of pixels according to the partition into a first subset of pixels and at least a second subset of pixels using a scan order associated with the texture pattern; A scan sequence associated with the texture pattern is associated with the current texture pattern, and each of the pixel sequences includes at most N pixels, and at most M More additional pixel sequences Each of said further pixel sequences comprises either one of said pixel sequences or a concatenation of pixel sequences of sub-sequences of said metasequence, and said further pixel sequence Each of the sequences includes at most N pixels, generating a coefficient sequence by transforming the further pixel sequence, and generating a further coefficient sequence by a one-dimensional transformation across the coefficient sequence Generating a further coefficient sequence by quantizing and scanning the coefficients contained in the further coefficient sequence, wherein the scanning is performed according to an encoding scan order; Entropy encoding a process, and entropy encoding an index associated with the current texture pattern, the index being a reference that allows acquisition or reconstruction of the current texture pattern; and including.

  In the exemplary embodiment, the method is adapted such that the current texture pattern adaptive partition may be associated with a texture pattern that includes multiple strips, asymmetric texture patterns, or non-directional textures. The

  The at least second subset of pixels may be treated in the same manner as the first subset of pixels, but the encoding scan order may be different. Further, coefficient prediction between the first subset and the at least second subset may be applied.

  The at least part of the partitioning may be a result of edge detection, segmentation or intensity distribution classification.

  The invention further proposes a method for decoding an encoded image block of an image. The image block includes N times M pixels, where N is greater than or equal to M. The method decodes a first subset of at least a portion of the image block, (a) entropy decodes an encoded sequence of first coefficients, and (b) relates to a current texture pattern. Entropy-decode the encoded index, and use the index to generate the current texture pattern either by acquisition from memory or reconstruction based on the index, and (c) using the decode scan order Arranging the first coefficients in a meta-sequence of at most M coefficient sequences, each of the at most M coefficient sequences including at most N first coefficients; (d) in step (c) Dequantizing the first coefficient before, during or after arranging the first coefficient, and (e) at most M inverses Generating at most M further coefficient sequences by a one-dimensional transformation across the quantized coefficient sequences; and (f) at most M first pixel sequences by transforming the at most M further coefficient sequences. And (g) distributing the generated pixels in the image block using a distribution pattern associated with the current texture pattern.

  In one embodiment, the decoding method further includes at most M by applying steps (a), (b), (c), (d), (e) and (f) to the second sequence of coefficients. Decoding a second subset of the at least portion of the encoded image block by generating a second pixel sequence of the encoded image block and using the distribution pattern to determine the remainder of the at least portion of the image block. Filling with at most M generated pixels of the second pixel sequence. The remainder is an empty remaining portion of the at least part of the image block after generation of the first subset.

  A corresponding device for decoding a first subset of at least a portion of the encoded image block of the image is then proposed. The at least part is encoded using a texture pattern adaptive split block transform, the image block including N times M pixels, where N is greater than or equal to M. The decoding apparatus is decoding means for entropy decoding the encoded sequence of the first coefficient, and the decoding means entropy decodes the encoded index related to the current texture pattern, and Using means adapted to generate the current texture pattern either by acquisition from memory or reconstruction based on the index, and using the decode scan order, the first coefficient at most First arrangement means adapted to arrange into a metasequence of M coefficient sequences, wherein each of the at most M coefficient sequences includes at most N first coefficients; and Through a one-dimensional transformation through inverse quantization means for inversely quantizing the coefficients of the image and at most M dequantized coefficient sequences. Means for generating at most M further coefficient sequences, means for generating at most M first pixel sequences by transforming said at most M further coefficient sequences, and relating to said current texture pattern Second array means for generating a subset of pixels included in the at least a portion of the image block by distributing pixels of the generated first pixel sequence to the image block using a distribution pattern Have

  The present invention further provides a physical storage medium, such as an optical disc, a semiconductor memory or a hard disk, carrying a bitstream containing encoded coefficients and encoded mask information, from which the encoded coefficients are decoded. And a pixel sequence can be generated by inverse transformation, wherein the mask information allows to reconstruct a mask used to select pixels of an image block and to determine a distribution pattern associated with the mask; A pattern suggests a storage medium that allows a pixel sequence of pixels to be distributed over the image block.

  Further advantageous embodiment features are specified in the dependent claims.

Exemplary embodiments of the invention are illustrated in the drawings and are explained in more detail in the following description. The illustrative embodiments are set forth only to illustrate the invention, not to limit the disclosure, scope, or spirit of the invention as defined in the claims.
FIG. 3 illustrates an example image block showing a texture pattern. FIG. 6 illustrates an exemplary binary partition mask. FIG. 5 is a diagram illustrating a first example of partitioning a block into subblocks and modifying the subblock by accumulating the pixels of the subblock. FIG. 6 is a diagram illustrating a second different example of partitioning a block into subblocks and modifying the subblock by accumulating pixels of the subblock. 4a-4c are diagrams illustrating a first example of pixel accumulation of a subset of blocks using a two-step texture pattern dependent mapping that can be reversed. FIGS. 5a to 5c are diagrams illustrating a second example in which the accumulation of pixels of a subset of blocks is different using a two-step texture pattern-dependent mapping that can be reversed. FIG. 6 illustrates an exemplary embodiment of a scan principle associated with a texture pattern. It is a figure which shows the example of the conversion procedure by which the DC coefficient (s) after 1st and 2nd conversion is shown by the square. It is a figure which shows the example of the flow of information in an encoding and decoding process, showing the corresponding mapping in both processes. FIG. 6 illustrates an exemplary encoding scan order that is different for two exemplary partitions into which an exemplary image block has been partitioned using an exemplary binary mask. FIG. 4 is a diagram illustrating encoding using an exemplary texture pattern adaptive split block transform for encoding at the encoder side. FIG. 5 is an exemplary flowchart of corresponding image block decoding at the decoder side. FIG.

  The present invention can be implemented in any electronic device having a correspondingly adapted processing device. For example, the present invention may be implemented in a television, a mobile phone, a personal computer, a digital still camera, or a digital video camera.

  In an exemplary embodiment of a method for encoding an image block of an image, the image block has a number of rows and a number of columns, wherein the method relates at least a portion of the image block to a current texture pattern. Encoding the first subset of pixels resulting from dividing into a first subset of pixels and at least a second subset of pixels according to a current texture pattern adaptive partition. In the exemplary embodiment, encoding the first subset comprises first and second metasequences comprising a plurality of pixel sequences using invertible mappings associated with texture patterns. Generating. Here, the number of pixel sequences in the metasequence does not exceed the number of rows, the number of pixels in each pixel sequence does not exceed the number of columns, or the number of pixel sequences in the metasequence The number does not exceed the number of columns and the number of pixels in each pixel sequence does not exceed the number of rows. The step of encoding the first subset also applies first and second one-dimensional transforms to the generated pixel sequence to generate a sequence of coefficients, and the coefficients included in the sequence of coefficients are Quantization and scanning, scanning is performed in entropy scan order, including entropy encoding of the quantized and scanned coefficients and entropy encoding of data that allows reconstruction or acquisition of the current texture pattern. The reversible mapping associated with the first texture pattern and the reversible mapping associated with the second texture pattern are associated with the current texture pattern.

  More particularly, an exemplary rectangular block in a picture may include several video objects or some texture. This phenomenon results in a large number of non-zero coefficients after the traditional DCT transform, especially when it appears with multiple strips or edges along directions other than vertical / horizontal. Therefore, encoding efficiency is low. A typical example of a directional texture with multiple strips is shown in FIG.

  Given the partitioning rules, each pixel in the exemplary image block is assigned to one of two or more subgroups. In the following, the number of subgroups is illustratively set to 2, but those skilled in the art will readily see how to extend the partition and accumulation principle when a partition allows more than two subgroups. You will understand.

  The partitioning procedure may be implemented by one or more binary masks having the same size as the block. For example, the mask in FIG. 2 means that pixels in the upper half of the block belong to one partition and pixels in the lower half belong to the other partition. Thus, a partition rule that allows n subgroups has a one-to-one mapping relationship with an n-value mask. Thus, the exemplary binary partition rule discussed below has a one-to-one mapping relationship with a binary mask. For n greater than 2, the n-value mask may be represented by a set of binary masks.

  The number of pixels in each partition (sub-block) may not be equal. This means that asymmetric partitioning is allowed. By doing so, partition boundaries within a block can be detected more accurately and energy concentration within each partition (sub-block) can be improved.

  In an exemplary embodiment, the mask is predefined at the encoder and at the decoder side. For example, the example binary mask included in the table at the end of this article represents considerations for horizontal, vertical, diagonal, and other pattern orientations with one or more strips, resulting from encoding. In the encoded data, for example, it may be referred to by an encoded mask number.

  Furthermore, a new mask may be derived by analyzing the pattern in the block. Possible methods include, but are not limited to, edge detection, statistical analysis, transform domain analysis, and the like. The new mask so derived can then be encoded the first time it is used and then specified by a new reference number or reference number assigned to the new mask.

  Mask information obtained by indexing from a predefined list of patterns or obtained from local analysis of image attributes should be stored and transmitted to the decoder for correct decoding. For the former approach, you only need to send the mask index. For the latter, at least when used for the first time, the mask shape itself should be transmitted. These two approaches are briefly described in FIG.

  After partitioning, the pixels in each subgroup are shifted and thereby accumulated. Thereby, subsequent conversions benefit in that the number of non-zero coefficients is reduced. The shift is accomplished by an invertible mapping that may be specific to the applied partition rule or may be a default invertible mapping independent of the applied partition rule. For example, the pixels in each subpartition (subgroup) are shifted towards the upper limit of the block (empty columns are skipped). By doing this, the pattern in the block is removed to some extent and pixels with similar intensities are merged together. This is good for energy concentration. An example of block merging is shown in FIGS. 3a and 3b.

  The partitioning and merging operations described here by way of example are a more general framework and actually generate more than one adjunct block from a single block, with pixel intensity within each auxiliary block. Distribution is more uniform.

  In the following, examples of partition rules specific flippable mappings are described. This example includes two steps and is applicable to any partition or mask related to a texture pattern that exhibits a certain orientation or direction (vertical, horizontal, diagonal, bent, etc.).

  In a first step of an exemplary, non-default invertible mapping, the pixels in the sub-block are shifted along the direction indicated by the mask pattern orientation. An example of this step is given in Fig. 4a and Fig. 5a. The direction of the mask may be vertical, horizontal, diagonal, or others.

  After the first step, a series of pattern strips (which may be different in length from one another) are acquired and organized in an intermediate array as shown in FIGS. 4b and 5b. . For a given mask, the order of those strips (left to right) in the rows of the intermediate array is determined by the fixed pattern scanning principle (eg, the leftmost strip of the pattern indicated by the smallest x coordinate on average) To the rightmost strip; in addition, on average, from the top strip to the bottom strip of the pattern indicated by the smallest y coordinate). An example of the scanning principle is shown in FIG.

  This first step of an exemplary non-default invertible mapping results in a series of intermediate pixel sequences (also referred to as an intermediate array of strips). Each intermediate pixel sequence or strip-if the image block contains N x N pixels-contains at most N pixels, but the number of intermediate pixel sequences (strips) included in the sequence (intermediate array) Is not necessarily limited to N at most.

  Thus, an exemplary non-default invertible mapping includes a second step, after which the number of pixel sequences included in the further sequence resulting from the second step is not greater than N It is guaranteed.

  Within the second step, partial sequences of intermediate pixel sequences included in the sequence are merged together by concatenation of intermediate pixel sequences included in the partial sequence. The concatenation is performed under the constraint that each resulting pixel sequence does not contain more than N pixels. In other words, in the second step, each strip in the intermediate array is sequentially inserted into one column from one side of the intermediate array to the other. If the length of the sequence after adding the next strip to the end of the current sequence is greater than N, this strip is used to start a new sequence. Subsequent strips are then sequentially added again to the end of this new column until adding further subsequent strips causes the number of pixels in this new column to be greater than N. Done. This operation continues until all strips in the intermediate array are filled. The output of this operation is a 2D block with at most N column lengths. An example of the result of the second step is shown in FIG. 4c and FIG. 5c.

  For partitioning rules that ensure that all sub-blocks resulting from partitioning contain at least N × N / 4 pixels, the second step results in a further sequence containing at most N pixel sequences, It can be ensured that each of the N pixel sequences includes at most N rearranged pixels.

  If there is no clear direction in the mask, a default shift operation (shift towards the upper limit) is performed.

  After a block assembling operation, a 2D transform, for example a 2D DCT transform, is applied separately to the integrated sub-blocks. The first 1D conversion of 2D conversion is performed in the vertical direction. This corresponds to the first 1D conversion being performed along the pattern direction. This is because the shift operation of the first step of pixel assembling is performed according to the pattern direction. After the first 1D transformation, the resulting intermediate coefficients are shifted towards the left side of the block, and then the second 1D transformation is performed in the horizontal direction. An example of the conversion procedure is shown in FIG. In FIG. 7, the DC coefficient (s) after the first and second transformations are separately indicated by squares.

  To further explore the correlation between these two sub-blocks after the two-stage transform, DC (AC) coefficient prediction between the sub-blocks may be used. That is, the coefficient value of the first partition may be used as a prediction for coefficient values at similar positions in the second partition, and the residual of the second partition relative to the prediction of the first partition is further processed. For example quantized, zigzag scanned and entropy encoded.

  Zigzag scanning and entropy coding are processed separately in sub-blocks—or residuals. For example, if the mask in FIG. 9 is used, the scan order for the transformed coefficients in partitions 1 and 2 can be designed differently as shown in FIG. The entropy coding method for these two partitions is not limited to this, but includes any entropy coding scheme such as Huffman coding, CAVLC, CABAC, etc., resulting in an output bitstream that is physically It is recorded on a storage medium, for example, an optical disk such as DVD or BD, a semiconductor storage medium such as flash, or a hard disk drive, and transmitted or broadcasted wirelessly or by wire. The output bitstream contains a code for data that allows the reconfiguration of predefined partition rules or partition masks and data that references the corresponding inverse mapping of the invertible mapping used for accumulation in the integration step. Including.

  A flow chart of the integration operation and conversion and their reverse operation is shown in FIG. The decoding process is described in the next section.

  The input to this step is a compressed bitstream. The output from this step is scanned and includes quantized coefficients, mask index (if using a predefined pattern) / mask (if using local analysis) and other sub-information.

  For decoding, the decoding device receives an input bitstream by reading from an optical storage medium or by receiving it wirelessly or wired. From the received input bitstream, the code of the data that allows the reconstruction or acquisition of the corresponding inverse mapping of the partition mask and the invertible mapping is separated and decoded. The rest of the received bitstream is reverse scanned and inverse quantized to achieve an accumulated coefficient sub-block ready for inverse transformation. An inverse 1D transform may be applied in the horizontal direction and the resulting intermediate coefficients may then be shifted to the right. Here, how much of the direct coefficient is shifted is determined using the partition mask information.

  The direct coefficients are then inversely 1D transformed in the vertical direction, resulting in a sub-block of pixels accumulated at one edge of the sub-block.

  Using the mask information, the original position of each of the accumulated pixels can be determined. That is, using the fixed pattern scan principle associated with the first step accumulation, the pixels accumulated in the sub-block can be redistributed into the block, after which each partition is restored. Similarly, other partition (s) can be restored and all partitions can be combined to reproduce the entire image block.

  According to the mask information, a one-to-one mapping between the original pixel and the pixel for the first transformation can be established (referred to as mapping 1). The intermediate coefficients after the first conversion are shifted to the left for the second conversion. Then, another one-to-one mapping from the first transformed intermediate coefficient to the intermediate coefficient for the second transformation can be established (referred to as mapping 2). These two mapping relationships can be obtained once the decoder knows the mask information. Since the two mapping operations can be reversed, the decoder can implement an inverse mapping process to establish a positional relationship between the reconstructed pixels and the transformed coefficients. A flowchart of the inverse mapping operation in the decoder is shown in FIG.

  A texture pattern adaptive split block transform diagram and encoding framework of the present invention is shown in FIG. In the “Mask Generation” module, a mask can be generated with a predefined pattern as described in the appendix. The mask can also be generated by a local content analysis result. For example, pixels within a block can be classified into two (or more) classes according to the intensity distribution, or can be classified based on edge or segmentation detection results.

Claims (9)

A method of encoding an image block of an image using a texture pattern adaptive split block transform, wherein the image block includes N times M pixels, where N is greater than or equal to M, and the method includes:
a) partitioning the image block into a first subset of pixels and at least a second subset of pixels according to a current texture pattern adaptive partition associated with a current texture pattern;
b) arranging the first subset of pixels into a metasequence of a pixel sequence using a scan order associated with a texture pattern, wherein the scan order associated with the texture pattern is the current texture In relation to a pattern, each of the pixel sequences comprising at most N pixels;
c) generating further metasequences of at most M further pixel sequences, each of said further pixel sequences concatenating one of said pixel sequences or a pixel sequence of a subsequence of said metasequence And each of the further pixel sequences includes at most N pixels;
d) generating a coefficient sequence by transforming the further pixel sequence;
e) generating a further coefficient sequence by a one-dimensional transformation across the coefficient sequence;
f) generating a still further coefficient sequence by quantizing and scanning the coefficients contained in the further coefficient sequence, wherein the scanning is performed according to an encoding scan order;
g) entropy encoding the further coefficient sequence;
h) entropy encoding an indicator associated with the current texture pattern;
Method. The current texture pattern adaptive partition is
A texture pattern containing multiple strips,
The method of claim 1, wherein the method is associated with one of a textured pattern including an asymmetric texture pattern and a non-directional texture.   The method of claim 1 or 2, further comprising applying steps b), c), d), e), f) and g) to the at least second subset of pixels.   4. The method of claim 3, wherein for the at least second subset, a different further encoding scan order is used in step f).   5. A method according to claim 3 or 4, further comprising coefficient prediction between the first subset and the at least second subset. An apparatus for encoding an image block of an image using a texture pattern adaptive split block transform, wherein the image block includes N times M pixels, where N is greater than or equal to M;
Means for partitioning the image block into a first subset of pixels and at least a second subset of pixels according to a current texture pattern adaptive partition associated with a current texture pattern;
Arranging the first subset of pixels in a metasequence of a pixel sequence using a scan order associated with a texture pattern, wherein the scan order associated with the texture pattern is the current texture pattern And wherein each of the pixel sequences includes at most N pixels;
Means for generating further metasequences of at most M further pixel sequences, each of said further pixel sequences concatenating one of said pixel sequences or a pixel sequence of a subsequence of said metasequence Means, each of said further pixel sequences comprising at most N pixels;
Means for generating a coefficient sequence by transforming the further pixel sequence;
Means for generating further coefficient sequences by a one-dimensional transformation across the coefficient sequences;
Means for generating still further coefficient sequences by quantizing and scanning coefficients contained in said further coefficient sequence, wherein the scanning is performed according to an encoding scan order;
Means for entropy encoding the further coefficient sequence;
Means for entropy encoding an indicator associated with the current texture pattern;
apparatus. A method of decoding a bitstream representing an encoded image block of an image, wherein the image block is encoded using a texture pattern adaptive split block transform, comprising N times M pixels, N Is greater than or equal to M and the method is
a) entropy decoding the first coefficient sequence included in the bitstream;
b) entropy-decode the index included in the bitstream associated with the current texture pattern to determine an inverse texture pattern adaptive scan order associated with the current text pattern;
c) inverse quantize the coefficients of the first coefficient sequence;
d) using inverse encoding scan order to arrange the dequantized first coefficients into two or more further first coefficient sequences;
e) generating a further first coefficient sequence by an inverse one-dimensional transformation across the further first coefficient sequence;
f) generating a pixel sequence by inverse transformation of said further first coefficient sequence;
g) distributing the pixels of the pixel sequence across the image block using the determined inverse texture pattern adaptive scan order;
Method.   The method further comprises entropy decoding a second coefficient sequence included in the bitstream and applying steps c), d), e), f) and g) to the second coefficient sequence. The method described. An apparatus for decoding a bitstream representing an encoded image block of an image, wherein the image block is encoded using a texture pattern adaptive split block transform and includes N times M pixels, N Is greater than or equal to M and the device is
Means for entropy decoding a coefficient sequence included in the bitstream;
Means for entropy decoding an indicator included in the bitstream associated with a current texture pattern to determine an inverse texture pattern adaptive scan order associated with the current text pattern;
Means for dequantizing the coefficients of the first coefficient sequence;
Means for arranging the dequantized first coefficients into two or more further coefficient sequences using inverse encoding scan order;
Means for generating a still further first coefficient sequence by an inverse one-dimensional transformation across the further first coefficient sequence;
Means for generating a pixel sequence by inverse transformation of said further first coefficient sequence;
Means for distributing pixels of the pixel sequence across the image block using the determined inverse texture pattern adaptive scan order;
apparatus.

Images