JP2005217872A - Tilesize changing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a tilesize changing apparatus capable of fast changing tilesize and reducing distortion in a tile. <P>SOLUTION: A small region data extracting unit 51 extracts position information a of an extracted small region and code information corresponding to the small region. A determining unit 61 determines on the basis of the information a whether a coefficient necessary for correction is included in the small region. When the coefficient necessary for correction is determined to be included, the small region is transmitted to a small region entropy decoding unit 52 and the coefficient is encoded. The decoded coefficient is held in a coefficient buffer 53. A size determining unit 62 determines to couple or divide the small region in accordance with the tilesize changing, and a small region size changing unit 54 couples or divides the coefficient of the small region read from the coefficient buffer 53. A distortion correcting unit 55 previously reduces the distortion occurring incident to the tilesize changing. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a tile size conversion apparatus, and more particularly to a tile size conversion apparatus suitable for application to a gateway apparatus that converts a tile size of an image encoded by a tiled wavelet transform in a network during transmission.

  When a large-capacity image such as a high resolution is encoded under a memory constraint mainly caused by a hardware environment, a method is used in which the image is divided into rectangular regions (tiles) and encoded independently. Naturally, on the decoding side, since processing can be performed only in units of tiles, processing is performed with the tile size at the time of encoding regardless of the memory implementation amount of the decoder.

  However, in general, the encoding environment and the decoding environment are not always the same, and therefore, the size of the tile size that was optimal at the time of encoding is not necessarily optimal at the time of decoding. In addition, in a decoder that restricts the tile size of an image that can be decoded due to restrictions on the mounted memory, even if the data is encoded using the same encoding method, the decoding may not be possible due to the difference in tile size. .

  Therefore, the invention shown in the following Patent Document 1 has been made. The present invention has the configuration shown in FIG. 13, and after extracting the encoding parameters from the tiled code data a by the tile data dividing unit, the decoder decodes all the codes to obtain all the wavelet transform coefficients. Next, the arrangement conversion unit, that is, the tile size conversion unit, performs tile size conversion (conversion from the tile size M to N) for this coefficient.

FIG. 14 is a diagram illustrating a tile size conversion method in the tile size conversion unit. By rearranging the wavelet transform coefficients, the entire image is divided into four tiles from a state where the entire image is one tile. An example of converting to a thing is shown.
JP 2003-274184 A

  However, in the above-described conventional technology, it is impossible to convert to an arbitrary tile size, and in the reproduced image after conversion, the tile boundary exists before the conversion and the portion where the tile boundary exists after conversion. There is a problem that distortion occurs. In addition, the above-described conventional technique can reduce the processing time as compared with a method of once reproducing all images and converting the tile size, but still requires a lot of time, so there is room for improvement.

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a tile size conversion device that can shorten the processing time required for tile size conversion and reduce tile distortion. .

  In order to achieve the above-described object, the present invention relates to image data divided into small areas and encoded in a wavelet transform coefficient area, and based on the position data, means for extracting position data of the small area, Determining means for determining whether the coefficients in the small area need to be corrected in accordance with tile size conversion or for correcting other coefficients, and decoding the small area that has been corrected by the determining means A small region decoding unit that performs the decoding, a buffer that holds the coefficient of the decoded small region, a size determination unit that determines a size of the attention small region after conversion in accordance with the tile size conversion, and a size determined by the size determination unit Thus, a small area size converting means for reading out a new small area from the coefficient buffer, a distortion reducing means for reducing distortion caused by the tile size conversion in advance, and the small area recovery. Decoded by means, encoding means for re-encoding the coefficients of the tile size conversion and corrected small region, it is characterized in that and means rearranges the small region.

  According to the present invention, when tile size conversion is performed, only a small region corresponding to a tile boundary is entropy decoded and a wavelet transform coefficient of only that region is reproduced.

  According to the present invention, in tile size conversion, it is possible to omit a decoding process of an area that does not need to be entropy decoded. Therefore, it is possible to provide a tile size conversion apparatus that is significantly accelerated.

  Hereinafter, the present invention will be described in detail with reference to the drawings. First, the configuration of an encoding apparatus suitable for the present invention and the outline of its operation will be described with reference to FIGS. FIG. 1 is a block diagram showing a schematic configuration of the encoding apparatus, and FIG. 2 is an explanatory diagram of the main part of the operation.

  The tile dividing unit 1 divides image data into tiles having a certain size. For example, as shown in FIG. 2, the image data 11 is divided into rectangular areas of a certain size, that is, tiles 1, 2, 3,. The discrete wavelet transform unit (DWT unit) 2 performs discrete wavelet transform for each of the tiles 1, 2, 3,..., P to obtain transform coefficients. In the case of lossy encoding, the quantization unit 3 encodes the transform coefficient.

Next, the small region dividing unit 4 divides the subband of the transform coefficient into smaller regions (hereinafter referred to as small regions). For example, the tile 1 is divided into small areas 21, 22, 23, 24, 25,..., 2q as shown in FIG. Here, the size of the small area is a rectangle composed of coefficients of 2 ^m × 2 ⁿ (m and n are natural numbers) as shown by taking the small area 21 as a representative. It is assumed that the size is constant throughout the image coding.

  However, it is assumed that a small area of the same size is cut out from sub-bands of the same wavelet transform level arranged for all tiles. In this case, as shown in the tile 1a of FIG. 2, even if the small areas 31, 32,... Are cut off halfway at the end of one subband of a certain wavelet transform level, the small areas 31, Since the continuation of 32,... Exists at the end of the subband of the same wavelet transform level of the adjacent tile (for example, tile 2) (for example, 31 ′ in the drawing), no problem occurs. An exception is made for the end of the subband corresponding to the end of the image.

  Next, the entropy encoding unit 5 performs encoding for each coefficient in the small area divided as described above. Here, based on the coding in the bit plane using the arithmetic code, a context coding technique for switching the internal state of the encoder according to the situation of surrounding coefficients is used. By this processing, the encoded sequences 41 and 42 shown in FIG. 2 are generated. The encoded sequences are stored in the same file in the order of tiles 1, 2, 3,. This file includes the codes 21, 22,..., 31, 32,..., 31 ',. Finally, the rate control unit 6 performs a process of truncating the code so that a desired bit rate is obtained. This is a technique adopted in JPEG2000. Note that when decoding is performed, processing opposite to the above processing is performed.

  The encoded sequence 41 or 42 generated as described above is based on the size divided by the tile dividing unit 1. When transmitting a tiled wavelet transform encoded image typified by JPEG2000, which is expected to become widespread in the future, the decoder on the receiving side needs to perform decoding processing with the same tile size as that used for encoding. However, when the tile size at the time of encoding is large, the amount of memory required for the wavelet transform increases, which may hinder decoding in an environment where the amount of memory such as a mobile terminal is limited. Therefore, for example, if the tile size conversion apparatus according to the present invention is arranged on a network in the middle of transmission, decoding can be performed regardless of the memory mounting amount of the decoder.

  Hereinafter, an embodiment of a high-speed tile size conversion apparatus according to the present invention will be described. FIG. 3 is a block diagram showing a configuration of an embodiment of the accelerated tile size conversion apparatus. It is assumed that tile sizes before and after conversion performed by the high-speed tile size conversion apparatus are known in advance.

  In the figure, the data receiving unit 50 reads the code corresponding to each small area from the received code sequence. The small area data extraction unit 51 extracts the position information a of the extracted small area and code information corresponding to the small area. Based on the position information a, the determiner 61 determines whether or not the small area includes a coefficient that needs to be corrected. In this determination, if this small area is within the coefficient range affected by the tile boundary by the tile size conversion, it is determined that the correction is necessary. If it is determined that a coefficient requiring correction is included, the small area is sent to the small area entropy decoding unit 52, and the coefficient is decoded. On the other hand, if it is determined that the coefficient that needs to be corrected is not included in the small area, the small area is sent to the data rearrangement unit 57 without being decoded.

  Since the coefficient range affected by the tile boundary is uniquely determined by the number of repetitions of the wavelet transform, the determination unit 61 determines that the target small region extracted by the small region data extraction unit 51 is the wavelet transform. Whether or not the correction is a small area is determined depending on whether or not the coefficient range affected by the tile boundary determined by the number of repetitions overlaps. FIG. 4 shows, as an example, a small region (a small region with a shadow) that is affected by the tile size conversion from the tile boundary when the wavelet transform is repeated three times. Therefore, if the small region extracted by the small region data extraction unit 51 corresponds to the small region with the shadow, the determiner 61 sends the small region to the small region entropy decoding unit 52 to add the shadow. If it is not a small area, it is sent to the data rearrangement tile size conversion unit 57.

  The size determination unit 62 determines the size of the target small region for tile size conversion from the small region position information a.

  The small region size conversion unit 54 reads a new small region from the coefficient buffer 53 with the size determined by the size determination unit 62, and converts the small region size. FIG. 5 shows an example of size conversion (combination) of small areas when existing tile boundaries disappear. In the example of FIG. 4, the subband and the small area are completely separated, but if not, the size of the small area is smaller than the normal size around the subband boundary as shown in FIG. . In this case, since the remaining part of the corresponding small area exists in the subbands of the same wavelet transform level of the adjacent tile, when this tile boundary disappears due to the tile size change, on the coefficient buffer 53 Thus, these small area fragments are combined into one small area. In the example of FIG. 5, an area indicated by ◯ is defined as one small area. For example, two small region fragments a1 and a2 are combined, or four small region fragments C1, C2, C3, and C4 are combined to form one small region. Then, the small area is read from the coefficient buffer 53. Conversely, when a new tile boundary must be provided in the small area, one small area is divided and read from the coefficient buffer 53, and the fragments read separately by the division are newly set. Place on both tiles that can be.

FIG. 6 is an explanatory diagram when the small areas a1 and a2 of FIG. 5 are combined, and shows an encoded sequence of the tiles T ₁ , T ₂ ,. P1, p2, p3,... Indicate tile boundaries. As shown, the tile _{T 1, T 2, T 3} , the size changes to for example large tile size _{··· T '1, T' 2} , if you ..., belonging to the tile _{T 1} one subregion A1 which was belonged to fragment a1 and tile T ₂ of the small region and the fragment b1 subregions bound to is made. Conversely, when a new tile boundary must be provided in the small area A1, conversely to the above, the small area A1 is divided into fragments a1 and a2, which are arranged on both the tiles T ₁ and T _2. Is done.

In this way, when a small region or a fragment thereof is combined or divided, distortion occurs at the tile boundary. Therefore, the distortion correction unit (or distortion reduction means) 55 corrects this distortion in advance. This tile boundary distortion correction processing is performed as follows. This distortion correction processing is performed in accordance with “2002 Multimedia” announced by the present inventors on December 1-6, 2002.
Since it is described in detail in pages 97 to 105 “Tile Boundary Artifact Reduction Algorithms for Tile Size Conversion of Wavelet Image” of “Conference”, the outline thereof will be described here.

  First, a lifting configuration of discrete wavelet transform (DWT) will be described. In the present invention, as a technique for realizing DWT used in a wavelet transform encoding method such as JPEG2000, a lifting configuration that can easily realize a digital filter is used.

The following expression is a process performed in each step of 5 × 3 DWT decomposition by lifting.
Y (2n + 1) = X (2n + 1)-[{X (2n) + X (2n + 2)} / 2]
Y (2n) = X (2n) + [{Y (2n-1) + Y (2n + 1) +2} / 4]
Here, X (n) represents the coefficient or pixel value before DWT at the position n. Y (n) represents a coefficient or pixel value after DWT at the position n.

The following equation shows the processing performed in each step of 5 × 3 DWT synthesis (inverse DWT).
X (2n) = Y (2n)-[{Y (2n-1) + Y (2n + 1) +2} / 4]
X (2n + 1) = Y (2n + 1) + [{X (2n) + X (2n + 2)} / 2]

  FIG. 7 is a buffer transition diagram of one-dimensional 5 × 3 DWT decomposition using the lifting configuration. FIG. 8 shows a buffer transition diagram of one-dimensional 5 × 3 DWT synthesis using the lifting configuration.

Further, the following expression shows the processing performed in each step of 9 × 7 DWT decomposition defined in JPEG2000 together with 5 × 3 DWT.
Y (2n + 1) = X (2n + 1) + p ₁ · {X (2n) + X (2n + 2)}
Y (2n) = X (2n) + u ₁ · {Y (2n-1) + Y (2n + 1)}
Y (2n + 1) = Y (2n + 1) + p _2. {Y (2n) + Y (2n + 2)}
Y (2n) = Y (2n) + u _2. {Y (2n-1) + Y (2n + 1)}
Y (2n + 1) = K ₀ · Y (2n + 1)
Y (2n) = K ₁ · Y (2n)
Here, p ₁ , u ₁ , p ₂ , u ₂ , K ₀ , K ₁ are the following values.
p ₁ = -1.586134342059924
u ₁ = −0.052980118572961
p ₂ = +0.8829110755530934
u ₂ = + 0.4435506852043971
K ₀ = 1 / K ₁ = + 1.2301741414914001

Further, processing performed in each step of 9 × 7 DWT synthesis is shown in the following equation.
X (2n) = (1 / K ₁ ) · Y (2n)
X (2n + 1) = (1 / K ₀ ) · Y (2n + 1)
X (2n) = X (2n) -u _2. {X (2n-1) + X (2n + 1)}
X (2n + 1) = X (2n + 1) -p _2. {X (2n) + X (2n + 2)}
_{X (2n) = X (2n} ) -u 1 · {X (2n-1) + X (2n + 1)}
X (2n + 1) = X (2n + 1) -p ₁ · {X (2n) + X (2n + 2)}

  Next, a distortion correction method applicable to an arbitrary size tile will be described. First, the primary distortion correction will be described.

  Conventionally, as a means for realizing tile size conversion, the conversion coefficient is handled for each subband as shown in FIGS. 13 and 14, but in this embodiment, the buffer arrangement of the DWT filter realized by the lifting configuration is used. Is used as it is. First, the flow of the correction process will be described in a one-dimensional manner, taking an example where the decomposition level is 2.

  However, in order to simplify the description, the case of a 5 × 3 reversible filter used in JPEG2000 will be described. In JPEG2000, a 9 × 7 irreversible filter can be used in addition to this filter, but it can be easily realized by extending the method described here. Here, only correction processing from tileless to tiled will be described.

Processes 1 to 4 are shown in FIG. 9, and processes 5 to 8 are shown in FIG.
(Process 1) already the coefficients existing set of H _2, to produce a coefficient L ₁ by inverse DWT using the coefficients L ₂ required correction.
(Process 2) The coefficient L ₁ is generated by inverse DWT using the coefficient of L ₁ generated in process 1 and the coefficient H ₂ that needs to be corrected.
Through the above processing, L ₁ corresponding to the positions of the coefficients L ₂ and H ₂ that need to be corrected is generated.
(Process 3) A pixel is generated by inverse DWT using the already existing coefficient of H ₁ and the coefficient L ₁ generated as described above.
(Process 4) and pixels created in process 3, and generates a pixel by inverse DWT using the coefficient H ₁ requiring correction.

In the following processes 5 to 8, it is assumed that a tile is present, and filter folding processing is performed at the end of the tile.
(Process 5) The coefficient H ₁ is generated by DWT using the pixel generated in Process 3 and the pixel obtained in Process 4.
(Process 6) A coefficient L ₁ is generated by DWT using the coefficient H ₁ obtained in process 5, the already existing coefficient H ₁ and the pixels obtained in process 3.
(Process 7) A coefficient H ₂ is generated by DWT using the coefficient L ₁ obtained in Process 6 and the already existing coefficient L ₁ .
Generating a coefficient L ₂ by DWT with (processing 8) processing coefficients L ₁ already exists a coefficient L ₁ obtained by the coefficient H ₂ and process 6 already exists a coefficient H ₂ obtained in 7.
By performing the above correction processing, in the case of the 5 × 3 reversible filter, since the coefficient is completely matched with the coefficient originally created from the tiled state, there is no distortion at the tile boundary.

  Next, two-dimensional distortion correction will be described. Distortion correction for two-dimensionally arranged DWT coefficients can be easily realized by extending one-dimensional distortion correction. When two-dimensional distortion correction is performed, correction is performed on a reference grid defined by JPEG2000 in order to make it easy to obtain the coefficients that need correction, the coefficients necessary for correction, and the pixel positions.

  FIG. 11A shows the concept of a reference grid. In FIG. 11A, only the 4 × 4 coefficient in the upper left is shown in detail, but other coefficients are arranged in the same manner. FIG. 5B shows the coefficient arrangement obtained by the wavelet transform, and is shown to show the correspondence with the coefficient arrangement on the reference grid of FIG. The coefficient in the shaded area in FIG. 4B is a 4 × 4 coefficient in the upper left of FIG. As can be seen from FIG. (A), since the coefficient components are alternately arranged on the reference grid, the correction can be made in the same way as the one-dimensional distortion correction.

The two-dimensional distortion correction processing is the following processing 9-12. FIG. 12 shows this process. Also, let N be the decomposition level of the DWT coefficient. Here, as an example, a correction process from no tile to tile is described.
(Process 9) The partial or entire line is inversely DWT (IDWT) as shown in FIG. 12 in the order of the X direction and the Y direction, and the decomposition level of the coefficient requiring correction is returned to N-1.
(Process 10) The same process is repeated to return some coefficients to the pixel state.
(Process 11) Considering the tile boundary, the coefficient returned to the pixel state is DWT in the order of Y direction and X direction to return the coefficient decomposition level to 1.
(Process 12) The same process is repeated to return the coefficient to the original decomposition level N.

  By performing the above correction processing, in the case of the 5 × 3 reversible filter, since the coefficient is completely matched with the coefficient originally created from the tiled state, there is no distortion at the tile boundary. Even in the case of the 9 × 7 irreversible filter, a subjectively and objectively good image can be obtained. With the above processing, the processing of the distortion correction unit 55 in FIG. 3 is completed.

  Next, the small region entropy encoding unit 56 in FIG. 3 performs encoding for each coefficient in the small region. Next, in the data rearrangement 57, the coefficient that has not been corrected and the corrected coefficient are rearranged, and the tile size is converted. The data in which the tile size is converted in this way is transmitted over a network having, for example, a mobile terminal.

It is a block diagram which shows the structure of the encoder for producing | generating an image data code sequence. It is explanatory drawing of operation | movement of the principal part of the said encoder. It is a block diagram which shows the structure of the high-speed tile size conversion apparatus of one Embodiment of this invention. It is a figure which shows an example of the small area | region adjacent to a tile boundary. It is explanatory drawing of the coupling | bonding or division | segmentation of the fragment | piece of the small area accompanying tile size conversion. It is explanatory drawing of the coupling | bonding of the fragment | piece of the small area accompanying tile size conversion. It is explanatory drawing of 5x3 DWT decomposition | disassembly processing by a lifting structure. It is explanatory drawing of 5x3 DWT synthetic | combination processing by a lifting structure. It is explanatory drawing of a tile boundary distortion correction process (DWT coefficient-> pixel). It is explanatory drawing of a tile boundary distortion correction process (pixel-> DWT coefficient). It is a figure which shows the coefficient arrangement | positioning (decomposition level 2) on a reference grid. It is explanatory drawing of two-dimensional tile boundary distortion correction | amendment (4 tiles) on a reference grid. It is a block diagram which shows the structure of the conventional tile size conversion apparatus. It is explanatory drawing of the tile size conversion by the conventional coefficient rearrangement.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Tile division part, 2 ... DWT part, 3 ... Quantization part, 4 ... Small area division part, 5 ... Entropy encoding part, 6 ... Rate control part, 50 ... Data receiver 51 ... Small region data extraction unit 52 ... Small region entropy decoding unit 53 ... Coefficient buffer 54 ... Small region size conversion unit 55 ... Distortion correction 56, small area entropy encoding unit, 57 ... data rearrangement unit, 61 ... determiner, 62 ... size determination unit.

Claims (3)

Means for extracting the position data of the small area with respect to the image data divided into small areas and encoded in the wavelet transform coefficient area;
A determination means for determining whether a coefficient in the small area needs to be corrected in accordance with the tile size conversion based on the position data, or whether correction is necessary for other coefficients;
A small area decoding means for decoding a small area that needs to be corrected by the determination means;
A buffer holding the coefficients of the decoded small area;
A size determination means for determining the size of the attention small area after conversion along with the tile size conversion,
A small area size converting means for reading a new small area from the coefficient buffer at the size determined by the size determining means;
Distortion reducing means for reducing distortion caused by the tile size conversion in advance;
Encoding means for re-encoding the coefficients in the small area decoded and tile-sized and corrected by the small area decoding means;
A tile size conversion apparatus comprising: means for rearranging the small areas. The buffer is a digital filter buffer for realizing discrete wavelet transform,
2. The tile size conversion apparatus according to claim 1, wherein a coefficient position in the buffer is used as it is to convert an arbitrary size tile into a different arbitrary size tile.   The distortion reducing means uses the coefficient position in the buffer as it is, partially synthesizes the coefficient that needs correction, the coefficient necessary for correction, and the pixel, and then re-decomposes the tile size conversion. The tile size conversion apparatus according to claim 1 or 2, wherein distortion occurring at the time is reduced in advance.

Images