JP4118049B2 - Image processing apparatus and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus. And methods More specifically, an image processing apparatus applicable to compression / decompression of image data in the fields of digital cameras, the Internet, medical images, satellite communication images, etc. And image processing method using the apparatus About.
[0002]
[Prior art]
Conventionally, for example, the invention described in Japanese Patent Application Laid-Open No. 11-285006 (image encoding device and image decoding device) extrapolates tile peripheral data during wavelet conversion, generates management information, attaches it to code data, and decodes Sometimes it is decoded with reference to the management information. On the other hand, the present invention applies a low-pass filter to the vicinity of the tile boundary for the decoded (expanded) image data, and does nothing at the time of encoding (compressing). Therefore, no extra data is added to the code data. Also, unlike the invention described in the above publication, the present invention changes the low-pass filter mask size or controls the position where the low-pass filter is applied according to the decoding composition level (number of wavelet transform layers). Can be.
[0003]
The invention described in Japanese Patent No. 2940422 (a method for reducing block distortion generated when decoding transform-coded image data and a decoding device for transform-coded image data) The low-pass filter is adaptively applied based on the difference. On the other hand, the present invention adaptively applies a low-pass filter to the decoded image according to the number of wavelet transform layers, and is not based on the difference in the degree of quantization of adjacent blocks. The structure is different.
[0004]
The invention described in Japanese Patent Application Laid-Open No. 05-014735 (image processing apparatus) has a means for discriminating a block boundary and applies a uniform smoothing filter to the discriminated block boundary. Yes. On the other hand, in the present invention, the block boundary information is embedded in the compressed code, and there is no need to determine the block boundary. In the present invention, an adaptive smoothing filter is applied according to the inter-pixel distance or the edge amount, and uniform smoothing is not performed, and the configuration is different from the invention described in the above publication.
[0005]
[Problems to be solved by the invention]
Due to advances in image input technology and output technology, the demand for higher definition of images has increased greatly in recent years. For example, taking a digital camera (Digital Camera (DC)) as an example of an image input device, the price of a high-performance electrified coupling element (CCD) having a number of pixels of 3 million or more has been advanced, Widely used in products. A product with 5 million pixels is also close. And it is said that this increasing trend in the number of pixels will continue for a while.
[0006]
On the other hand, with regard to image output / display devices, for example, products in the hard copy field such as laser printers, ink jet printers, sublimation printers, and flats such as CRTs, LCDs (liquid crystal display devices), and PDPs (plasma display devices). The high definition and low price of products in the soft copy field of panel displays are remarkable. High-definition images are becoming popular due to the effects of high-performance, low-cost image input / output products on the market. In the future, demand for high-definition images is expected to increase in every situation. In fact, the development of technology related to networks such as personal computers (PCs) and the Internet is accelerating these trends. In particular, recently, mobile devices such as mobile phones and notebook personal computers have become very popular, and opportunities for transmitting or receiving high-definition images from any point using communication means are rapidly increasing.
[0007]
Against this background, it is inevitable that the demand for higher performance or higher functionality for image compression / decompression technology that facilitates the handling of high-definition images will become stronger in the future. One compression method that satisfies these requirements is JPEG2000, which can restore high-quality images even at high compression rates. In JPEG2000, an image is divided into generally rectangular areas (tiles). Each tile is a basic unit for executing the compression / decompression process. Therefore, the compression / decompression operation is performed independently for each tile. At this time, if the compression / decompression process is performed under a condition with a high compression rate, there arises a problem that a “tile boundary” becomes conspicuous. Conventionally, in order to solve this problem, it has been proposed that adjacent tiles overlap with each other. However, in the JPEG 2000 baseline, adjacent tile boundaries are not overlapped.
[0008]
Another conventional invention makes a tile boundary inconspicuous by applying a uniform low-pass filter only in the vicinity of the tile boundary. This is effective in suppressing tile boundary distortion, but when the edge degree of the tile boundary is strong, the edge is blurred in the vicinity of the tile boundary and band-like image quality deterioration appears due to the low-pass filter. Had problems.
[0009]
The present invention has been made in view of the above circumstances, and an object of the present invention is to make distortion of a tile boundary inconspicuous by applying a low-pass filter only in the vicinity of the tile boundary.
[0010]
As the wavelet transform decomposition level (number of layers) increases, the mask size of the low-pass filter is increased, or the inter-pixel distance from the tile boundary to which the low-pass filter is applied is increased. An object of the present invention is to suppress tile boundary distortion.
[0011]
Furthermore, an object of the present invention is to control so as not to generate a band-like blurred image even when the edge amount near the tile boundary is large while suppressing tile boundary distortion.
[0012]
Specifically, the smoothing degree of the low-pass filter is controlled according to the inter-pixel distance from the tile boundary. For example, the smoothing degree of the low-pass filter is gradually weakened as the inter-pixel distance from the tile boundary increases. Also, an edge amount is calculated for pixels near the tile boundary, and the smoothing degree of the low-pass filter is controlled according to the edge amount. For example, the smoothing degree of the low-pass filter is gradually weakened as the edge amount increases. Further, the low-pass filter is controlled in accordance with the inter-pixel distance from the tile boundary and the edge amount at that pixel. That is, control is performed so that the smoothing degree of the low-pass filter becomes weaker as the distance from the tile boundary increases and the edge amount increases.
[0013]
[Means for Solving the Problems]
The invention according to claim 1 is an image processing apparatus for generating image data from a code stream encoded using wavelet transform, interpreting tag information added to the code stream, and converting the code stream into each component Tag processing means for decomposing the code stream of each tile, and a dequantization means for determining a position of a bit to be decoded in an order based on the tag information and generating a context from peripheral bits of the target bit Entropy decoding means for generating a target bit by decoding by probability estimation of the context and the codestream, and decoding by writing the target bit to the position of the target bit; Wavelet inverse transform is applied to the data decoded by the entropy decoding means. Wavelet inverse transform means for restoring each tile of each component of the image data, color space inverse transform means for generating the image data by color space inverse transform of the data restored by the wavelet inverse transform means, Tile boundary smoothing means for performing a smoothing process on the vicinity of the tile boundary of the image data generated by the color space inverse transform means, The tag processing means includes Based on the tag information, tile Locate the boundary, The tile boundary smoothing means is a tile specified by the tag processing means. Edge amount in the diagonal direction at pixels near the boundary The larger the value, the lower the strength of the low-pass filter. It is characterized by doing.
[0014]
The invention of claim 2 is the invention of claim 1, wherein tile The amount of edge in the diagonal direction in the pixels near the boundary, and tile The strength of the low-pass filter is controlled according to the distance between pixels from the boundary.
[0015]
The invention of claim 3 is an image processing method by an image processing apparatus for generating image data from a code stream encoded using wavelet transform, interpreting tag information added to the code stream, and A tag processing step for decomposing the stream into code streams of each tile of each component, and positions of bits to be decoded are determined in an order based on the tag information, and a context is generated from peripheral bits of the target bits Entropy decoding in which decoding is performed by generating a target bit by performing decoding based on a probability estimation between the context and the codestream, and writing the target bit to the position of the target bit. And the entropy decoding step A wavelet inverse transform step for performing wavelet inverse transform on the obtained data and restoring each tile of each component of the image data, and a color for generating the image data by color space inverse transform of the data restored in the wavelet inverse transform step A spatial inverse transform step, and a tile boundary smoothing step for performing a smoothing process on the vicinity of the tile boundary of the image data generated in the color space inverse transform step, The tag processing step includes Based on the tag information, tile Locate the boundary, The tile boundary smoothing step includes a tile specified in the tag processing step. Edge amount in the diagonal direction at pixels near the boundary The larger the value, the lower the strength of the low-pass filter. It is characterized by doing.
[0016]
The invention of claim 4 is the invention of claim 3, wherein tile The amount of edge in the diagonal direction in the pixels near the boundary, and tile The strength of the low-pass filter is controlled according to the distance between pixels from the boundary.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
(Example 1)
FIG. 1 is a block diagram for explaining the basics of the JPEG2000 algorithm. The algorithm of JPEG2000 includes a color space conversion / inverse conversion unit 11, a two-dimensional wavelet transform / inverse conversion unit 12, a quantization / inverse quantization unit 13, an entropy encoding / decoding unit 14, and a tag processing unit 15. Yes.
[0021]
FIG. 2 is a diagram illustrating an example of each component of a color image divided into tiles. As shown in FIG. 2, a color image generally has a rectangular area (tile) 21 in which each component 21, 22, 23 (here RGB primary color system) of the original image is rectangular. _t , 22 _t , 23 _t Divided by. Each tile, for example, R00, R01,..., R15 / G00, G01,..., G15 / B00, B01,. Become. Therefore, the compression / decompression operation is performed independently for each component and for each tile.
[0022]
At the time of encoding, the data of each tile of each component is input to the color space conversion unit 11 shown in FIG. 1 and subjected to color space conversion, and then the two-dimensional wavelet conversion unit 12 performs two-dimensional wavelet conversion (forward conversion). ) Is applied to divide the space into frequency bands.
[0023]
FIG. 3 is a diagram illustrating an example of subbands at each decomposition level when the number of decomposition levels is three. That is, the tile original image (0LL) (decomposition level 0 (30)) obtained by the tile division of the original image is subjected to two-dimensional wavelet transform, and the subband (1LL shown in the decomposition level 1 (31) is displayed. , 1HL, 1LH, 1HH). Subsequently, the low-frequency component 1LL in this hierarchy is subjected to two-dimensional wavelet transformation to separate the subbands (2LL, 2HL, 2LH, 2HH) shown in the decomposition level 2 (32). Similarly, the low-frequency component 2LL is also subjected to two-dimensional wavelet transform to separate the subbands (3LL, 3HL, 3LH, 3HH) indicated by the decomposition level 3 (33).
[0024]
Further, in FIG. 3, subbands to be encoded at each decomposition level are represented in gray. For example, when the number of decomposition levels is 3, the subbands shown in gray (3HL, 3LH, 3HH, 2HL, 2LH, 2HH, 1HL, 1LH, 1HH) are to be encoded, and the 3LL subband is not encoded. .
[0025]
Next, the bits to be encoded are determined in the specified encoding order, and the context is generated from the bits around the target bits by the quantization unit 13 shown in FIG. The wavelet coefficients that have undergone the quantization process are divided into non-overlapping rectangles called “precincts” for each subband. This was introduced to use memory efficiently in implementation.
[0026]
FIG. 4 is a diagram for explaining an example of the relationship between a precinct and a code block. An original image 40 is a tile 40 at a decomposition level 1. _t0 , Tile 40 _t1 , Tile 40 _t2 , Tile 40 _t3 Are divided into four tiles. As shown in FIG. 4, one precinct 40 _p4 Consists of three rectangular regions that are spatially matched, and the precinct 40 _p6 Is the same. Further, each precinct is divided into “code blocks” made up of non-overlapping rectangles. In this example, it is divided into 12 code blocks from 0 to 11, for example, code block 40 _b1 Indicates code block number 1. This is a basic unit for entropy coding.
[0027]
The coefficient values after the wavelet transform can be quantized and encoded as they are, but in JPEG2000, in order to increase the encoding efficiency, the coefficient values are decomposed into “bit plane” units, and “ Ranking can be performed on "bit planes".
[0028]
FIG. 5 is a diagram for explaining an example of a procedure for ranking bit planes. In this example, when the original image 50 (32 × 32 pixels) is divided into four 16 × 16 pixel tiles (the tile 50 shown in FIG. 5). _t0 , Tile 50 _t1 , Tile 50 _t2 , Tile 50 _t3 ), The size of the precinct and code block at the composition level 1 are 8 × 8 pixels and 4 × 4 pixels, respectively. Precincts and code block numbers are assigned in raster order. In this example, precincts are assigned numbers 0 to 3, and code blocks are assigned numbers 0 to 3. A mirroring method is used for pixel expansion outside the tile boundary, wavelet transform is performed with a reversible (5, 3) filter, and a wavelet coefficient value of decomposition level 1 is obtained.
[0029]
Tile 50 _t0 (Tile 0) / Precinct 50 _p3 (Precinct 3) / Code block 50 _b3 FIG. 5 is a conceptual diagram illustrating an example of a typical “layer” configuration for (code block 3). The converted code block 50w3 is divided into subbands (1LL, 1HL, 1LH, 1HH), and wavelet coefficient values are assigned to the subbands.
[0030]
The layer structure is easy to understand when the wavelet coefficient values are viewed from the lateral direction (bit plane direction). One layer is composed of an arbitrary number of bit planes. In this example, layers 0, 1, 2, and 3 are composed of 1, 3, 1, and 3 bit planes, respectively. A layer including a bit plane closer to the LSB is subject to quantization first, and conversely, a layer closer to the MSB remains unquantized until the end. Here, the method of discarding from a layer close to the LSB is called truncation, and the quantization rate can be finely controlled.
[0031]
In the entropy encoding unit 14 shown in FIG. 1 described above, encoding of tiles of each component is performed by probability estimation from the context and target bits. In this way, encoding processing is performed in tile units for all components of the original image.
[0032]
Finally, the tag processing unit 15 performs a process of combining all the encoded data from the entropy coder unit into one code stream and adding a tag thereto. FIG. 6 is a diagram simply showing an example of the structure of the code stream. The head of the code stream 60 and the head of the partial tiles constituting each tile are called headers (the main header 61 and the tile part header 62, respectively). Tag information is added, followed by encoded data (bitstream 63) for each tile. A tag is placed again at the end 64 of the code stream 60.
[0033]
On the other hand, at the time of decoding, contrary to the time of encoding, image data is generated from the code stream of each tile of each component. This will be briefly described with reference to FIG. In this case, the tag processing unit 15 interprets tag information added to the code stream input from the outside, decomposes the code stream into code streams of each tile of each component, and each code stream of each tile of each component. Decryption processing is performed. The position of the bit to be decoded is determined in the order based on the tag information in the code stream, and the inverse quantization unit 13 determines from the sequence of the peripheral bits (that has already been decoded) at the target bit position. A context is created. The entropy decoding unit 14 performs decoding by probability estimation from the context and the code stream, generates a target bit, and writes it in the position of the target bit.
[0034]
Since the data decoded in this way is spatially divided for each frequency band, the two-dimensional wavelet inverse transform unit 12 performs two-dimensional wavelet inverse transform on each of the tiles of each component of the image data. Is restored. The restored data is converted into original color system data by the color space inverse conversion unit 11.
[0035]
In the case of the conventional JPEG compression / decompression format, the tile described in JPEG 2000 may be read as a square block having a side of 8 pixels that performs two-dimensional discrete cosine transform.
[0036]
FIG. 7 is a block diagram for explaining an example of an image decompression unit according to the present invention, in which 71 is a tag processing unit, 72 is an entropy decoding unit, 73 is an inverse quantization unit, and 74 is a two-dimensional Wavelet inverse. A conversion unit, 75 is a color space inverse conversion unit, and 76 is a tile boundary smoothing unit. For the RGB data obtained by the color space inverse transform unit 75, the tile boundary smoothing unit 76 smoothes the pixels in the vicinity of the tile boundary, thereby making the tile boundary distortion inconspicuous. A specific example of processing in the tile boundary smoothing unit 76 is shown in FIGS.
[0037]
FIG. 8 is a diagram illustrating an example of processing in the tile boundary smoothing unit 76 in the case of using three-level Wavelet transform. In FIG. 8, for example, data encoded using the three-level Wavelet transform is affected by 8 pixels (16 pixels at both ends) inside the tile counted from the tile boundary by mirroring. For this reason, as shown in FIG. 8, a low pass filter is applied to 8 pixels (16 pixels at both ends) including tile boundary pixels.
[0038]
FIG. 9 is a diagram illustrating an example of processing in the tile boundary smoothing unit 76 in the case of using two-level Wavelet transform. In FIG. 9, for example, data encoded using two-level Wavelet transform is affected by four pixels (8 pixels on both ends) inside the tile counted from the tile boundary by mirroring. For this reason, as shown in FIG. 9, a low pass filter is applied to 4 pixels (8 pixels at both ends) including the tile boundary pixels. From these, that is, data encoded using Wavelet transform of n layers is subjected to a low-pass filter up to the inner 2 nth pixels counted from the tile boundary.
[0039]
As described above, it is possible to optimize the range in which the low-pass filter is applied according to the number of wavelet layers at the time of encoding. The number of pixels inside the tile that are counted from the tile boundary may be adjusted according to the cost, processing time, and image quality. Furthermore, the low pass filter may be switched according to the wavelet transform method. For example, there are a wavelet transform with a 5 × 3 filter and a wavelet transform with a 9 × 7 filter, and low pass filters having different frequency characteristics can be applied to each.
[0040]
According to the present invention, by applying a low-pass filter in the vicinity of a tile boundary, it is possible to suppress tile boundary distortion and to optimize a range in which the low-pass filter is applied.
[0041]
(Example 2)
FIG. 10 is a diagram illustrating an example of a low-pass filter in which the mask size is variable according to the number of layers of Wavelet transform. In the present embodiment, as shown in FIG. 10, the mask size of the low-pass filter is made variable according to the number of wavelet transform layers. For example, as shown in FIG. 10A, a 7 × 5 mask size low-pass filter is used for a code generated by five-layer Wavelet transform, and as shown in FIG. A low-pass filter having a mask size of 5 × 3 is used for codes generated by hierarchical wavelet transform. This is also because the number of pixels affected by mirroring increases inside the tile boundary as the number of layers increases. Here, the mask size given in this embodiment is one example, and an optimal mask size effective for suppressing tile boundary distortion may be selected as appropriate.
[0042]
(Example 3)
FIG. 11 is a block diagram for explaining another example of the image decompression unit in the present invention, in which 81 is a tag processing unit, 82 is an entropy decoding unit, 83 is an inverse quantization unit, and 84 is a two-dimensional block. A Wavelet inverse transform unit, 85 is a tile boundary smoothing unit, and 86 is a color space inverse transform unit. In this embodiment, a low-pass filter is applied in the vicinity of the tile boundary with the YCbCr signal before being converted into the RGB signal. At this time, a low-pass filter is applied only to the luminance signal Y, and the Cb and Cr signals are made through. This is because the tile boundary distortion largely depends on the luminance signal Y. Since the low-pass filter is not applied to all the RGB signals as in the first and second embodiments, the processing time can be shortened.
[0043]
However, if a low-pass filter for suppressing tile boundary distortion only for luminance is applied, there is a possibility that color continuity cannot be maintained at the boundary. If aiming at more accurate tile boundary distortion suppression, a low-pass filter may be applied to all YCbCr signals or all RGB data after inverse color space conversion. Furthermore, low-pass filters having different frequency characteristics for Y and Cb, Cr, and G and R, B may be used.
[0044]
According to the present invention, by applying a low-pass filter in the vicinity of the tile boundary, the tile boundary distortion can be suppressed and the time required for decoding can be shortened. In addition, it is possible to improve the quality of the decoded image.
[0045]
Example 4
FIG. 12 is a diagram illustrating an example of a low-pass filter corresponding to the compression rate. In this embodiment, the strength of the low-pass filter is made variable according to the compression rate. In general, tile boundary distortion becomes more conspicuous as the compression ratio increases. Therefore, if a strong low-pass filter is applied to an image with a low compression rate, the portion where the low-pass filter is applied becomes blurred and conspicuous. Conversely, it is difficult to obtain the effect of suppressing the tile boundary distortion even if a weak low-pass filter is applied to an image with a large compression rate.
[0046]
For this reason, as shown in FIG. 12, the strength of the low-pass filter is made variable according to the compression rate. For example, when the compression ratio is 1/20, a low-pass filter as shown in FIG. When the compression ratio is 1/40, a low-pass filter as shown in FIG. FIG. 13 is a diagram showing the frequency characteristics of the low-pass filters shown in FIGS. 12 (A) and 12 (B). 13A is a diagram showing the frequency characteristics of the low-pass filter shown in FIG. 12A, and FIG. 13B is a diagram showing the frequency characteristics of the low-pass filter shown in FIG. . Since the value shown in FIG. 13A is larger than the value shown in FIG. 13B in any frequency band and direction, the low-pass filter shown in FIG. It can be seen that the low-pass filter is weaker than the low-pass filter shown in FIG.
[0047]
According to the present invention, it is possible to suppress tile boundary distortion by applying a low-pass filter in the vicinity of the tile boundary. Furthermore, by applying a low-pass filter having an optimum strength in accordance with the compression rate, it is possible to realize tile boundary distortion suppression at any compression rate.
[0048]
(Example 5)
FIG. 14 is a block diagram for explaining another example of the image decompression unit according to the present invention, in which 91 is a tag processing unit, 92 is an entropy decoding unit, 93 is an inverse quantization unit, and 94 is a two-dimensional unit. A Wavelet inverse transform unit, 95 is a color space inverse transform unit, 96 is a corrected tile boundary search unit, and 97 is a tile boundary smoothing unit. In this embodiment, instead of applying a low-pass filter to all tile boundaries, a tile boundary to be subjected to the low-pass filter is searched and a low-pass filter is applied only to the tile boundary. Typical examples are shown in FIGS. 15 and 16 to be described later.
[0049]
15 and 16 are diagrams for explaining an example of processing for applying a low-pass filter only to the tile boundary in the ROI region. This ROI region is a part of the entire image when it is cut out and enlarged from the entire image or emphasized as compared with other parts.
[0050]
FIG. 15 shows a case where the ROI area is an area along the tile boundary. When the ROI boundary is set as shown in FIG. 15A, the tile boundary to which the low-pass filter is applied is set to the dotted line portion shown in FIG. A low-pass filter is not applied to the bold ROI boundary shown in FIG.
[0051]
FIG. 16 shows a case where the ROI region is a region not along the tile boundary. When the ROI boundary is set as shown in FIG. 16A, the tile boundary to which the low-pass filter is applied is set to the dotted line portion shown in FIG. Whether the tile boundary pixel is inside the ROI is calculated by calculation. If the tile boundary pixel is inside the ROI, a low pass filter is applied to the pixel. If the tile boundary pixel is outside the ROI, no low pass filter is applied to the pixel.
[0052]
In this embodiment, whether or not to apply a low-pass filter is determined depending on whether or not it is inside the ROI, but other than that, the low-pass filter is applied only to the tile boundary pixels in the portion where the edge amount of the vertical or horizontal component is large. Control may be performed.
[0053]
According to the present invention, by controlling the tile boundary pixels to be subjected to the low-pass filter, for example, it is possible to perform control such that the low-pass filter is applied only inside the ROI region. Thereby, the processing time for suppressing tile boundary distortion can be shortened.
[0054]
(Example 6)
FIG. 17 is a diagram for explaining an example of processing in the tile boundary smoothing unit. In this example, a low-pass filter is applied to pixels in the vicinity of the tile boundary (pixels in the gray area shown in FIG. 17). 19, FIG. 21, and FIG. 22, which will be described later, show an enlarged state of the dotted line portion shown in FIG.
[0055]
FIG. 18 is a diagram for explaining an example of a method for calculating the inter-pixel distance from the tile boundary. In each pixel shown in FIG. 18, the distance from the tile boundary from the top, bottom, left, and right is calculated. Those minimum values are set as the distance from the tile boundary in each pixel.
[0056]
FIG. 19 is a diagram for explaining an example of adaptive low-pass filter processing using the inter-pixel distance from the tile boundary. FIG. 19A is a diagram illustrating a state where the dotted line portion illustrated in FIG. 17 is enlarged, and a value of the inter-pixel distance from the tile boundary is set. FIG. 19B is a diagram illustrating an example of an adaptive low-pass filter. In this embodiment, the low-pass filter changes only the coefficient of the position of the target pixel. As the inter-pixel distance increases, the coefficient value is gradually increased. In the example shown in FIG. 19B, the central coefficient value: m is calculated by the following equation (1).
m = 8 + 64 * d (d: distance between pixels from tile boundary) Expression (1)
This means that the smoothing degree of the low-pass filter to be applied is weakened as the inter-pixel distance d from the tile boundary increases.
[0057]
Further, according to the image processing apparatus of the present invention, the threshold value of the inter-pixel distance from the tile boundary to which the low pass filter is applied can be controlled according to the number of layers. For example, in the data encoded using the three-level Wavelet transform, the inner 8 pixels (16 pixels at both ends) of the tile boundary are affected by mirroring. For this reason, a low pass filter is applied to the inner 8 pixels (16 pixels at both ends) of the tile boundary as shown in FIG. In addition, for example, in data encoded using two-level Wavelet transform, four pixels (8 pixels on both ends) inside the tile boundary are affected by mirroring. Therefore, a low pass filter is applied to the inner 4 pixels (8 pixels at both ends) of the tile boundary as shown in FIG. In other words, the data encoded using the n-layer Wavelet transform is subjected to a low pass filter up to the 2nd power of the pixel inside the tile boundary. As described above, the range in which the low-pass filter is applied can be optimized according to the number of wavelet layers at the time of encoding.
[0058]
In addition, the number of pixels inside the tile boundary to which the low-pass filter is applied can be adjusted according to the cost, the processing time, and the image quality. Furthermore, the low pass filter may be switched according to the wavelet transform method. For example, there are a wavelet transform with a 5 × 3 filter and a wavelet transform with a 9 × 7 filter, but low-pass filters having different frequency characteristics may be applied to images compressed and expanded by the respective conversion methods.
[0059]
Further, as shown in FIG. 10 described above, the mask size of the low-pass filter can be made variable according to the number of wavelet transform layers. In the example shown in FIG. 10A, a low-pass filter having a 7 × 5 mask size is used for a code generated by Wavelet transform of five layers, and in the example shown in FIG. A low-pass filter having a mask size of 5 × 3 is used for the code generated by the Wavelet transform. This is also because the number of pixels affected by mirroring increases inside the tile boundary as the number of layers increases. The mask sizes shown in FIGS. 10A and 10B are an example, and an optimal mask size that can obtain the effect of suppressing tile boundary distortion may be appropriately selected.
[0060]
Further, a low-pass filter may be applied to the vicinity of the tile boundary in the YCbCr signal before being converted into the RGB signal. At this time, a low-pass filter may be applied only to the luminance signal Y, and the Cb and Cr signals may be passed. This is because the tile boundary distortion largely depends on the luminance signal Y. By not applying a low-pass filter to all RGB signals, the processing time can be shortened.
[0061]
Furthermore, the strength of the low-pass filter may be varied according to the compression rate. In general, tile boundary distortion becomes more conspicuous as an image has a higher compression rate. Therefore, when a strong low-pass filter is applied to an image with a low compression rate, the portion where the low-pass filter is applied becomes blurred and conspicuous. Conversely, it is difficult to obtain the effect of suppressing the tile boundary distortion even if a weak low-pass filter is applied to an image with a large compression rate.
[0062]
FIG. 20 is a diagram illustrating an example of a low-pass filter corresponding to the compression rate. In this embodiment, the strength of the low-pass filter is made variable according to the compression rate. For example, when the compression ratio is 1/20, a low-pass filter with a low smoothing degree as shown in FIG. When the compression ratio is 1/40, a low-pass filter having a strong smoothing degree as shown in FIG. 20B is used.
[0063]
In this embodiment, the JPEG2000 compression / decompression method has been described as a representative example. However, the compression / decompression method is not limited to this, and any compression / decompression method can be used as long as it can include block boundary information in a compression code. A method may be used.
[0064]
According to the present invention, by adaptively controlling the strength of the low-pass filter according to the inter-pixel distance from the block boundary, the image quality degradation that occurs when the edge is strong near the block boundary is conspicuous while suppressing the block boundary distortion. Can be eliminated.
[0065]
(Example 7)
FIG. 21 is a diagram for explaining an example of adaptive low-pass filter processing using the edge amount of pixels near the tile boundary. FIG. 21A is a diagram illustrating a state in which the dotted line portion illustrated in FIG. 17 is enlarged. FIG. 21B shows an example of an edge amount calculation filter, and FIG. 21C shows an example of an adaptive low-pass filter. In this embodiment, the strength of the low-pass filter is switched according to the edge amount of the pixel near the tile boundary. For the pixels in the vicinity of the tile boundary, the edge amount is calculated using the edge amount calculation filter shown in FIG. This edge amount calculation filter calculates an edge amount in an oblique direction. This is for avoiding the conspicuous tile boundary because the edge amount becomes large at the boundary portion of the tile boundary when extracting the edge amount in the vertical and horizontal directions.
[0066]
The strength of the low-pass filter is controlled according to the above-described oblique edge amount. The larger the absolute value of the edge amount, the larger the coefficient value m at the center of the low-pass filter. In the example of the adaptive low-pass filter shown in FIG. 21C, when the central coefficient value is m and the edge amount calculated by the edge amount calculation filter shown in FIG. ) To calculate the center coefficient value: m.
m = 8 + abs (E) (2)
This means that the greater the absolute value of the diagonal edge amount of the pixel near the tile boundary, the weaker the smoothness of the low-pass filter applied to that pixel.
[0067]
In this embodiment, the JPEG2000 compression / decompression method has been described as a representative example. However, the compression / decompression method is not limited to this, and any compression / decompression method can be used as long as it can include block boundary information in a compression code. A method may be used.
[0068]
According to the present invention, the band-like image quality generated when the edge is strong near the block boundary while suppressing the block boundary distortion by adaptively controlling the strength of the low-pass filter according to the edge amount in the pixel near the block boundary. Deterioration can be eliminated.
[0069]
(Example 8)
FIG. 22 is a diagram for explaining an example of adaptive low-pass filter processing using the inter-pixel distance from the tile boundary and the edge amount of the pixel near the tile boundary. FIG. 22A is a diagram showing a state where the dotted line portion shown in FIG. 17 is enlarged. FIG. 22B shows an example of an edge amount calculation filter, and FIG. 22C shows an example of an adaptive low-pass filter. In this embodiment, the strength of the low-pass filter is switched according to the inter-pixel distance from the tile boundary and the edge amount of the pixel near the tile boundary. The following (1) and (2) are calculated for the pixels near the tile boundary.
(1) Distance between pixels from tile boundary
(2) Edge amount of pixel near tile boundary (calculated by edge amount calculation filter shown in FIG. 22B)
[0070]
The strength of the low-pass filter is controlled according to (1) the inter-pixel distance and (2) the amount of edge in the oblique direction. In this embodiment, the coefficient value m at the center of the low-pass filter is increased as the inter-pixel distance is increased and the absolute value of the edge amount is increased. In the example of the adaptive low-pass filter shown in FIG. 22C, the center coefficient value is m, the inter-pixel distance from the tile boundary is d, and the edge calculated by the edge amount calculation filter shown in FIG. When the quantity is E, the central coefficient value: m is calculated by the following formulas (3) and (4).
If d = 0, m = 8 Equation (3)
If d> 0, m = max (8 + 64 * d, 8 + abs (E)) (4)
This means that the greater the inter-pixel distance from the tile boundary and the greater the absolute value of the edge amount of the pixels near the tile boundary, the smaller the smoothing degree of the low-pass filter to be applied. In the above equation (3), the independent control is performed only when d = 0. If the low-pass filter with strong smoothness is not applied unconditionally to the nearest pixel of the tile boundary, the tile boundary is reversed. This is because it stands out.
[0071]
In this embodiment, the JPEG2000 compression / decompression method has been described as a representative example. However, the compression / decompression method is not limited to this, and any compression / decompression method can be used as long as the block boundary information can be included in the compression code. A method may be used.
[0072]
According to the present invention, the edge is strong in the vicinity of the block boundary while suppressing the block boundary distortion by adaptively controlling the strength of the low-pass filter according to the distance between the pixels from the block boundary and the edge amount in the target pixel. The image quality degradation that occurs in some cases can be made less noticeable. Further, by controlling the low-pass filter according to the number of layers at the time of compression, block boundary distortion can be suppressed without any side effects at any number of layers in the wavelet transform. Furthermore, by applying a low-pass filter having an optimum strength in accordance with the compression rate, it is possible to realize tile boundary distortion suppression at any compression rate.
[0073]
【The invention's effect】
According to the present invention, by applying a low-pass filter in the vicinity of a tile boundary, it is possible to suppress tile boundary distortion and to optimize a range in which the low-pass filter is applied. Also, by adaptively controlling the strength of the low-pass filter, it is possible to suppress image quality degradation that occurs when the edge is strong near the block boundary while suppressing block boundary distortion.
[Brief description of the drawings]
FIG. 1 is a block diagram for explaining the basics of a JPEG2000 algorithm.
FIG. 2 is a diagram illustrating an example of each component of a tiled color image.
FIG. 3 is a diagram illustrating an example of subbands at each decomposition level when the number of decomposition levels is 3. FIG.
FIG. 4 is a diagram illustrating an example of a relationship between a precinct and a code block.
FIG. 5 is a diagram for explaining an example of a procedure for ranking bit planes;
FIG. 6 is a diagram simply illustrating an example of the structure of a code stream.
FIG. 7 is a block diagram for explaining an example of an image expansion unit in the present invention.
FIG. 8 is a diagram illustrating an example of processing in a tile boundary smoothing unit when three-layer Wavelet transform is used.
FIG. 9 is a diagram illustrating an example of processing in a tile boundary smoothing unit when two-level Wavelet transform is used.
FIG. 10 is a diagram illustrating an example of a low-pass filter in which a mask size is variable according to the number of layers of Wavelet conversion.
FIG. 11 is a block diagram for explaining another example of the image expansion unit in the present invention.
FIG. 12 is a diagram illustrating an example of a low-pass filter corresponding to a compression rate.
13 is a diagram showing frequency characteristics of the respective low-pass filters shown in FIG. 12. FIG.
FIG. 14 is a block diagram for explaining another example of the image expansion unit in the present invention.
FIG. 15 is a diagram for explaining an example of a process for applying a low-pass filter only to a tile boundary in the ROI region.
FIG. 16 is a diagram for explaining an example of a process for applying a low-pass filter only to a tile boundary in the ROI region.
FIG. 17 is a diagram for describing an example of processing in a tile boundary smoothing unit.
FIG. 18 is a diagram for describing an example of a method for calculating a distance between pixels from a tile boundary.
FIG. 19 is a diagram for explaining an example of adaptive low-pass filter processing using the inter-pixel distance from the tile boundary.
FIG. 20 is a diagram illustrating an example of a low-pass filter corresponding to a compression rate.
FIG. 21 is a diagram for describing an example of adaptive low-pass filter processing using an edge amount of a pixel near a tile boundary.
FIG. 22 is a diagram for describing an example of adaptive low-pass filter processing using the inter-pixel distance from the tile boundary and the edge amount of the pixel near the tile boundary.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Color space transformation / inverse transformation part, 12 ... Two-dimensional wavelet transformation / inverse transformation part, 13 ... Quantization / inverse quantization part, 14 ... Entropy encoding / decoding part, 15 ... Tag processing part, 21 and 22 , 23 ... Component, 21 _t , 22 _t , 23 _t ... Tile, 30 ... Decomposition level 0 tile, 31 ... Decomposition level 1 tile, 32 ... Decomposition level 2 tile, 33 ... Decomposition level 3 tile, 40 _t0 ~ 40 _t3 , 50 _t0 ~ 50 _t3 ... Tile, 40 _p4 , 40 _p6 , 50 _p0 ~ 50 _p3 ... Precinct, 40 _b1 , 50 _b3 , 50 _w3 Code block, 50 ... Original image, 60 ... Code stream, 61 ... Main header, 62 ... Tile part header, 63 ... Bit stream, 64 ... Code stream end, 71, 81, 91 ... Tag processing unit, 72, 82, 92 ... Entropy decoding unit, 73, 83, 93 ... Inverse quantization unit, 74, 84, 94 ... Two-dimensional Wavelet inverse transformation unit, 75, 86, 95 ... Color space inverse transformation unit, 76, 85, 97 ... Tile Boundary smoothing unit, 96... Corrected tile boundary searching unit.

Claims (4)

An image processing apparatus for generating image data from a code stream encoded using a wavelet transform,
Tag processing means for interpreting tag information added to the code stream and decomposing the code stream into a code stream of each tile of each component;
A position of a bit to be decoded is determined in an order based on the tag information, and inverse quantization means for generating a context from peripheral bits of the target bit;
Entropy decoding means for generating a target bit by performing decoding by probability estimation of the context and the code stream, and performing decoding by writing the target bit to a position of the target bit;
Wavelet inverse transform means for performing wavelet inverse transform on the data decoded by the entropy decoding means, and restoring each tile of each component of the image data;
Color space inverse transformation means for generating the image data by color space inverse transformation of the data restored by the wavelet inverse transformation means;
Tile boundary smoothing means for performing smoothing processing on the vicinity of the tile boundary of the image data generated by the color space inverse transform means;
The tag processing means identifies a position of a tile boundary based on the tag information,
It said tile boundary smoothing means, the image processing apparatus characterized by reducing the intensity of the low-pass filter larger the edge in an oblique direction at the tile border neighboring pixel specified by the tag processing unit. The image processing apparatus according to claim 1, the image processing and controlling the diagonal edge amount in the tile boundary neighboring pixels, the intensity of the low-pass filter in accordance with the inter-pixel distance from said tile boundary apparatus. An image processing method by an image processing apparatus for generating image data from a code stream encoded using a wavelet transform,
A tag processing step of interpreting tag information added to the code stream, and decomposing the code stream into a code stream of each tile of each component;
A position of a bit to be decoded is determined in an order based on the tag information, and an inverse quantization step of generating a context from peripheral bits of the target bit;
An entropy decoding step of generating a target bit by performing decoding by probability estimation of the context and the code stream, and performing decoding by writing the target bit to a position of the target bit;
A wavelet inverse transform step for performing wavelet inverse transform on the data decoded in the entropy decoding step and restoring each tile of each component of the image data;
A color space inverse transformation step for generating the image data by color space inverse transformation of the data restored in the wavelet inverse transformation step;
A tile boundary smoothing step for performing a smoothing process on the vicinity of the tile boundary of the image data generated in the color space reverse conversion step;
The tag processing step specifies a position of a tile boundary based on the tag information,
In the tile boundary smoothing step, the strength of the low-pass filter is reduced as the edge amount in the oblique direction in the pixel near the tile boundary specified in the tag processing step is increased. The image processing method according to claim 3, image processing and controlling the diagonal edge amount in the tile boundary neighboring pixels, the intensity of the low-pass filter in accordance with the inter-pixel distance from said tile boundary Method.

Images