JP3973089B2 - Image processing apparatus, image processing method, program, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to an image processing technique for suppressing distortion of a division unit boundary that occurs when an image compressed for each predetermined division unit is expanded for each division unit, and in particular, a tile boundary on an image expanded after compression by JPEG2000. The present invention relates to an image processing apparatus and method suitable for the purpose of suppressing distortion.
[0002]
[Prior art]
  In many cases, image data is compressed and then stored or transmitted, and the compressed image data is decompressed and output or processed when necessary or at a place. For compression of image data, an image compression method such as JPEG for encoding an image using a discrete cosine transform in a predetermined division unit called a block is widely used. One problem with such an image compression scheme is that, when a compressed image is expanded, block boundary distortion occurs in the expanded image. In order to make this block boundary distortion inconspicuous, a technique is known in which block boundary distortion is detected and a low-pass filter is applied to pixels at the block boundary of the expanded image (see Patent Documents 1, 2, 3, etc.). Further, Non-Patent Document 1 discusses an image division method in the compression / decompression process described later.
[0003]
  The applicant of the present application has filed a patent application for an invention related to the present invention (Japanese Patent Application No. 13-4000064, Japanese Patent Application No. 2002-109890).
[0004]
[Patent Document 1]
      JP 5-316361
[Patent Document 2]
      JP-A-9-307855
[Patent Document 3]
      Patent No. 2839987
[Non-Patent Document 1]
      J. X. Wei, M. R. Pickering, M.M. R. Frater and J. F. Arnold, “A New Method for Reducing Boundary Artifacts in Block-Based Wavelet Image Compression,” in VCIP 2000, K. N. Ngan, T .; Sikora, M-T Sun Eds., Proc. of SPIE Vol. 4067, pp. 1290-1295, 20-23 June 2000, Perth, Australia
[0005]
[Problems to be solved by the invention]
  In recent years, with the advancement of image input / output technology, there is an increasing demand for higher definition of images. Taking a digital camera as an example of an image input device, a high-performance electronic device with more than 3 million pixels.loadAs the cost of coupling elements has been reduced, they are widely used in products in popular prices. As for image output and display devices, there is a remarkable increase in definition and price in the hard copy field such as laser printers and inkjet printers. This is in the soft copy field such as CRT and LCD displays. Is no exception. Therefore, the demand for compression / decompression technology that can easily handle high-definition images will inevitably increase in the future.
[0006]
  One compression method that satisfies these requirements is JPEG2000, which can decode high-quality images even at high compression rates. In JPEG2000, it is possible to perform compression / decompression processing in a small memory environment by dividing an image into rectangular regions (tiles). That is, each tile is a basic unit for executing the compression / decompression process, and the compression / decompression operation can be performed independently for each tile.
[0007]
  Such division processing is called tiling and is an effective technique for saving memory and speeding up. However, as described in Non-Patent Document 1, the compression / decompression processing is performed under a high compression ratio. In this case, there is a problem that discontinuity of tile boundaries (tile boundary distortion) is easily noticeable in the expanded image.
[0008]
  In order to solve this tile boundary distortion, it is also effective to perform processing by overlapping the boundaries between adjacent tiles. However, since the basic specification of JPEG2000 (JPEG2000 Part 1) stipulates that adjacent tile boundaries do not overlap, the method of overlapping tile boundaries is not preferable from the viewpoint of complying with the regulations.
[0009]
  Further, similarly to the conventional technique, it is possible to expect suppression of tile boundary distortion by applying a low pass filter to the tile boundary. With this method, there is no restriction on the regulation as described above.
[0010]
  In the present applicant, a method for controlling the smoothing degree of the low-pass filter according to the distance from the tile boundary; a technique for gradually decreasing the smoothing degree of the low-pass filter as the distance from the tile boundary increases; A method of calculating an edge amount for a pixel in the vicinity of and controlling the smoothing degree of the low-pass filter in accordance with the edge amount; a method of gradually decreasing the smoothing degree of the low-pass filter as the edge amount increases; Furthermore, the method of controlling the low-pass filter according to the distance from the tile boundary and the edge amount at the pixel, that is, the smoothness of the low-pass filter is weakened as the distance from the tile boundary increases and the edge amount increases. A method has been proposed (Japanese Patent Application No. 13-400767). By applying this method, it is possible to prevent a band-like blurred image from being generated even when the edge amount near the tile boundary is large while suppressing the tile boundary distortion. For the purpose of speeding up the processing and reducing the processing amount, the applicant of the present application applies a low-pass filter that drops the frequency component in the horizontal direction to the vertical tile boundary, and sets the vertical tile boundary to the vertical tile boundary. There has also been proposed a method of applying a low-pass filter that drops the frequency component in the direction and applying a low-pass filter that drops the frequency component in the vertical and horizontal directions at the intersection of the tile boundaries (Japanese Patent Application No. 2002-109890).
[0011]
  However, it has been found that the conventional technique and the proposed method cannot always achieve the expected effect because of the peculiarity of tile boundary distortion of JPEG2000. In order to optimally suppress the tile boundary distortion of JPEG2000, as will be described in detail below, it is indispensable to sufficiently study the objectivity of the low-pass filter applied to the tile boundary pixels.
[0012]
  Therefore, an object of the present invention is to provide a new image processing apparatus and method that are extremely effective for suppressing tile boundary distortion of JPEG2000.
[0013]
[Means for Solving the Problems]
  Hereinafter, the present invention will be described with reference to the accompanying drawings in connection with the peculiarities of tile boundary distortion of JPEG2000.
[0014]
  FIG. 1 is a block diagram showing the flow of basic compression / decompression processing of JPEG2000. Image data to be subjected to compression processing is divided into tiles for each component and input to the color conversion / inverse color conversion unit 1000, where color conversion (described later) is performed. The tile image after color conversion is subjected to two-dimensional discrete wavelet transformation (forward transformation) by a two-dimensional wavelet transformation / inverse transformation unit 1001, and the obtained wavelet coefficients are entropy-coded after being quantized by a quantization unit 156. Encoded by the decoding unit 1003 In JPEG2000, both lossless compression (lossless compression) and lossy compression (lossy compression) are possible. In the case of lossless compression, the quantization step width is always 1, and substantially no quantization is performed. The entropy coding in the entropy coding / decoding unit 1003 uses a block-based bit-plane coding method called EBCOT (Embedded Block Coding with Optimized Truncation) consisting of block division, coefficient modeling, and binary arithmetic coding. . The wavelet coefficient to be encoded is an integer with a positive or negative sign (or an integer expressed as a real number), and scans them in a predetermined order, while the coefficients are expressed in absolute values, from the upper bits to the lower bits Encoding processing is executed in bit plane units. The code string generated by the entropy encoding / decoding unit 1003 is output as a bit stream of a predetermined format by the tag processing unit 1004.
[0015]
  The decompression process is the reverse of the compression process. The compressed image data (code data) is input to the tag processing unit 1004 and decomposed into code strings of each tile of each component. This code string is entropy-decoded by the entropy encoding / decoding unit 1003. The decoded wavelet coefficients are dequantized by the quantization / inverse quantization unit 1002 and then subjected to the two-dimensional inverse wavelet transform by the two-dimensional wavelet transformation / inverse transformation unit 1001, whereby each tile of each component. Images are played back. Each tile image of each component is input to the color space conversion unit 152, subjected to reverse color conversion processing, and output as a tile image composed of components such as RGB.
[0016]
  The image processing apparatus or method according to the present invention is subjected to suppression of tile boundary distortion or reverse color conversion processing in the image data 1201 (RGB value or CMY value) after reverse color conversion by the color conversion / reverse color conversion unit 1000. This is applied to suppression of tile boundary distortion in image data 1202 (luminance value Y, color difference values Cb, Cr) before image processing.
[0017]
  FIGS. 2 to 6 show a two-dimensional (vertical and horizontal) wavelet transform called a 5 × 3 transform adopted in JPEG2000 for a 16 × 16 monochrome image (or one component image of a color image). It is a figure for demonstrating the process to give. FIG. 2 shows an original image before conversion. As shown in the figure, XY coordinates are taken, and for a certain x, the pixel value of a pixel whose Y coordinate is y is expressed as P (y) (0 ≦ y ≦ 15). In JPEG 2000, first, a coefficient C (2i + 1) is obtained by applying a high-pass filter in the vertical direction (Y-coordinate direction) around a pixel whose Y-coordinate is an odd number (y = 2i + 1). Next, a low-pass filter is applied around the pixel whose Y coordinate is an even number (y = 2i) to obtain a coefficient C (2i) (this is performed for all x). Here, the high-pass filter and the low-pass filter are expressed by Expression (1) and Expression (2), respectively. The floor (x) in the expression is a floor function of x (a function that replaces the real number x with an integer that does not exceed x and is closest to x).
[0018]
  C (2i + 1) = P (2i + 1) −floor ((P (2i) + P (2i + 2)) / 2) Equation (1)
  C (2i) = P (2i) + floor ((C (2i-1) + C (2i + 1) +2) / 4) Equation (2)
[0019]
  Note that there may be no adjacent pixel group at the edge of the image with respect to the central pixel, and in this case, a deficient pixel value is compensated by a technique called “mirroring”. Mirroring is an operation that literally folds the pixel value around the boundary, and regards the folded value as the value of the adjacent pixel group. FIG. 7 is a schematic diagram of pixel value (or coefficient value) mirroring. This figure shows how pixel values indicated by hatched circles are supplemented with pixel values indicated by white circles in the tile when the pixel P at the right boundary of the tile is converted.
[0020]
  For simplicity, if the coefficient obtained by the high-pass filter is expressed as H and the coefficient obtained by the low-pass filter is expressed as L, the image in FIG. 2 is converted into the L coefficient and the H coefficient as shown in FIG. Converted to an array.
[0021]
  Subsequently, with respect to the coefficient array of FIG.x= 2i + 1) is applied to the center of the coefficient, and then the low-pass filter is applied to the center of the coefficient whose X coordinate is an even number (x = 2i). (P, (2i), etc. in (1) and (2) are read as those representing coefficient values).
[0022]
  For simplicity, LL is obtained by applying a low-pass filter centered on the L coefficient, HL is obtained by applying a high-pass filter centered on the L coefficient, and is obtained by applying a low-pass filter centered on the H coefficient. 3 is converted into a coefficient array as shown in FIG. 4, if LH is a coefficient obtained, and HH is a coefficient obtained by applying a high-pass filter around the H coefficient. Here, coefficient groups with the same symbol are called subbands, and FIG. 4 is composed of four subbands.
[0023]
  With the above processing, one wavelet transform (one decomposition (decomposition)) is completed, and only the LL coefficients are collected (collected for each subband as shown in FIG. 5 and only the LL subband is taken out). ), An “image” having a resolution half that of the original image is obtained. In this way, the classification for each subband is called deinterleaving, and the arrangement in the state shown in FIG. 4 is called interleaving.
[0024]
  The second wavelet transform is performed by the same processing as described above, regarding the LL subband as an original image. The processing resultDeWhen interleaved, subband coefficients as shown in FIG. 6 are obtained. The prefixes 1 and 2 of the coefficients in FIGS. 5 and 6 indicate how many times the wavelet transform has been obtained, and are called a decomposition level. Therefore, the resolution is lower as the subband has a higher decomposition level. In the above discussion, when only one-dimensional wavelet transform is desired, processing in only one direction may be performed.
[0025]
  On the other hand, in the wavelet inverse transform, an inverse low-pass filter is first applied to the array of interleaved coefficients as shown in FIG. 4 in the horizontal direction, centering on the coefficient whose X coordinate is an even number (x = 2i), and then X An inverse high-pass filter is applied around a coefficient whose coordinates are odd (x = 2i + 1). This is done for all y. Here, the inverse low-pass filter and the inverse high-pass filter are expressed by the following equations (3) and (4), respectively. As in the case of the wavelet transform (forward transform), there may be no adjacent coefficient group for the center coefficient at the edge of the image. In this case, the coefficient value is supplemented by mirroring.
[0026]
  P (2i) = C (2i) −floor ((C (2i-1) + C (2i + 1) +2) / 4) Equation (3)
  P (2i + 1) = C (2i + 1) + floor ((P (2i) + P (2i + 2)) / 2) Equation (4)
[0027]
  Through the above processing, the coefficient array in FIG. 4 is converted (inversely converted) into a coefficient array as shown in FIG. Subsequently, similarly, in the vertical direction, an inverse low-pass filter is applied centering on a coefficient whose Y coordinate is an even number (y = 2i), and then an inverse high-pass filter is centered on a coefficient whose Y coordinate is an odd number (y = 2i + 1). Apply. By performing this operation for all x, one wavelet inverse transformation is completed and the image of FIG. 2 is reconstructed. If wavelet transformation is performed a plurality of times, FIG. 2 is regarded as an LL subband, and an image is reconstructed by repeating similar inverse transformation using other coefficients such as HL.
[0028]
  In the wavelet transform as described above, the tile size as a processing unit takes a value of “power of 2” (that is, an even number) so that the resolution component of “one power of 2” of the original image can be easily obtained. In addition, the origin of the XY coordinates is easily taken at the apex of the image as shown in FIG. Under such conditions, when a filter is applied in the vertical direction as shown in FIG. 3 (as is apparent when the entire image in FIG. 2 is considered as one tile), the pixels adjacent to the upper end of the tile are low-pass. In this filter, a pixel adjacent to the lower end is subjected to a high-pass filter. Similarly, when a filter is applied in the horizontal direction as shown in FIG. 4, a low-pass filter is applied to a pixel adjacent to the left end of the tile, and a high-pass filter is applied to a pixel adjacent to the right end.
[0029]
  Here, as pointed out by Non-Patent Document 1, when a mirroring is performed, if a quantization error occurs in the H coefficient at the lower end of the tile boundary, the error appears as a pixel value error in the final decompressed image. It is known. Similarly, it is known that when a quantization error occurs in the H coefficient at the right end of the tile boundary, the error appears as a final pixel value error. On the other hand, it is known that even if a quantization error occurs in the L coefficient at the upper and left ends of the tile boundary, the error does not appear as a large error in the pixel value of the final decompressed image.
[0030]
  Therefore, when the average value of the pixel value errors occurring in the pixels in the tile is taken, the distribution is as shown in FIG. However, the distribution is for eight tiles that are continuous in the left-right and up-down directions, in which FIG. As can be seen from FIG. 8, at the bottom edge of the tile rather than the top edge of the tile,Left edgeThan tileRight edgeThus, the error takes a remarkably large value. That is, when the tile boundary is the center, the average error of the pixel value is extremely asymmetric.
[0031]
  When a filter having a weight symmetrical to the tile boundary direction is used under such an error distribution, for example, as shown in FIG. 9B, four adjacent tiles as shown in FIG. When the low-pass filter is applied to the vertical boundary at the right end of the tile 0 and the low-pass filter as shown in FIG. 9C is applied to the horizontal boundary at the lower end of the tile 1, it matches the asymmetric error described above. In some cases, a relatively large error occurs in the pixel value after smoothing. As is clear from FIG. 9B, the maximum weighting (= 4) is performed on a pixel having a large error. This is because only a half weight (= 2) is given to a pixel having an error smaller than that. In other words, the filter weighting factor itself may be determined based on the pixel error asymmetry and the magnitude relationship expected in advance.
[0032]
  The image processing apparatus according to the first aspect of the present invention is to effectively suppress the tile boundary distortion in consideration of the asymmetry of the error generated at the tile boundary as described above. )EveryCompressed / decompressed using wavelet transformIn order to suppress distortion at the division unit boundary in the image, a low pass filter is applied to the pixels near the division unit boundary.An image processing apparatus for performingThe weighting factor of the low-pass filter is asymmetric with respect to the direction of the division unit boundary.
[0033]
  An image processing apparatus according to a second aspect of the present invention is intended to effectively suppress tile boundary distortion in consideration of the magnitude relationship of errors occurring at tile boundaries, and is a predetermined division unit.(tile)EveryCompressed / decompressed using wavelet transformIn order to suppress distortion at the division unit boundary in the image, a low pass filter is applied to the pixels near the division unit boundary.An image processing apparatus for performingFor weighting factor of low-pass filter, Taken for pixels with equal distance from the vertical or horizontal division boundaries in the division unit (tile)AverageNaReflects the magnitude relationship of errorsMakeIt is characterized by this.
[0034]
  Further, the filter as described above needs to be applied to pixels in a predetermined range from the boundary where the tile boundary distortion extends. For example, when the 5 × 3 filter is used at the decomposition level 3, it may be applied to about 8 pixels near the boundary. As is clear from FIG. 9B, the target pixel to be filtered is from the tile boundary. As the distance increases, the weight of the filter multiplied by the H coefficient (arrow coefficient) having the largest error naturally decreases. That is, the asymmetry of the error depends on the distance between the target pixel and the boundary. For example, as the target pixel moves away from the boundary, the asymmetry of the filter may be weakened (may be closer to a symmetric filter). A symmetric low-pass filter is not preferable for the asymmetric error, but when the error asymmetry is small, the use of a symmetric filter may be less harmful and safe.
[0035]
  In view of this point, the invention according to claim 3 is the image processing apparatus according to claim 1 or 2, wherein the weight coefficient of the low-pass filter is asymmetric orSaid to the weighting factor of the low pass filterAverageNaThe degree of reflection of the magnitude relationship between errors isVertical or horizontal division units (tile)It depends on the distance from the boundary. In other words, the weight coefficient of the low-pass filter is controlled according to the distance from the tile boundary, and the tile boundary distortion is more effectively suppressed.
[0036]
  As is clear from FIG. 9B, when the target pixel moves away from the tile boundary, the range covered by the filter deviates from the tile boundary. In this case, the premise that “the asymmetry of the error around the tile boundary” described above is not satisfied.
[0037]
  The invention described in claim 4 focuses on this point and tries to suppress the tile boundary distortion more effectively. In the image processing apparatus according to claim 1 or 2, the tap of the low-pass filter isDivision unit (tile)Only when crossing the boundary, the weighting factor of the low-pass filter is made asymmetric or the weighting factor of the low-pass filterNaIt is characterized by reflecting the magnitude relationship of errors.
[0038]
  Further, as shown in FIG. 10, even if the pixel is adjacent to the tile boundary, when the position corresponds to the L coefficient, the filter weight coefficient is usually set small from the beginning. Even if the error is asymmetric, a symmetric filter with less harmful effects may be sufficient.
[0039]
  In view of this point, the invention according to claim 5 is the pixel of interest in the image processing apparatus according to claim 1 or 2.The average atOnly when the error is large, the weighting factor of the low-pass filter is asymmetric, or the weighting factor of the low-pass filterNaIt is characterized by reflecting the magnitude relationship of errors. That is, tile reflection is more effectively suppressed by performing the reflection only on pixels at positions where the average error is large.
[0040]
  Now, for example, when compression is performed on a color image composed of three RGB components, wavelet transformation can be performed with the RGB values as they are. However, in order to increase the compression rate, it is usual to perform color conversion to convert three new components of one luminance and two color differences, and individually perform wavelet conversion on the converted components. .
[0041]
  As an example of color conversion adopted in JPEG2000, there is the following conversion called RCT (Reversible Component Transform).
[0042]
      Luminance Y = floor ((R + 2G + B) / 4)
      Color difference Cr = RG
      Color difference Cb = BG Formula (5)
Its inverse transform is
      R = G + Cr
      G = Y-floor ((Cr + Cb) / 4)
      B = Cb + G Formula (6)
It is.
[0043]
  Here, the degree of asymmetry of the error differs between RGB components and between luminance / color difference components. The difference depends on the quantization method. For example, in the case of a luminance / color difference component, when the color difference is extremely quantized, the asymmetry may be larger in the color difference than in the luminance.
[0044]
  In view of this point, the invention according to claim 6 is the image processing apparatus according to claim 1 or 2, wherein the weighting coefficient of the low-pass filter is asymmetric or the weighting coefficient of the low-pass filter.To the aboveAverageNaThe degree of reflection of the magnitude relation of the error is different depending on the image component. In other words, by controlling the reflection by the component, the tile boundary distortion in the color image is more effectively suppressed.
[0045]
  The asymmetry is caused by the quantization error of the wavelet coefficient at the time of compression, and becomes larger as the compression rate becomes higher.
[0046]
  Therefore, the invention according to claim 7 is the image processing apparatus according to claim 1 or 2, wherein the weighting factor of the low-pass filter is asymmetric orSaid to the weighting factor of the low pass filterAverageNaThe degree of reflection of the magnitude relationship of errors differs depending on the compression rate of the compressed image. That is, the reflectionDegree ofThe tile boundary distortion is more effectively suppressed by controlling in accordance with the compression rate.
[0047]
  The degree of asymmetry depends on the mirroring range in wavelet transform (forward / reverse transform), that is, the tap length of the wavelet filter, and depends on the frequency characteristics. The degree gets bigger.
[0048]
  Therefore, the invention according to claim 8 is the image processing apparatus according to claim 1 or 2, wherein the weighting factor of the low-pass filter is asymmetric orSaid to the weighting factor of the low pass filterAverageNaThe degree of reflection of the magnitude relationship between errors isWavelet transformIt depends on the type of wavelet filter used in the above. That is, the reflectionDegree ofThis is to suppress the tile boundary distortion more effectively by performing according to the difference of the wavelet filter.
[0049]
  Now, as discussed in terms of asymmetry and reflection of the average error, it is very effective to use the method proposed by the applicant of the present invention in combination with the present invention. As is apparent from FIG. 8, the error that occurs in the pixel tends to decrease as the distance from the tile boundary increases. Therefore, the smoothness of the low-pass filter should also decrease as the distance increases. Also, as in the previous discussion of asymmetry, the discontinuity itself, that is, the tile boundary distortion, depends on the component, the compression rate, and the tap length of the wavelet filter. In particular, when the luminance / color difference system is YCbCr, it is experimentally known that the error in Cb is less perceptible than the error in Cr due to human visual characteristics. In itself, Cb may be weak. The low-pass filter is for suppressing the tile boundary distortion. However, if there is an actual edge (that reflects the original image) that is not distorted at the tile boundary, the actual edge is also smoothed. May occur.
[0050]
  Therefore, according to a ninth aspect of the present invention, in the image processing apparatus according to the first or second aspect, the frequency characteristic of the low-pass filter is a division unit.Vertical or horizontal directionIt depends on the distance from the boundary. As described above, the tile boundary distortion can be more effectively suppressed by changing the frequency characteristic of the low-pass filter in accordance with the distance from the boundary.
[0051]
  The image processing apparatus according to claim 10 is characterized in that in the image processing apparatus according to claim 1 or 2, the frequency characteristics of the low-pass filter differ depending on the component of the image. It can be effectively suppressed.
[0052]
  According to an eleventh aspect of the present invention, in the image processing apparatus according to the first or second aspect, the frequency characteristics of the low-pass filter depend on the compression rate of the compressed image. Thus, tile boundary distortion can be more effectively suppressed by providing the low-pass filter with frequency characteristics corresponding to the compression rate.
[0053]
  The invention according to claim 12 is the image processing apparatus according to claim 1 or 2, wherein the frequency characteristic of the low-pass filter isWavelet transformThe tile boundary distortion can be more effectively suppressed by being different depending on the type of wavelet filter used in the above.
[0054]
  According to a thirteenth aspect of the present invention, in the image processing apparatus according to the first or second aspect, the frequency characteristics of the low-pass filter depend on the edge degree near the division unit boundary, and the edge existing in the original image Thus, the tile boundary distortion is effectively suppressed without causing any adverse effects.
[0055]
  The invention described in claim 14 is the image processing apparatus according to claim 9, wherein the frequency characteristic of the low-pass filter also depends on the edge degree near the division unit boundary. In this way, by considering both the distance from the boundary and the edge that actually exists in the original image, the tile boundary distortion can be more effectively suppressed without causing any adverse effects on the actual edge.
[0056]
  The invention described in claim 15 is the image processing apparatus according to any one of claims 1 to 14, wherein the low-pass filter is applied after the inverse color conversion of the expanded image (applied to pixel values such as RGB). And has an advantage that the calculation for suppressing the tile boundary distortion is simplified.
[0057]
  According to a sixteenth aspect of the present invention, in the image processing device according to any one of the first to fourteenth aspects, a low-pass filter is applied before the inverse color conversion of the expanded image (the luminance value and the color difference value such as YCrCb). This is particularly advantageous when the edge degree needs to be calculated.
[0058]
  Further, the invention according to claim 17 is provided for each predetermined division unit (tile).Wavelet transform Compressed and decompressed usingIn order to suppress distortion at the division unit boundary in the image, a low pass filter is applied to the pixels near the division unit boundary.An image processing method to be applied,The weighting factor of the low-pass filter is asymmetric with respect to the direction of the division unit boundaryage,Tile boundary distortion can be effectively suppressed by taking into consideration the asymmetry of the error occurring at the boundary.
[0059]
  The invention according to claim 18 is provided for each predetermined division unit (tile).Compressed / decompressed using wavelet transformIn order to suppress distortion at the division unit boundary in the image, a low pass filter is applied to the pixels near the division unit boundary.An image processing method to be applied,The weighting factor of the low-pass filter isFor pixels with the same distance from the vertical or horizontal division boundary within the division unitAverageNaReflects the magnitude relationship of errorsage,Tile boundary distortion can be effectively suppressed by considering the magnitude relationship of errors occurring at the boundary.
[0060]
  The invention according to claim 19 is the image processing according to any one of claims 1 to 16.As a deviceA program that causes a computer to function. The invention according to claim 20 is a computer-readable recording medium on which the program according to claim 19 is recorded.
[0061]
DETAILED DESCRIPTION OF THE INVENTION
  The embodiment of the present invention described above will be described with reference to FIGS. Although the present invention can be implemented by any of hardware, software, and combinations thereof, an embodiment in which the present invention is implemented by software on a general-purpose computer system such as a personal computer will be described here. Therefore, each step in the following description and its processing content should be understood as corresponding means and functions included in the image processing apparatus of the present invention.
[0062]
  FIG. 11 is a simplified block diagram of a computer system for explaining one embodiment of the present invention. This computer system has a schematic configuration in which a CPU 1100, a main memory 1102, a hard disk device 1104, and a monitor device 1106 are connected by a system bus 1108. The main memory 1102 stores an image processing program 1110 for executing the functions of the image processing apparatus of the present invention or the processing procedure of the image processing method. The overall flow of processing is as follows.
[0063]
  (1) In accordance with a command from the CPU 1100, image data obtained by compressing an original image by JPEG2000 from the hard disk device 1104 is read into the main memory 1102. (2) The CPU 1100 reads compressed image data from the main memory 1102 according to the image processing program 1110, performs decompression processing, executes processing for suppressing tile boundary distortion on the expanded image data, and suppresses tile boundary distortion. The generated image data is generated on the main memory 1102. (3) The image data in which the tile boundary distortion is suppressed is displayed on the monitor device 1106 or stored in the hard disk device 1104 according to a command from the CPU 1100. Of course, the present invention includes a mode in which compressed image data is captured and processed via a network such as the Internet or a LAN.
[0064]
  A more detailed procedure of the processing stage (2) is shown in FIG. Here, it is assumed that the original image is divided into four tiles having the same number of even pixels in the vertical and horizontal directions. Therefore, the right end and the lower end of the tile are the positions of the H coefficient, and the left end and the upper end of the tile are the positions of the L coefficient. In addition, here, in the compression processing of the compressed image data, a case where the wavelet transform of Decomposition 3 is performed using a 5 × 3 wavelet filter will be described. (The case where a 9 × 7 wavelet filter is used will also be described later).
[0065]
  In step 1400 of FIG. 12, the compressed image of all tiles is decompressed to obtain the characteristic value of the pixel. The characteristic values are luminance / color difference values such as RGB values and YCbCr. In relation to FIG. 1, when tile boundary distortion suppression is performed on the expanded image data 1201, RGB values are characteristic values, and when tile boundary distortion suppression is performed on the expanded image data 1202, luminance Y and color difference Cb. Cr is a characteristic value. The characteristic value of the pixel of the expanded image is stored in a specific area of the main memory 1102. This step or means is a step or means for decompression processing, and the next steps 1401 to 1404 are processing steps or means for suppressing tile boundary distortion.
[0066]
  In step 1401, a characteristic value of a pixel within a predetermined distance from the tile boundary is subjected to a low pass filter for suppressing tile boundary distortion. Here, the distance from the tile boundary is the shortest distance from the pixel to the boundary as shown in FIG. In the example shown here, description will be made assuming that a low-pass filter is applied to a pixel whose distance from the boundary is 3 or less. The reason why the distance is 3 or less is to reduce the amount of calculation. The description will be made assuming that the tap length of the low-pass filter is 5.
[0067]
  In step 1402, the characteristic value of the pixel after applying the low pass filter is stored in a specific area of the main memory 1102.
[0068]
Is completed (step 1403, Yes), the characteristic value of the corresponding pixel of the decompressed image stored in step 1400 is replaced with the characteristic value stored in step 1402. Thus, image data in which tile boundary distortion is suppressed is obtained.
[0069]
  A more detailed procedure of step 1401 in FIG. 12 will be described with reference to the flowchart of FIG.
[0070]
  First, if the characteristic value to which the low-pass filter is applied is luminance or G, k = 5 is set (steps 2000 and 2001), and if the color difference is Cb, Cr or B or R, k = 4 is set (steps 2000 and 2002). This k is a value for determining the weight of the low-pass filter at the target pixel position (weight of the filter center). If the distance from the tile boundary of the target pixel is d,
      Weight of low-pass filter = k + 64d
Given in. That is, in this example, the weight of the center of the low-pass filter is changed according to the distance from the boundary (claim 9). Also,
        For luminance Y or G, k = 5
        For color difference Cb or B, k = 4
        For color difference Cr or R, k = 4
The weight of the center of the low-pass filter is changed depending on the component (claims 10, 15, 16).
[0071]
  In the next step 2003, a characteristic value of a pixel whose distance from the tile boundary is 3 or less is input. In step 2004, it is checked whether the pixel of interest is a pixel in contact with the tile boundary intersection. If the pixel is in contact with the tile boundary intersection, a low-pass filter having a weighting coefficient as shown in FIG. 17 is applied (step 2005). If the pixel of interest does not touch the tile boundary intersection, it is checked whether the distance from the tile boundary in the vertical direction is within 3 (step 2006). If so, a weighting coefficient as shown in FIG.HaveA low-pass filter is applied (step 2007), and if so, a low-pass filter having a weighting coefficient as shown in FIG. 16 is applied (step 2008). However, the weight at the center of the low-pass filter shown in FIGS. 15, 16, and 17 is a value when applied to the luminance or G component. In this example, a one-dimensional or cross-shaped low-pass filter is applied to reduce the amount of calculation, but it is needless to say that a two-dimensional low-pass filter can also be applied.
[0072]
  As can be seen from FIG. 15, the weighting factor of the low-pass filter is asymmetrical in the boundary direction only when the tap of the low-pass filter crosses the tile boundary in the vicinity of the right end of the tile. Further, in order to reflect the magnitude of the average error in the low-pass filter, the weight of the H coefficient position of the low-pass filter is reduced (part 2100) and the weight of the L position is increased (part 2101). ). Further, in the vicinity of the left end of the tile, a symmetrical low-pass filter is applied even when the tap crosses the tile boundary. In the low-pass filters shown in FIGS. 16 and 17, weights are set in the same way.
[0073]
  In the above description, pixels using asymmetric filters exist for all components. However, it is possible to use all symmetrical filters without performing this for luminance. The low-pass filter for such an embodiment is not shown because it is clear from the above description.
[0074]
  Also, the compression rate is checked in step 1401 (FIG. 12), and an asymmetric filter is used only when the compression rate exceeds 20, and a symmetric filter with less harmful effects is used when the compression rate is 20 or less. It is possible (Claim 7).
[0075]
  Furthermore, in this aspect, when the compression rate is 20 or less,
      For luminance Y or G, k = 5
      K = 4 for color difference Cb or B
      K = 4 for color difference Cr or R
If the compression ratio exceeds 20,
      K = 4 for luminance Y or G
      K = 3 for color difference Cb or B
      K = 3 for color difference Cr or R
(Claim 11). This is because in general, a stronger low-pass filter should be applied as the compression ratio increases. The weight and processing flow of the low-pass filter for such an aspect are clear from the above description and are not shown.
[0076]
  In addition, priority is given to accuracy over calculation amount, and as shown in FIG. 18, even when the tap crosses the tile boundary in the vicinity of the left end of the tile, the filter can be asymmetrical. Included in the invention.
[0077]
  The example of the low-pass filter in the case of the 5 × 3 wavelet transform has been described above.
[0078]
  In the case of 9 × 7 wavelet transform, a low-pass filter having a configuration as shown in FIG. 19 or 20 can be applied (claims 8 and 12). The low-pass filter shown in FIGS. 19 and 20 is an example of a filter applied to the tile boundary in the vertical direction. If this filter is rotated, the filter is applied to the tile boundary in the horizontal direction. When combined, a filter applied to the tile boundary intersection can be obtained as in FIG.
[0079]
  In step 1401 of FIG. 12, an edge degree calculation filter as shown in FIG. 21 is applied to each target pixel to detect the edge degree, and the weight of the center of the low-pass filter shown in FIGS. It can also be set as a function of edge degree and distance from tile boundaries.13, 14). FIG. 22 is an explanatory diagram of such a filter. In the example shown here,
If the distance d = 0,
If the absolute value abs (e) of the edge amount calculated by the edge degree calculation filter, that is, the edge degree is 255 or more, the weight m at the center of the low-pass filter is set.
    m = 5 + abs (E)
If the edge degree is less than 255
    m = 5
It is said.
If the distance d> 0,
    m = max (5 + 64d, 5 + abs (E))
It is said. Such filter weight control can effectively suppress tile boundary distortion while preserving edges that actually exist in the original image. Note that the low-pass filter shown in FIG. 22 assumes a case of 5 × 3 wavelet transform.
[0080]
  Note that the low-pass filter coefficients are normalized so that the sum of the weights is 1. That is,
    a b c
    d e f
    g hi
In the center pixel of 3 × 3 pixels having a pixel value of (a pixel having a pixel value of e)
    o p q
    r st
    u v w
When the low-pass filter is applied using the weight, the weight is multiplied by the pixel value in a normalized state so that the sum of them becomes 1
Σ (o + p + q + r + s + t + u + v + w) = X
The value of the center pixel after applying the low-pass filter is
(Ao + bp + cq + dr + es + ft + gu + hv + iw) / X
It becomes. The same applies to the case where the filter mask shape is 1 × n and the cross shape is normalized so that the sum of the weights of the low-pass filter is 1.
[0081]
  A program (not limited to an application program, including a device driver such as a printer driver) for realizing the processing procedure or the function of the corresponding means in the embodiment described above on the computer, and various records in which the program is recorded (Storage) media are also encompassed by the present invention.
[0082]
【The invention's effect】
  As described in detail above, according to the present invention, it is possible to extremely effectively suppress tile boundary distortion on an image expanded after being compressed by an image compression algorithm compliant with JPEG2000, The effect is great.
[Brief description of the drawings]
FIG. 1 is a block diagram for explaining the flow of JPEG2000 compression / decompression processing;
FIG. 2 is a diagram showing an original image and a coordinate system.
3 is a diagram illustrating a coefficient array obtained by performing one-dimensional wavelet transform in the vertical direction on the original image of FIG. 2;
4 is a diagram showing a coefficient array obtained by performing a one-dimensional wavelet transform in the horizontal direction on the coefficients of FIG. 3; FIG.
FIG. 5 is a diagram showing a coefficient array in which the coefficients in FIG. 4 are rearranged.
FIG. 6 is a diagram showing a coefficient array in which coefficients obtained by a two-dimensional wavelet transform at a decomposition level 2 are rearranged.
FIG. 7 is an explanatory diagram of mirroring at a tile boundary.
FIG. 8 is a diagram illustrating a distribution of mean square errors of pixel values generated in pixels in a tile.
FIG. 9 is a diagram illustrating an example of a symmetric low-pass filter applied to vertical and horizontal tile boundaries.
FIG. 10 is a diagram illustrating application of a low-pass filter when a target pixel is located at an L coefficient position.
FIG. 11 is a simplified block diagram for explaining an embodiment of the present invention.
FIG. 12 is a flowchart for explaining an embodiment of the present invention;
FIG. 13 is a diagram for explaining a distance from a tile boundary.
FIG. 14 is a flowchart for explaining the processing procedure of step 1401 in FIG. 12;
FIG. 15 is a diagram illustrating an example of a low-pass filter applied to a vertical tile boundary in the present invention.
FIG. 16 is a diagram illustrating an example of a low-pass filter applied to a horizontal tile boundary in the present invention.
FIG. 17 is a diagram illustrating an example of a low-pass filter applied to a pixel in contact with a tile boundary intersection in the present invention.
FIG. 18 is a diagram showing another example of a low-pass filter applied to a vertical tile boundary in the present invention.
FIG. 19 is a diagram showing another example of a low-pass filter applied to a vertical tile boundary in the present invention.
FIG. 20 is a diagram illustrating another example of a low-pass filter applied to a vertical tile boundary in the present invention.
FIG. 21 is a diagram illustrating an example of an edge amount calculation filter.
FIG. 22 is a diagram showing another example of a low-pass filter applied to a tile boundary in the present invention.
[Explanation of symbols]
  1000 color conversion / reverse color conversion unit
  1001 Two-dimensional wavelet transform / inverse transform unit
  1002 Quantization / inverse quantization unit
  1003 Entropy encoding / decoding unit
  1004 Tag processing section
  1100 CPU
  1102 main memory
  1104 Hard disk device
  1106 Monitor device
  1110 Image processing program

Claims (20)

An image processing apparatus that applies a low-pass filter to pixels in the vicinity of a division unit boundary in order to suppress distortion of the division unit boundary in an image compressed and decompressed using wavelet transform for each division unit, the weight of the low-pass filter An image processing apparatus, wherein the coefficient is asymmetric with respect to the direction of the division unit boundary. An image processing apparatus that applies a low-pass filter to pixels in the vicinity of a division unit boundary in order to suppress distortion of the division unit boundary in an image compressed and decompressed using wavelet transform for each division unit, the weight of the low-pass filter the coefficients, the image processing apparatus, characterized in that to reflect the average error magnitude of the distance taken for the same pixels from the vertical or horizontal division boundary in the dividing unit. The image processing apparatus according to claim 1 or 2, wherein the degree of reflection of the average error magnitude relationship to the degree or the weighting coefficient of the low-pass filter of the asymmetric weighting coefficients of the low-pass filter, the vertical division unit An image processing apparatus that depends on a distance from a boundary in a direction or a horizontal direction . The image processing apparatus according to claim 1 or 2, wherein the weighting coefficients of the low pass filter with an asymmetric or to make reflect the average error magnitude of the weighting coefficients of the low pass filter taps of the low-pass filter An image processing apparatus, characterized in that it is a case where the division unit boundary is crossed. The image processing apparatus according to claim 1 or 2, wherein the make reflect an average error magnitude of the weighting coefficient is asymmetric or the weighting coefficient of the low-pass filter of the low-pass filter, the average in the pixel of interest An image processing apparatus characterized by having a large error . 3. The image processing apparatus according to claim 1, wherein the degree of asymmetry of the weighting factor of the low-pass filter or the degree of reflection of the magnitude relationship of the average error in the weighting factor of the low-pass filter depends on the component of the image. An image processing apparatus characterized by being different. The image processing apparatus according to claim 1 or 2, the degree of reflection of the average error magnitude relationship to the degree or the weighting coefficient of the low-pass filter of the asymmetric weighting coefficients of the low pass filter, compressed image An image processing apparatus, which differs depending on the compression rate. The image processing apparatus according to claim 1 or 2, wherein the degree of reflection of the average error magnitude relationship to the degree or the weighting coefficient of the low-pass filter of the asymmetric weighting coefficients of the low-pass filter, using the wavelet transform the image processing apparatus being dependent on the type of wavelet filter to be. The image processing apparatus according to claim 1, wherein the frequency characteristic of the low-pass filter depends on a distance from a vertical or horizontal boundary of a division unit.   The image processing apparatus according to claim 1, wherein a frequency characteristic of the low-pass filter varies depending on an image component.   The image processing apparatus according to claim 1, wherein the frequency characteristics of the low-pass filter depend on a compression rate of the compressed image. 3. The image processing apparatus according to claim 1, wherein a frequency characteristic of the low-pass filter differs depending on a type of a wavelet filter used for wavelet transform . 4.   The image processing apparatus according to claim 1, wherein the frequency characteristic of the low-pass filter depends on an edge degree near a division unit boundary.   The image processing apparatus according to claim 9, wherein the frequency characteristic of the low-pass filter also depends on an edge degree near a division unit boundary.   The image processing apparatus according to claim 1, wherein the low-pass filter is applied after inverse color conversion of the decompressed image.   15. The image processing apparatus according to claim 1, wherein the low-pass filter is applied before inverse color conversion of the decompressed image. An image processing method for applying a low-pass filter to pixels in the vicinity of a division unit boundary in order to suppress distortion of the division unit boundary in an image compressed and decompressed using a wavelet transform for each division unit, the weight of the low-pass filter An image processing method, wherein the coefficient is asymmetric with respect to the direction of the division unit boundary. An image processing method for applying a low-pass filter to pixels in the vicinity of a division unit boundary in order to suppress distortion of the division unit boundary in an image compressed and decompressed using a wavelet transform for each division unit, the weight of the low-pass filter the coefficients, an image processing method, characterized in that to reflect the average error magnitude of the distance taken for the same pixels from the vertical or horizontal division boundary in the dividing unit. A program for causing a computer to function as the image processing equipment according to any one of claims 1 to 16.   A computer-readable recording medium on which the program according to claim 19 is recorded.

Images