JP4111447B2 - Image processing apparatus, image processing method, program, and information recording medium - Google Patents

Description

  The present invention relates to image processing. More specifically, the image is divided into tiles that do not overlap, and the frequency coefficient obtained by frequency conversion in units of tiles is subjected to inverse frequency conversion after quantization and inverse quantization. The present invention relates to detiling processing for removing tile boundary distortion of an obtained image.

  In recent years, with the advancement of image input / output technology, there is no end to the trend of high-definition images, and there is a strong demand for technology that can easily compress and expand high-definition images.

  One compression / decompression method that satisfies these requirements is JPEG2000, which can process a high-definition image by dividing it into smaller units and can decode a high-quality image even at a high compression rate. In JPEG 2000, it is possible to perform compression / decompression processing in a small memory environment by dividing an image into rectangular regions (tiles) that do not overlap. That is, each tile becomes a basic unit for executing the compression / decompression process, and the compression / decompression operation can be performed independently for each tile.

  Here, an outline of JPEG2000 will be described. FIG. 1 is a block diagram showing the basic flow of JPEG2000 compression (encoding) / decompression (decoding) processing.

  First, compression (encoding) processing will be described. For example, a color image composed of three RGB components is divided into one or more non-overlapping tiles for each component, and processing is performed for each tile of each component. First, for each tile, the color conversion / inverse color conversion unit 100 performs component conversion (color conversion) to luminance / color difference components, and then the two-dimensional wavelet conversion / inverse conversion unit 101 performs component conversion. A two-dimensional wavelet transform (discrete wavelet transform) is performed for each tile. The wavelet coefficients are quantized by the quantization / inverse quantization unit 102 as necessary for each subband, and then entropy coded in units of bit planes by the entropy coding / decoding unit 103 ( To be precise, the bit plane is subdivided into three sub-bit planes and encoded). Then, the code forming / tag processing unit 104 truncates unnecessary codes, generates packets by collecting necessary codes, arranges packets in a predetermined order, and adds necessary tags and tag information, thereby adding predetermined tags. A code stream (encoded data) of the format is formed.

  The color conversion / reverse color conversion unit 100 is not essential. JPEG2000 defines a reversible wavelet transform called a 5 × 3 transform and an irreversible wavelet transform called a 9 × 7 transform. When the 5 × 3 wavelet transform is used, quantization (scalar quantization) by the quantization / inverse quantization unit 102 is not performed, but truncation in the code formation / tag processing unit 104 is equivalent to coefficient quantization. . Therefore, in the specification and claims of the present application, the quantization of wavelet coefficients (frequency conversion coefficients in a broad sense) includes quantization by code truncation.

  The decompression (decoding) process is just the reverse of the compression process. The code stream is decomposed into a code stream of each tile of each component by the code formation / tag processing unit 104, entropy-decoded in bit plane units by the entropy encoding / decoding unit 103, and the decoded wavelet coefficients are quantized / inverted. Inverse quantization is performed by the quantization unit 102. Then, the two-dimensional wavelet transform / inverse transform unit 101 performs two-dimensional inverse wavelet transform on the wavelet coefficients for each component, and then the color transform / inverse color transform unit 100 performs inverse component transform (inverse color transform). ) Is restored to the original RGB pixel value.

  Now, the method of dividing the image into tiles that do not overlap and performing processing for each tile is an effective technique for saving memory and increasing the speed, but in an image obtained by expanding a code stream compressed under a condition with a high compression rate, There is a problem that the tile boundary becomes discontinuous.

  This tile boundary distortion is apparently similar to block distortion in discrete cosine transform (DCT).

Regarding removal of block distortion of this DCT, conventionally,
(A) A method of extracting a block boundary from a DCT component of an image divided into blocks and applying a one-dimensional low-pass filter perpendicular to the block boundary (see Patent Document 1),
(B) A method of determining the presence or absence of block distortion from the relationship between the direction of the block boundary and the direction of the edge, and applying a low-pass filter to pixels with block distortion (see Patent Document 2),
(C) A method of detecting block distortion and applying a low-pass filter to the detected block distortion (see Patent Document 3)
Etc. are known. All of these apply adaptive low-pass filtering to pixel values according to the direction and edge degree of the block boundary, smoothing the distortion at the block boundary, but smoothing to the true edge located at the block boundary Is intended to avoid. However, such a low-pass filter approach is a trade-off between “smoothing distortion (scheduled effect)” and “true edge smoothing (unplanned side effects)”, no matter how adaptive it is. Will be.

  On the other hand, in the compression / decompression system using the wavelet transform, the present applicant performs correction processing (detiling) of the high-pass coefficient adjacent to the tile boundary using, for example, the low-pass coefficient and the high-pass coefficient of the adjacent tile. A method for removing tile boundary distortion has been proposed (see Patent Document 4).

JP 05-316361 A Japanese Patent No. 2839987 JP 09-307855 A Japanese Patent Laid-Open No. 2001-257596

  The tile boundary distortion removal method described in Patent Document 4 is a technique similar to image restoration in a wavelet coefficient space. In image restoration, in order to restore lost information, a certain empirical constraint condition is adopted, and a solution to be restored is calculated based on the condition.

In the method described in Patent Document 4,
Tile boundary distortion occurs when the compression ratio is high, and all high-pass coefficients are quantized to 0 (approximate 1) regardless of the presence or absence of tiling.
・ The low-pass coefficient with tiling is equal to the low-pass coefficient without tiling (approximate 2)
The approximation or assumption
{Tile boundary pixel value when inverse wavelet transform is performed using a corrected high pass coefficient (quantized to 0)} =
{Pixel value at the same position when inverse wavelet transform is performed without tiling when the high-pass coefficient is 0}
From this equation, the correction value of the high-pass coefficient is calculated (note that the process of calculating a non-zero correction value for the high-pass coefficient quantized to 0 is “image restoration”).

  Described below is the derivation of the correction formula for obtaining the correction value, taking the JPEG 2000 5 × 3 wavelet transform as an example.

  FIGS. 2 to 5 show an example of a process in which wavelet transformation (positive transformation) called 5 × 3 transformation is performed in two dimensions (vertical direction and horizontal direction) on a 16 × 16 monochrome image.

  As shown in FIG. 2, xy coordinates are taken, and for a certain x, the pixel value of a pixel whose y coordinate is y is expressed as P (y) (0 ≦ y ≦ 15). In JPEG2000, a coefficient C (2i + 1) is obtained by first applying a high-pass filter in the vertical direction (y-coordinate direction) around a pixel having an odd y-coordinate (y = 2i + 1). Next, a coefficient C (2i) is obtained by applying a low-pass filter around a pixel having an even y coordinate (y = 2i) (this is performed for all x coordinates).

Here, the high-pass filter is expressed by equation (1), and the low-pass filter is expressed by equation (2). The symbol | _x_ | in the equation represents a floor function of x (a function that replaces the real number x with an integer that does not exceed x and is closest to x). step1 and step2 are the order in which they are applied.
C (2i + 1) = P (2i + 1) − | _ (P (2i) + P (2i + 2)) / 2_ | Equation (1) (step1)
C (2i) = P (2i) + | _ (C (2i-1) + C (2i + 1) +2) / 4_ | Equation (2) (step2)

  Note that there may be no adjacent pixel group at the edge of the image with respect to the central pixel. In this case, the insufficient pixel value is compensated by a technique called “mirroring” shown in FIG. . The mirroring is literally a process in which the pixel value is folded back symmetrically around the boundary, and the folded value is regarded as the value of the adjacent pixel group.

  For simplicity, if the coefficient obtained by the high-pass filter is denoted by H, and the coefficient obtained by the low-pass filter is denoted by L, the image in FIG. 2 is arranged as shown in FIG. Converted to.

Next, a high-pass filter is applied to the coefficient array shown in FIG. 3 in the horizontal direction, centering on coefficients whose x coordinate is an odd number (x = 2i + 1), and then the x coordinate is an even number (x = 2i). A low-pass filter is applied centering on the coefficient of (this is performed for all y). In this case, P (2i) and the like in the expressions (1) and (2) are read as those representing coefficient values.

For simplicity,
The coefficient obtained by applying a low pass filter around the L coefficient is LL,
The coefficient obtained by applying a high-pass filter around the L coefficient is HL,
The coefficient obtained by applying a low-pass filter around the H coefficient is LH,
HH is a coefficient obtained by applying a high-pass filter around the H coefficient,
3 is converted into a coefficient array as shown in FIG. Here, the coefficient group which attached | subjected the same symbol is called a subband. That is, FIG. 4 is composed of four subbands.

  When one wavelet transform (one decomposition (decomposition)) is completed and only LL coefficients are collected (collecting coefficients for each subband as shown in FIG. 5 and extracting only the LL subbands), An “image” having a resolution of 1/2 that of the original image is obtained (in this way, classification for each subband is called deinterleaving, and arrangement in the state shown in FIG. 3 is called interleaving).

  In the second wavelet transform, the above LL subband may be regarded as an original image and the same transformation as described above may be performed. In this case, when rearrangement is performed, a schematic FIG. 6 is obtained. The prefixes 1 and 2 of the coefficients in FIGS. 5 and 6 indicate how many times the coefficient has been obtained by the wavelet transform, and are called a decomposition level. In the above discussion, when only one-dimensional wavelet transform is desired, processing in only one direction may be performed.

On the other hand, in the wavelet inverse transform, an inverse low-pass filter is first applied to the array of interleaved coefficients as shown in FIG. 4 in the horizontal direction, centering on the coefficient whose x coordinate is an even number (x = 2i). Then, an inverse high-pass filter is applied around the coefficient whose x coordinate is an odd number (x = 2i + 1) (this is performed for all y). Here, the inverse low-pass filter is represented by Expression (3), and the inverse high-pass filter is represented by Expression (4). Similar to the wavelet positive transformation, there may be no adjacent coefficient group for the coefficient at the center of the edge of the image. In this case as well, the coefficient value is appropriately compensated by the mirroring of FIG.
P (2i) = C (2i) − | _ (C (2i−1) + C (2i + 1) +2) / 4_ | Equation (3) (step1)
P (2i + 1) = C (2i + 1) + | _ (P (2i) + P (2i + 2)) / 2_ | Formula (4) (step2)

  As a result, the coefficient array in FIG. 4 is converted (inversely converted) into a coefficient array as shown in FIG. Subsequently, in the vertical direction, an inverse low-pass filter is applied centering on a coefficient whose y coordinate is an even number (y = 2i), and then an inverse high pass filter is centered on a coefficient whose y coordinate is an odd number (y = 2i + 1). If applied (this is performed for all x), one wavelet inverse transformation is completed, and the image of FIG. 2 is returned (reconstructed). When wavelet transformation is performed a plurality of times, FIG. 2 is regarded as an LL subband, and similar inverse transformation may be repeated using other coefficients such as HL.

As described above, in the 5 × 3 wavelet inverse transform, for the coefficient sequence interleaved in the order of L, H, L, H,
Reverse low pass filter is applied to the center of the even position. Reverse high pass filter is applied to the center of the odd position.

  As can be seen from the above-described filter equations (1) to (4), which are developed by omitting the floor function, the number of taps is reversed between the forward transformation and the inverse transformation, and the filter coefficients are also interleaved.

Therefore, the positive conversion filter coefficient is high pass (HIGH): -0.5, 1, -0.5
Low pass (LOW): -0.125, 0.25, 0.75, 0.25, -0.125
In case of, the inverse transform filter coefficient is inverse low pass (LOW): -0.25, 1, -0.25
Reverse high path (HIGH): -0.125,0.5,0.75,0.5, -0.125
(Here, the filter coefficient is expressed without considering the floor function part).

“When tiling is not performed and all the high-pass coefficients are 0 (approximate 1)”, an interleaved coefficient sequence
L1 H1 L2 H2 L3 H3
The value obtained by applying an inverse high-pass filter to the center of the H2 position is
-0.125H1 + 0.5L2 + 0.75H2 + 0.5L3-0.125H3 = 0.5L2 + 0.5L3 (i)
It becomes.

Here, the above coefficient is calculated after being divided into two tiles, there is a tile boundary between H2 and L3, L1, H1, L2, H2 are the coefficients of the left tile, L3, H3 are Suppose that it is the coefficient of the right tile. H2 is a high-pass coefficient (high-frequency coefficient) that is affected by mirroring during positive conversion (a 3-tap high-pass filter) and is to be corrected. Tiling is a process that performs wavelet transform using only the pixels in each tile, or a process that performs inverse wavelet transform using only the coefficients in each tile.
L1 H1 L2 H2
Is supplemented to the right
L1 H1 L2 H2 L2 H1
It becomes.

Therefore, the value obtained by applying an inverse high-pass filter around the H2 position is
-0.125H1 + 0.5L2 + 0.75H2 + 0.5L2-0.125H1 = -0.25H1 + L2 + 0.75H2 (ii)
It becomes.

In order to aim (i) = (ii)
H2 = 1 / 3H1-2-2L2 + 2 / 3L3 (iii)
Get.

  This is a “correction formula” for a high-pass coefficient that is adjacent to the tile boundary and affected by mirroring during forward conversion. However, since only the tiled coefficients exist when calculating the correction value by this correction formula, L2 uses the left tile coefficient and L3 uses the right tile coefficient (approximation 2).

Note that the same equation can be used for the low-pass coefficient (L3) adjacent to the tile boundary.
H3 = 0 (iv)
Can be obtained. However, in the case of decomposition level 1, there is no need to correct H3 because it does not include mirroring errors. (If decomposition level is 2 or higher, L3 itself adjacent to H3 is affected by mirroring at decomposition level 1.) To compensate for that)

  The correction works very well and in most cases the tile boundary distortion is removed. However, there is a problem that the approximation 2 does not hold in a certain “specific condition or case”, and a side effect occurs due to correction of the high-pass coefficient (high-frequency coefficient) by the correction equation. Such side effects and the specific conditions or cases in which they occur will be described later.

  The present invention aims to prevent tile boundary distortion more reliably by suppressing such side effects.

In order to achieve the above object, the invention of claim 1 divides an image into tiles that do not overlap and performs frequency conversion on each tile, and then performs inverse frequency conversion after quantization and inverse quantization. An image processing apparatus for removing tile boundary distortion of an image obtained by:
Condition determining means for performing a condition determination on a coefficient near the tile boundary;
Coefficient correction means for performing correction processing of coefficients for removing tile boundary distortion according to the determination result by the condition determination means, and the low frequency coefficient when tiling is performed by the condition determination means is tiling. When the condition that the approximation that is equal to the low frequency coefficient is not satisfied (hereinafter, referred to as a specific condition) is not determined, the coefficient correction unit performs the specific high frequency near the tile boundary that is affected by mirroring during frequency conversion. When a specific condition is determined by the condition determination unit, the coefficient correction unit performs a correction process on a specific low-frequency coefficient near the tile boundary that is affected by mirroring during frequency conversion. Next, correction processing is performed for a specific high-frequency coefficient near the tile boundary that is affected by mirroring during frequency conversion. An image processing apparatus according to claim.

  According to a second aspect of the present invention, in the image processing apparatus according to the first aspect of the present invention, in the correction process for the specific low frequency coefficient by the coefficient correction means, the value of the specific low frequency coefficient is the specific low frequency coefficient. And the value of the low frequency coefficient in the adjacent tile located closest to the specific low frequency coefficient.

According to a third aspect of the present invention, in the image processing apparatus according to the first aspect of the invention, the frequency conversion is a 5 × 3 wavelet transform, and the correction sequence for the specific low frequency coefficient by the coefficient correction means is an interleaved coefficient sequence. H1 L2 H2 L3 H3 L4 (where H1, H2, and H3 are high frequency coefficients, L2, L3, and L4 are low frequency coefficients, and the tile boundary is located between H2 and L3) The image processing apparatus is characterized in that the value is corrected to a value of (L2 + L3) / 2.

A fourth aspect of the present invention, in the image processing apparatus of the invention of claim 1, correction processing is the specific for said specific low-frequency coefficients by the factor correcting means when said specific condition is judged by said condition determining means The image processing apparatus is integrated with a correction process for the high frequency coefficient.

According to a fifth aspect of the present invention, in the image processing apparatus according to the fourth aspect of the present invention, the frequency conversion is a 5 × 3 wavelet transform, and the specific low frequency coefficient correction processing by the coefficient correction unit is integrated. In the correction processing for high frequency coefficients, interleaved coefficient sequences H1 L2 H2 L3 H3 L4 (where H1, H2, H3 are high frequency coefficients, L2, L3, L4 are low frequency coefficients, and tile boundaries are H2 and L3 The value of the high frequency coefficient H2 in the
H2 = 1/3 ・ H1−1 / 3 ・ L2 + 1/3 ・ L3
The image processing apparatus is characterized by being corrected by the above.

According to a sixth aspect of the present invention, in the image processing apparatus according to the fourth aspect of the present invention, the frequency conversion is a 5 × 3 wavelet transform, and the specific low frequency coefficient correction processing by the coefficient correction means is integrated. In the correction processing for high frequency coefficients, interleaved coefficient sequences H1 L2 H2 L3 H3 L4 (where H1, H2, H3 are high frequency coefficients, L2, L3, L4 are low frequency coefficients, and tile boundaries are H2 and L3 The value of the high frequency coefficient H3 in the following equation is H3 = L3-L2
The image processing apparatus is characterized by being corrected by the above.

According to a seventh aspect of the present invention, in the image processing apparatus according to the fourth aspect of the present invention, the frequency conversion is a 9 × 7 wavelet transform, and the specific low frequency coefficient correction processing by the coefficient correction means is integrated. In the correction process for high frequency coefficients, interleaved coefficient sequences H0 L1 H1 L2 H2 L3 H3 L4 H4 (where H0, H1, H2, H3, and H4 are high frequency coefficients, and L1, L2, L3, and L4 are low frequency coefficients. And the tile boundary is located between H2 and L3).
H2 = (-0.0535H0 + 0.15644H1 + 0.09127L1-0.295635L2 + 0.295635L3-0.09127L4) /0.60295
The image processing apparatus is characterized by being corrected by the above.

According to an eighth aspect of the present invention, in the image processing apparatus according to the fourth aspect of the present invention, the frequency conversion is a 9 × 7 wavelet transform, and the specific low frequency coefficient correction processing by the coefficient correction unit is integrated. In the correction process for high frequency coefficients, interleaved coefficient sequences H0 L1 H1 L2 H2 L3 H3 L4 H4 (where H0, H1, H2, H3, and H4 are high frequency coefficients, and L1, L2, L3, and L4 are low frequency coefficients. And the tile boundary is located between H2 and L3).
H3 = -0.5L2 + 0.557545L3-0.05754L4 + 0.03372H4
The image processing apparatus is characterized by being corrected by the above.

According to a ninth aspect of the present invention, in the image processing apparatus according to the first or fourth aspect of the invention, the condition determining means detects an edge component parallel to the tile boundary existing in the vicinity of the tile boundary, and the edge component is predetermined. The image processing apparatus is characterized in that the specific condition is determined when the value is exceeded.

A tenth aspect of the present invention is the image processing apparatus according to the ninth aspect of the present invention, wherein the condition determining means excludes edge components on tile boundaries from detection targets.

According to an eleventh aspect of the present invention, in the image processing apparatus according to the ninth aspect of the invention, the frequency conversion is a wavelet transform, and the condition determination means is less than half the tap length of the wavelet transform low-pass filter from the pixel in contact with the tile boundary. The image processing apparatus is characterized in that an edge component within a distance of is set as a detection target.

A twelfth aspect of the present invention is the image processing apparatus according to the ninth aspect of the present invention, wherein the condition determining means detects an edge component using a low frequency coefficient.

  A thirteenth aspect of the present invention is the image processing apparatus according to the twelfth aspect of the present invention, wherein the condition judging means detects an edge component by taking a difference between adjacent low frequency coefficients. is there.

According to a fourteenth aspect of the present invention, in the image processing apparatus according to the twelfth aspect of the present invention, the frequency conversion is a 5 × 3 wavelet transform, and the condition determining means is located in the right tile located at a distance of 0 and a distance of 2 from the tile boundary. The image processing apparatus is characterized in that the edge component is detected by taking the difference of the low frequency coefficients.

According to a fifteenth aspect of the present invention, in the image processing apparatus according to the twelfth aspect of the present invention, the frequency conversion is a 5 × 3 wavelet transform, and the condition determining means is a low frequency in the right tile located at a distance of 0 from the tile boundary. An image processing apparatus that detects an edge edge component by calculating a difference between a coefficient and a low frequency coefficient in a left tile located at a distance of 1 from a tile boundary.

The invention of claim 16 is the image processing apparatus of the invention of claim 14, wherein the predetermined values for the edge component is an image processing apparatus characterized by being selected on the basis of the 24.

According to a seventeenth aspect of the present invention, in the image processing apparatus according to the fifteenth aspect , the predetermined value related to the edge component is selected based on 6.

The invention of claim 18 divides an image into tiles that do not overlap, and frequency coefficients obtained by frequency conversion in units of tiles are quantized and inverse-quantized, and then image tiles obtained by inverse frequency conversion. An image processing method for removing boundary distortion, comprising processing steps corresponding to the functions of the condition determination means and the coefficient correction means in the image processing apparatus according to any one of claims 1 to 17. This is a featured image processing method.

A nineteenth aspect of the present invention is a program that causes a computer to function as the condition determination unit and the coefficient correction unit in the image processing apparatus according to any one of the first to seventeenth aspects.

A twentieth aspect of the invention is a computer-readable information recording medium on which the program of the nineteenth aspect of the invention is recorded.

The inventions of claims 1 to 17 described above will be described. In the following description, an edge component may be simply called an edge.

  First, the side effect in the tile boundary distortion removal method described in Patent Document 4 and a specific condition or case in which the side effect occurs are analyzed.

  8 and 9 are representative examples of the above-mentioned “specific conditions or cases” in the 5 × 3 conversion.

  The example of FIG. 8 is a case where there is an edge component parallel to the boundary between a pixel whose distance from the boundary is 0 and a pixel whose distance is 1 of the tile located to the right or below the tile boundary (FIG. 8). The horizontal axis in FIG. 6 is absolute coordinates, and the pixels of absolute coordinates 5 and 6 are pixels whose distance from the tile boundary is 0. The same applies hereinafter. The edge itself may have various directions with respect to the boundary. However, as described above, the wavelet transform applies a linear filter in one direction (a filter matrix is 1 × n). Whether there is an edge component with respect to the direction is a problem. Here, the “distance from the tile boundary” is as shown in FIG.

  Similarly, the example of FIG. 9 is a case where there is an edge component between a pixel whose distance from the boundary is 1 and a pixel whose distance is 2 of the tile located to the right or below the boundary.

  In FIGS. 8 and 9, for the sake of simplicity, the smaller pixel value is set to 0. However, since the wavelet transform is linear, other values may be simply translated. .

  In the case of FIG. 8 and FIG. 9, assuming that wavelet transformation is applied only in one direction, 1L coefficient is not quantized, and 1H coefficient is quantized to 0, and correction according to the equation (iii) is performed, The results are as shown in FIGS. 11 and 12, respectively.

  An ideal example of the result of detiling (tile boundary distortion removal processing) is equal to the case where tile division is not performed (that is, in the case of one tile), so FIGS. 11 and 12 show the same edge in one tile. A decoded pixel value when processed (when tile division is not performed), a decoded pixel value when tile divided (when two tiles are used), and a decoded pixel value after detiling when two tiles are illustrated are shown.

  As is apparent from FIGS. 11 and 12, when detiling is performed, “undulation” (up and down movement of the pixel value) that does not exist (that is, in the case of one tile) is near the pixel position 4 of the left tile ( This occurs in the part surrounded by a broken line). These “swells” are the above-mentioned side effects, and look like pseudo edges with respect to the original edges, which is extremely undesirable in terms of image quality.

  FIGS. 13 and 14 are diagrams for confirming the cause of the side effect in the cases of FIGS. 11 and 12, respectively. The filter (Low or High) to be applied according to the odd / even pixel position, and the pixel values of the left and right tiles. {Coefficient value after forward wavelet transform, value after quantizing the coefficient and inverse quantization, pixel value after inverse wavelet transform} in the case of 1 tile (when tiling is not performed) {Coefficient value after forward wavelet transform, value after quantization and inverse quantization of the coefficient, pixel value after inverse wavelet transform}, difference between forward wavelet transform coefficients for 1 tile and 2 tiles, detile (tile {Wavelet coefficient value after high-pass coefficient correction, pixel value after inverse wavelet transform} for boundary distortion removal processing) are shown in order from top to bottom. In the figure, “◯” in the third row from the bottom indicates that there is no difference in the positive wavelet transform coefficient in the case of 1 tile and 2 tiles, and “×” indicates that the difference in the case of 1 tile and 2 tiles is large. Is shown. In either case, the low-pass coefficient L3 is “x”, and it can be seen that the approximation 2 is not established.

  The occurrence of the “swell” in the case of FIGS. 11 and 12 can be easily derived from the equations (1) to (4) and (iii). It is clear that the same “swell” occurs when the wavelet transform is performed in two dimensions.

  Now, when the edges in FIGS. 8 and 9 are generalized to step edges having a height s composed of a pixel value 0 and a pixel value s, FIGS. 13 and 14 can be expressed as FIGS. 15 and 16. Here, for the sake of simplicity, the approximate calculation is performed by omitting the floor function in the equations (1) to (4). As is clear from the comparison of the thick frames in both figures, the difference in the positive conversion coefficient between 1 tile and 2 tiles can be formulated as s / 8 or -s / 8, and the value is 2 tiles. L3 in the case is twice L3 in the case of one tile.

When this is further generalized and the same calculation is performed on the step edge composed of the pixel value b and the pixel value (b + s), L2, L3, and L3 are expressed as shown in FIGS. In either case,
Positive conversion coefficient L3 = 1 tile
{(2-tile positive conversion coefficient L2) + (2-tile positive conversion coefficient L3)} / 2 (vii)
(This is not limited to the step edge, and it can be easily confirmed that an impulse having a height s exists at the position of the step edge in FIGS. 17 and 18).

  Therefore, if the L3 in the case of 2 tiles is corrected to the value of L3 in the case of 1 tile by the above formula (vii) under the condition that the above-mentioned problem occurs, the error of the approximation 2 is eliminated. That is, since approximation 2 which is a premise of the equation (iii) is established, the above-described problem does not occur even if the high frequency coefficient H2 is corrected by the equation (iii) thereafter. Further, the correction of the L3 coefficient eliminates the influence of the approximation error of L3 even in the correction using the equation (iv) for the coefficient of the right tile.

  The invention of claim 1 is based on the analysis result, and performs an approximation on the approximation 2 by correcting a specific low frequency coefficient near the tile boundary before correcting the specific high frequency coefficient near the tile boundary. It is intended to eliminate the influence of errors.

  However, the equation (vii) is established on the premise of complete step edges and impulses as is apparent from the derivation process. In the case of a rounded edge or the like, the value of the coefficient L3 in the case of one tile is not exactly 1/2 of {(2-tile positive conversion coefficient L2) + (2-tile positive conversion coefficient L3)}. A value between the value of the two-tile positive conversion coefficient L2 and the value of the two-tile positive conversion coefficient L3 is taken.

According to the second and third aspects of the invention, an appropriate correction is made for a specific frequency coefficient based on the analysis.

  When mirroring is performed, the equation (vii) is established for an arbitrary wavelet transform with an odd number of taps. This can be easily confirmed by following the same derivation process.

In the first to third aspects of the invention, under specific conditions, correction for a specific low frequency coefficient is performed, and then correction for a specific high frequency coefficient is performed. It is possible to integrate the correction for the latter specific high frequency coefficient.

  That is, the formula (iii), which is a correction formula for the high frequency coefficient of the left tile, is shown again.

H2 = 1/3 · H1-2/3 · L2 + 2/3 · L3
In this correction formula, L2 uses the coefficient of the left tile, and L3 uses the coefficient of the right tile. However, as described above, the fact that the approximation for L3 is not established is the cause of the above-mentioned problem. Therefore, the following formula is obtained by replacing L3 in formula (iii) with formula (vii):
H2 = 1/3 · H1-1/3 · L2 + 1/3 · L3 Equation (viii)
Can be used to prevent the inconvenience due to the approximation error of L3.

  Also, the formula (iv), which is a correction formula for the right tile coefficient H3, is shown again.

H3 = 0
By replacing L3 in the case of one tile in the process of making this correction formula with the above formula (vii)
H3 = L3-L2 Formula (ix)
Can be obtained. If this correction equation is used, inconvenience due to the approximation error of L3 can be prevented.

  That is, the equations (viii) and (ix) are correction equations for correction for H2 and H3, which are integrated corrections for L3 under specific conditions.

Based on this consideration, the inventions of claims 4 , 5 and 6 eliminate the effect of approximation error by integrating correction for a specific low frequency coefficient under specific conditions with correction for a specific high frequency. It is something to try.

Up to this point, the case of the 5 × 3 wavelet transform has been described, but the correction formula can be derived based on the same concept for the case of the 9 × 7 wavelet transform. The 9 × 7 wavelet transform is given by the following equation.
<Positive conversion>
C (2n + 1) = P (2n + 1) + α * (P (2n) + P (2n + 2)) [step1]
C (2n) = P (2n) + β * (C (2n-1) + C (2n + 1)) [step2]
C (2n + 1) = C (2n + 1) + γ * (C (2n) + C (2n + 2)) [step3]
C (2n) = C (2n) + δ * (C (2n-1) + C (2n + 1)) [step4]
C (2n + 1) = K * C (2n + 1)) [step5]
C (2n) = (1 / K) * C (2n) [step6]
<Inverse transformation>
P (2n) = K * C (2n) [step1]
P (2n + 1) = (1 / K) * C (2n + 1) [step2]
P (2n) = P (2n) -δ * (P (2n-1) + P (2n + 1)) [step3]
P (2n + 1) = P (2n + 1) -γ * (P (2n) + P (2n + 2)) [step4]
P (2n) = P (2n) -β * (P (2n-1) + P (2n + 2)) [step5]
P (2n) = P (2n + 1) -α * (P (2n) + P (2n + 2)) [step6]
α = -1.586134342059924
β = -0.052980118572961
γ = 0.882911075530934
δ = 0.443506852043971
K = 1.230174104914001

In such a 9 × 7 wavelet transform, an interleaved coefficient sequence
In H0 L1 H1 L2 H2 L3 H3 L4 H4 (there is a tile boundary between H2 and L3), the correction formula for H2 and H3 under the condition that approximation error does not matter
H2 = (-0.0535H0 + 0.15644H1 + 0.09127L1-0.59127L2 + 0.59127L3-0.09127L4) /0.60295
... Formula (v)
H3 = (0.05754L2-0.05754L4 + 0.03372H4) /0.53372 ・ ・ ・ Formula (vi)
It is.

Under the condition that approximation error is a problem, the correction formulas for the correction of H2 and H3 that integrate the correction of the low frequency coefficient are the above formulas (v), (vi), and (vii).
H2 = (-0.0535H0 + 0.15644H1 + 0.09127L1-0.295635L2 + 0.295635L3-0.09127L4) /0.60295
... Formula (x)
H3 = -0.5L2 + 0.557545L3-0.05754L4 + 0.03372H4 ・ ・ ・ Formula (xi)
It becomes.

The inventions of claims 7 and 8 are intended to remove the influence of the approximation error in detiling when using the 9 × 7 wavelet transform by using such a correction formula.

As described above, the defect due to the approximation error is caused by an edge component parallel to the tile boundary existing near the tile boundary. The invention of claim 9 is intended to determine a condition for causing a failure by paying attention to such an edge component.

  FIG. 19 shows the processing result when there is an edge component parallel to the boundary on the tile boundary, and FIG. 20 shows the pixels whose distance from the boundary is 2 and 3 for the tile located to the right or below the boundary. The processing results when there is an edge between are shown.

  In the case of FIG. 19, there is almost no side effect due to the edge (detiling works well), and the value when not detiling is significantly different from the value of 1 tile, so it can be seen that the detiled value should be adopted. . FIG. 21 and FIG. 22 are diagrams for clarifying the reason why detiling works well in this case. From both figures, it can be seen that approximation 2 does not hold for the two low frequency coefficients L2 and L3, but the approximation error occurs with the same polarity. Therefore, these approximation errors are canceled out by the subtracting unit of the above formula (iii), so that no side effect occurs. It can be easily confirmed that this is true not only at the step edge but also when an impulse having a height s exists at the position of the step edge in FIGS.

  Also in the case of FIG. 20, the side effects do not occur. FIG. 23 is a diagram for clarifying the reason. The reason why Approximation 2 is not satisfied is that the value of L3 of the right tile when mirroring is greatly different from the value of L3 when there is one tile due to the presence of an edge in the right tile. For this reason, whether or not Approximation 2 is established is determined by whether or not there is an edge in the pixel referred to when calculating the L3 coefficient. As apparent from FIG. 23, in the case of FIG. 20, there is no edge in the pixel referred to when calculating the L3 coefficient. In this way, there is a limit to the range of edges that determine the quality of detiling.

  The same argument holds even when there is an edge only in the left tile, and the establishment / non-establishment of approximation 2 is determined by whether there is an edge in the pixel referenced when calculating the L2 coefficient. 24, 25, and 26 show the processing results when the left tile has an edge. In the case of FIG. 24, there are few side effects like the case of FIG. In the case of FIG. 26, there is no side effect due to L2 as in the case of FIG. In the case of FIG. 25, the side effect is small because of the difference that the “swell” that is originally in the case of one tile is eliminated.

The inventions of claims 10 and 11 are intended to efficiently perform edge detection for condition determination based on the above examination.

The reason why low frequency coefficients are used for edge detection as described above is that , generally, high frequency coefficients are subjected to large quantization and have large errors (if tile boundaries are a problem, they may be quantized to zero). This is because the detection accuracy of the edge tends to be deteriorated when a high frequency coefficient is used.

The inventions of claims 12 and 13 attempt to perform edge detection with high accuracy by using a low frequency coefficient for edge detection based on such consideration.

  In the case of 5 × 3 wavelet transform, it is easy to use the difference (L4−L3) between L4 and L3 for detection and determination of the edge component of the right tile. In the case of FIG. 17, (L4−L3) = 7/8 · s, and in the case of FIG. 18, (L4−L3) = 9/8 · s, both values being close to s. In the case of FIG. 18 as well, the same threshold value can be used for determining whether or not the size of the edge component exceeds a predetermined value, and the threshold value may be set based on the value of s, for example. . Further, in the case of FIGS. 19 and 20 in which detileing is excluded, (L4−L3) = 0, which is also convenient.

The invention of claim 14 is based on such a consideration, it is an attempt to edge detection easily.

As described above, the difference (L4-L3) is close to s only when the edge is a step edge. When an impulse having a height s exists at the position of the step edge in FIGS. 8 and 9, the difference (L4−L3) in each case does not become a value close to s . Therefore, when it is desired to accurately detect the impulse, a more precise detection formula or the like should be used.

  However, the edge detection is not limited to the method using the difference (L4−L3), and may be a method using more differences of low frequency coefficients. Further, the difference between L3 and L2 (L3-L2) may be used. The difference (L3−L2) is s / 4 in the case of FIG. 17 and −s / 4 in the case of FIG. 18, which is advantageous in that the absolute values thereof are the same.

The invention of claim 15 is intended to easily perform edge detection based on such consideration.

  However, in the case of FIG. 21 and FIG. 22, since the difference (L3-L2) = s, it is necessary to devise how to determine the threshold value for determination.

  Instead of detecting the edge component from the decoded coefficient value as described above, the edge component is detected and the value is determined at the time of encoding, and the result is separated from the code or the comment part in the code, etc. It is also possible to control the coefficient correction by referring to the edge detection / determination result held at the time of decoding.

  The above-mentioned “swell” occurs at the position L2 in FIGS. 15 and 16, and the magnitude is a function of the s. As shown in FIGS. 15 and 16, L2 = −s / 24 or L2 = s / 24, and if s is less than 24, L2 after inverse transformation is rounded to 0, so that “undulation” does not occur. Since the absolute value of the difference (L4-L3) is almost equal to s as described above, if (L4-L3) = s, if (L4-L3) exceeds 24, whether or not (L4-L3) exceeds 24 It can be determined whether or not “swell” occurs in the position. 15 and 16 use approximate expressions (1) to (4), and the correction expression is also rounded down to an integer (the coefficient takes only an integer value in the 5 × 3 conversion). Therefore, the value of 24 can vary depending on the approximation and rounding method.

  15 and 16, since the difference (L3-L2) is approximately s / 4, whether the difference exceeds 6 (equivalent to whether s exceeds 24) is at the position of L2. It can be determined whether or not “swell” occurs. However, this value of 6 can also be varied for the same reason as described for the difference (L4-L3).

  The above values 24 and 6 are both for the coefficient of decomposition 1. In the case of wavelet transforms that are not non-normal, the coefficient value increases as the decomposition level increases, so the difference (L4-L3) and (L3-L2) values also tend to increase. It also depends on the type of conversion (see FIG. 42). Further, Y, Cb. The coefficient value of each component varies depending on the conversion formula used for component conversion to Cr (see FIG. 43). Therefore, it is necessary to adjust the threshold for edge detection determination using the differences (L4-L3) and (L3-L2) according to the type of wavelet transform, the decomposition level, and the component. FIG. 42 shows the coefficient absolute value increase rate by 5 × 3 conversion and 9 × 7 conversion estimated based on JPEG2000 Standardization Committee document WG1N2133 “Guard Bit Requirements for JPEG2000 Part1 Filters, Junichi Hara”. FIG. 43 shows an increase estimation of the component absolute value for RCT and ICT which are component conversions defined by JPEG2000.

  As described above, the threshold for edge detection determination needs to be appropriately changed. However, when the difference (L4-L3) or (L3-L2) is used for edge detection by 5 × 3 wavelet transform, the above-described 24 Alternatively, a value of 6 can be a reference or a guideline for selecting a threshold for edge detection determination.

Accordingly, the invention of claim 16 attempts to select a predetermined value (determination threshold) relating to the edge component with reference to 24, and the invention of claim 17 sets the predetermined value (determination threshold) relating to the edge component to 6 as a reference. It is what you want to choose.

  Note that when the magnitude of the “swell” when the impulse of height s is present at the position of the step edge in FIGS. 8 and 9 is calculated, −s / 12 and s / 8 are obtained in order. Therefore, if it is desired to control the swell in the case of an impulse more accurately, it is desirable to use a determination threshold value smaller than 24 or 6.

This is described in claims 1 to 17 . Since the inventions of claims 18 , 19 and 20 correspond to the inventions of claims 1 to 17 , the description thereof will not be repeated.

As described above, according to the inventions of claims 1 to 18 , it is possible to effectively remove tile boundary distortion, which is a problem in dividing an image into tiles that do not overlap and performing frequency conversion in units of tiles. Furthermore, the occurrence of side effects of the detileing process can be suppressed. According to the present invention乃optimum 3, the correction for a particular low-frequency coefficients, the influence of approximation error associated with the high frequency coefficients in the tile both adjacent horizontally or vertically (the cause of side effects) simultaneously removed be able to. According to the fourth to eighth aspects of the present invention, the correction processing for the high frequency coefficients near the tile boundary for detiling is integrated with the correction processing for the low frequency coefficients near the tile boundary for suppressing the side effects. Compared to the case where the processing is executed sequentially, the number of processing steps is reduced, which is advantageous in simplifying and speeding up the processing. According to the invention of claims 9 to 17, can be easily carried out accurately by determining the edge detection in the vicinity of the tile boundary conditions side effects Detairu processing occurs. According to the eighteenth to twentieth inventions, it is possible to easily implement the inventions according to the first to seventeenth aspects by using a computer.

  One preferred embodiment of the present invention is an image processing apparatus that performs both image compression processing and decompression processing or decompression processing using an algorithm of JPEG2000.

  FIG. 27 is a block diagram showing an example of such an image processing apparatus. The image processing apparatus shown in FIG. 27 can perform both compression processing and decompression processing according to JPEG2000. Its basic configuration is the same as that shown in FIG. 1, and the same reference numerals as those in FIG. 1 denote corresponding blocks.

  A characteristic configuration of this image processing apparatus is that a coefficient correction unit 111 that is a coefficient correction unit for detiling is provided in the two-dimensional wavelet transform / inverse transform unit 101. The coefficient correction unit 111 includes an edge detection unit 112 that performs edge detection near the tile boundary as means for determining a condition related to a coefficient near the tile boundary when the tile is divided and wavelet transformed.

  In the case of compression processing, the code stream generated by the code formation / tag processing unit 104 is output to the outside or stored in the storage unit 110.

  In the case of decompression processing, a JPEG 2000 code stream is input from the outside to the code formation / tag processing unit 104, or a JPEG 2000 code stream read from the storage unit 110 is input. In this decompression processing, the deleting process is performed on the wavelet coefficients after dequantization by the quantization / inverse quantization unit 102 or the wavelet coefficients subjected to inverse wavelet transform by the wavelet transform / inverse transform unit 101. The At the time of detiling, the edge detection unit 112 detects a specific edge, and the coefficient correction content by the coefficient correction unit 111 is controlled based on the detection result (condition determination result).

  As described above, edge detection can be performed during the compression process, and the result can be used by the coefficient correction unit 111 during the decompression process. In this case, it is necessary to give the edge detection result to the coefficient correction unit 111. As the method, for example, there is a method of describing the edge detection result in the code stream. However, the edge detection result can be input as data independent of the code stream.

  Means or processing relating to detiling in the image processing apparatus shown in FIG. 27 can also be realized by a program using a computer. An example of such an embodiment will be briefly described with reference to FIG. In FIG. 28, reference numeral 200 denotes a CPU, 201 denotes a RAM, 202 denotes a hard disk device, and 203 denotes a display device, which are connected to a bus 204.

  The outline of the processing flow is as follows. First, a code stream (compressed image data) recorded in the hard disk device 202 is read into the RAM 201 by an instruction from the CPU 200 (1). The CPU 200 reads the code stream from the RAM 201 and performs decompression processing, and performs processing for detiling in the process (2). Then, the CPU 200 writes the decompressed image data in another area on the RAM 201 (3). This image data is displayed on the display device 203 by a command from the CPU 201, for example (4).

  Hereinafter, the detiling process in the image processing apparatus of the present invention will be specifically described in order.

  FIG. 29 shows an array of coefficients for four tiles after inverse quantization, and coefficients 2LL to 2HH at the decomposition level 2 are interleaved. Here, it is assumed that the number of pixels of each tile is an even number both vertically and horizontally. Therefore, the right end and the lower end of the tile are the positions of the high frequency coefficient (high pass coefficient), and the left end and the upper end of the tile are the positions of the low frequency coefficient (low pass coefficient).

  Of these coefficients at decomposition level 2, the coefficients in column A and column B shown in FIG. 30 are the high-frequency coefficients of the correction elephant. After correction, horizontal wavelet transform is performed (in JPEG2000, wavelet transform (forward transform) is performed in the order of vertical → horizontal, so reverse wavelet transform is performed in the order of horizontal → vertical). Become.

  Next, the coefficients in rows C and D shown in FIG. 32 are the high frequency coefficients to be corrected. After the correction, the inverse wavelet transform in the vertical direction is performed, and the coefficient (deinterleaved state) of decomposition level 1 as shown in FIG. 33 is obtained.

  Subsequently, these coefficients are interleaved as shown in FIG. Of these coefficients at decomposition level 1, in the case of 5 × 3 conversion, only the coefficients in column E shown in FIG. 35 are the high frequency coefficients to be corrected (in decomposition level 1, the coefficients in column F are mirrored). This is because the column F does not need to be corrected since the above error is not included.) On the other hand, in the case of 9 × 7 conversion, the number of examples in column E and the coefficient in column F are the high frequency coefficients to be corrected. After the coefficient correction, the inverse wavelet transform in the horizontal direction is performed and the state shown in FIG. 36 is obtained.

  In the case of 5 × 3 conversion, only the coefficient in the row G in FIG. 36 is the high frequency coefficient to be corrected. In the case of 9 × 7 conversion, the coefficients of row G and row H are the high frequency coefficients to be corrected. After the correction, the inverse wavelet transform in the vertical direction is performed, and the state before the wavelet transform (component value such as YCbCr when the component transform is performed) is returned.

  In the above, since it was assumed that correction for a specific low frequency coefficient was integrated with correction for a specific high frequency, the low frequency coefficient to be corrected was not mentioned. The correction of the low frequency coefficient will be described in the following embodiment. Further, the above is an example of 4 tiles and 2 decomposition levels, but the case of an arbitrary number of tiles and an arbitrary number of decomposition levels may be considered similarly.

  FIG. 37 shows a basic processing flow of detiling processing in the image processing apparatus of the present invention. Here, it is assumed that a compressed code stream is processed by dividing an image into tiles of the same size with an even number of pixels both vertically and horizontally. In this case, the right end and the lower end of the tile are positions of a high frequency coefficient (high pass coefficient), and the left end and the upper end of the tile are positions of a low frequency coefficient (low pass coefficient). The number of decomposition levels of the wavelet transform during compression is 3.

  Hereinafter, the processing content will be described with reference to FIG. Prior to this processing, for all four tiles, all the composition level coefficients of all the components are decoded by the entropy encoding / decoding unit 103, and the quantization / inverse quantization unit 102 performs inverse quantization. Is done.

  First, the decomposition level is set to 3 and the component number is set to 0 (step S1), respectively, and the detiling process for each component level is started.

  First, with respect to the component of component number 0 (for example, the luminance component), the horizontal direction detiling process and the vertical direction detiling process are sequentially performed on the wavelet coefficient at the decomposition level 3 (coefficient immediately after inverse quantization) (step S2, S2). S3). These detiling processes include an edge detection process, and also include inverse wavelet transforms in the horizontal direction and the vertical direction. That is, the main component of the detiling process is the coefficient correction unit 111, but the two-dimensional wavelet transform / inverse transform unit 101 is also involved. The high frequency coefficients to be corrected are the coefficients in columns A and B shown in FIG. 30 or the coefficients in rows C and D shown in FIG.

  When the detiling process for the coefficient at the decomposition level 3 is completed, the decomposition level is decremented to 2 (step S5), and the detiling process at steps S2 and S3 is executed for the coefficient at the decomposition level 2.

  When the detiling process of the coefficient at the decomposition level 2 is completed, the decomposition level is decremented (step S5), and the horizontal detileing process and the vertical detiling process are sequentially performed on the coefficient at the decomposition level 1 (steps S2 and S3). ). The high frequency coefficients to be corrected are the coefficients in columns E and F shown in FIG. 35 in the horizontal direction and in rows G and H shown in FIG. 36 in the vertical direction.

  When the processing for the composition 1 is completed (step S4), the component number is incremented to 1, and the decomposition level is returned to 3 (step S7), and the wavelet coefficient of the component of the component number 1 (for example, the color difference Cb component) On the other hand, the process after step S2 is performed.

  Subsequently, the process for the wavelet coefficient of the component of component number 2 (for example, the color difference Cr component) is executed. When the process is completed (No in step S6), the detiling process is completed.

  Hereinafter, steps S2 and S3 will be described in detail.

  In the first embodiment, horizontal detiling processing as shown in FIG. 38 is executed in step S2, and vertical detiling processing as shown in FIG. 39 is executed in step S3.

  With reference to FIG. 38, the horizontal detiling process in step S2 will be described. In the present embodiment, edge detection is performed based on the difference (L3-L2). Further, a case where 5 × 3 wavelet transform is used is assumed.

  First, in step S11, the edge detection unit 112 selects a type of wavelet transform to be applied, a decomposition level to be processed, and a threshold for edge detection determination suitable for the component.

  In the next step S12, the edge detection unit 112 relates coefficient values in the columns A and B in FIG. 30 to the coefficient values at the L3 position (the left adjacent position in the column B) and the L2 position (the left adjacent position in the column A). The difference (L3-L2) is compared with the threshold value. When it is determined that the difference value is not greater than the threshold value, that is, when an edge is not detected, the coefficient correction unit 111 performs the above equation (iii) for the high-frequency coefficient (H2) of interest in the column A of the left tile in step S14. ) (If the 9 × 7 conversion, the correction is performed using the equation (v)), and in step S15, the high frequency coefficient (H3) of interest in the column B of the right tile is calculated using the equation (iv). Correction is performed (in the case of 9 × 7 conversion, correction is performed by the formula (vi)).

  When the difference (L3-L2) is larger than the threshold value in step S12, that is, when an edge is detected, the above-mentioned side effects are large. In this case, the coefficient correction unit 111 corrects the low frequency coefficient (L3) adjacent to the left side of the coefficient of interest in the column B by using the equation (vii) in step S13. Thereafter, the coefficient correction unit 111 performs correction processing (steps S14 and S15) for the high-frequency coefficient. By correcting the low frequency coefficient L3 in step S13, side effects due to the correction of the high frequency coefficient in steps S14 and S15 are prevented.

  When steps S12 to S16 are repeated and the processing for all the coefficients in columns A and B is completed (step S16, Yes), the two-dimensional wavelet transform / inverse transform unit 101 applies the inverse wavelet in the horizontal direction to the corrected wavelet coefficients. Conversion is performed (step S17).

  In the detiling process for the coefficient at decomposition level 1, only the high frequency coefficient (H2) in column A (column E in FIG. 35) is corrected in step 14, and step S15 is skipped (if it is 9 × 7 conversion). Both steps S14 and S15 are executed, and the high frequency coefficients (H2, H3) of columns A and B (columns E and F in FIG. 35) are corrected).

  Next, with reference to FIG. 39, the vertical detiling process in step S3 will be described. First, in step S21, the edge detection unit 112 selects a type of wavelet transform to be applied, a decomposition level to be processed, and a threshold for edge detection determination suitable for the component.

  In the next step S22, the edge detection unit 112 relates the L3 position (the upper adjacent position of the row D) and the L2 position (the upper adjacent position of the row C) for the noted coefficient in the rows C and D shown in FIG. The numerical difference (L3-L2) is compared with a threshold value. When it is determined that the difference value is not larger than the threshold value, that is, when an edge is not detected, the coefficient correction unit 111 performs the above equation (iii) for the high frequency coefficient (H2) of interest in row C of the upper tile in step S24. ) (If the 9 × 7 conversion, the correction is performed using the equation (v)), and in step S25, the equation (iv) is applied to the high frequency coefficient (H3) of interest in the row D of the lower tile. (If 9 × 7 conversion, correction is performed using equation (vi)).

  When it is determined in step S22 that the difference (L3-L2) is greater than the threshold value, that is, when an edge is detected, the aforementioned side effects are large. In this case, the coefficient correction unit 111 corrects the low-frequency coefficient (L3) adjacent to the coefficient of interest in the row D by using the equation (vii) in step S23. Thereafter, the coefficient correction unit 111 performs correction processing (steps S24 and S25) on the high frequency coefficient. By correcting the low frequency coefficient L3 in step S23, side effects due to the correction of the high frequency coefficient in steps S24 and S25 are prevented.

  When steps S22 to S26 are repeated and the processing for all the coefficients in rows C and D is completed (step S26, Yes), the two-dimensional wavelet transform / inverse transform unit 101 performs the inverse wavelet in the vertical direction with respect to the wavelet coefficients after the correction processing. Conversion is performed (step S27).

  In the detiling process for the coefficient at decomposition level 1, only the high frequency coefficient (H2) of row C (row G in FIG. 36) is corrected in step 24, and step S25 is skipped (if 9 × 7 conversion is used). Both steps S24 and S25 are executed, and the high frequency coefficients (H2, H3) of rows C and D (rows G and H in FIG. 36) are corrected).

  In the second embodiment, horizontal detiling processing as shown in FIG. 40 is executed in step S2, and vertical detiling processing as shown in FIG. 41 is executed in step S3.

  First, the horizontal direction detiling process will be described with reference to FIG. In this embodiment, edge detection is performed based on the difference (L4-L3). Assume that a 5 × 3 wavelet transform is used.

  First, in step S31, the edge detection unit 112 selects the type of wavelet transform to be applied, the decomposition level to be processed, and a threshold for edge detection determination suitable for the component.

  In the next step S32, the edge detection unit 112 relates to the noted coefficient in the columns A and B in FIG. 30 and the difference (L4) between the coefficient values of the L3 position and the L4 position (the left adjacent position and the right adjacent position in the column B). -L3) is compared with the threshold. When it is determined that the difference value is not larger than the threshold value, that is, when an edge is not detected, the coefficient correction unit 111 calculates the above formula (iii) for the high frequency coefficient (H2) of interest in the column A of the left tile in step S33. ) (If the 9 × 7 conversion, the correction is performed using the equation (v)), and then the high frequency coefficient (H3) of interest in the column B of the right tile is corrected using the equation (iv) in step S34. (If 9 × 7 conversion is performed, correction is performed according to the equation (vi)).

  When it is determined in step S32 that the difference (L3-L2) is greater than the threshold value, that is, when an edge is detected, the aforementioned side effects are large. In this case, in step S35, the coefficient correction unit 111 corrects the high-frequency coefficient (H2) of interest in column B according to the above equation (viii) (if 9 × 7 conversion, the coefficient correction unit 111 corrects according to the above equation (x). Then, in step S36, the high frequency coefficient (H3) of interest in column B of the right tile is corrected by the equation (ix) (if 9 × 7 conversion, the equation (xi) is corrected). That is, the correction for the high frequency coefficients H2 and H3 in steps S35 and S36 is an integration of the correction for the low frequency coefficient L3. Therefore, the occurrence of side effects due to the correction of the high frequency coefficient in steps S35 and S36 is prevented.

  When steps S32 to S37 are repeated and the processing for all the coefficients of columns A and B is completed (step S37, Yes), the two-dimensional wavelet transform / inverse transform unit 101 applies the inverse wavelet in the horizontal direction to the corrected wavelet coefficients. Conversion is performed (step S38).

  In the detiling process for the coefficient at decomposition level 1, only the high frequency coefficient (H2) in column A (column E in FIG. 35) is corrected in step S33 or S35, and step S34 or S36 is skipped (9 × In the case of 7 conversions, the high frequency coefficients (H2, H3) of columns A and B (columns E and F in FIG. 35) are corrected).

  Next, the vertical direction detiling process in step S3 will be described with reference to FIG. First, in step S41, the edge detecting unit 112 selects a type of wavelet transform to be applied, a decomposition level to be processed, and a threshold for edge detection determination suitable for the component.

  In the next step S42, the edge detection unit 112 relates the L3 position (the upper adjacent position of the row D) and the L4 position (the lower adjacent position of the row D) with respect to the noted coefficient in the rows C and D shown in FIG. The numerical difference (L4-L3) is compared with the threshold value. When it is determined that the difference value is not larger than the threshold value, that is, when an edge is not detected, the coefficient correction unit 111 calculates the above formula (iii) for the high frequency coefficient (H2) of interest in row C of the upper tile in step S43. ) (If the 9 × 7 conversion, the correction is performed using the equation (v)), and in step S44, the equation (iv) is applied to the high frequency coefficient (H3) of interest in the row D of the lower tile. (If 9 × 7 conversion is performed, correction is performed according to the equation (vi)).

  When the difference (L4-L3) is larger than the threshold value in step S42, that is, when an edge is detected, the above-mentioned side effects are large. In this case, in step S45, the coefficient correction unit 111 corrects the high frequency coefficient (H2) of interest in row C of the upper tile according to the above equation (viii) (if 9 × 7 conversion, according to the above equation (x)). Then, in step S46, the high frequency coefficient (H3) of interest in the lower tile row D is corrected by the above equation (ix) (if 9 × 7 conversion, it is corrected by the above equation (xi)). I do). That is, the correction for the high frequency coefficients H2 and H3 in steps S45 and S46 is an integration of the correction for the low frequency coefficient L3. Therefore, the occurrence of side effects due to the correction of the high frequency coefficient in steps S45 and S46 is prevented.

  When steps S42 to S47 are repeated and the processing for all the coefficients in rows C and D is completed (step S47, Yes), the two-dimensional wavelet transform / inverse transform unit 101 performs the inverse wavelet in the vertical direction with respect to the corrected wavelet coefficients. Conversion is performed (step S48).

  In the detiling process for the coefficient of decomposition level 1, only the high frequency coefficient (H2) of row C (row G in FIG. 36) is corrected in step 45, and step S46 is skipped (if 9 × 7 conversion is used). Both steps S45 and S46 are executed, and the high frequency coefficients (H2, H3) of rows C and D (rows G and H in FIG. 36) are corrected).

  Although the present invention has been described in detail above, as described in relation to FIG. 28, the means for detiling processing (coefficient correction unit, edge detecting means) according to the present invention, and the reverse related to the detiling process. The wavelet transforming unit or the like, or the above-described detile processing procedure can be realized by a program using a computer. Such a program (for example, an application program or a device driver such as a printer driver) and various information records (such as a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor storage element) on which the program is recorded ( Storage) media are also encompassed by the present invention.

It is a block diagram for demonstrating JPEG2000. It is a figure which shows the example and coordinate system of an original image. It is a figure which shows the coefficient arrangement | sequence after filtering to a perpendicular direction. It is a figure which shows the coefficient arrangement | sequence after filtering to a horizontal direction. It is a figure which shows the coefficient arrangement | sequence which deinterleaved the coefficient of FIG. It is a figure which shows the de-interleaved coefficient arrangement | sequence after 2 times of wavelet transforms. It is explanatory drawing of mirroring. It is a figure which shows the edge in the right tile which produces a side effect by detiling. It is a figure which shows the edge in the right tile which produces a side effect by detiling. It is explanatory drawing of the distance from a tile boundary. It is a figure which shows the side effect by the edge of FIG. It is a figure which shows the side effect by the edge of FIG. It is a figure for demonstrating the reason which the side effect of FIG. 11 arises. It is a figure for demonstrating the use which the side effect of FIG. 12 produces. FIG. 14 is a diagram similar to FIG. 13 when the edge of FIG. 8 is a step edge composed of pixel values of 0 and s. FIG. 15 is a diagram similar to FIG. 14 when the edge of FIG. 9 is a step edge composed of pixel values of 0 and s. It is a figure which shows the difference of coefficient L2, L3, L4 at the time of making the edge of FIG. 8 into the step edge which consists of the image value of b and b + 1. FIG. 10 is a diagram illustrating values of coefficients L2, L3, and L4 when the edge in FIG. 9 is a step edge including image values of b and b + 1. It is a figure which shows the result of a detile process in case an edge exists on the tile boundary with a small side effect. It is a figure which shows the result of a detile process in case there exists an edge of the right tile without a side effect. It is a figure for demonstrating the reason for a small side effect in the case of FIG. It is a figure which shows the value of coefficient L2, L3, L4 in case the edge on a boundary is a step edge which consists of a pixel value of b and b + 1. It is a figure for demonstrating the reason a side effect does not arise in the case of FIG. It is a figure which shows the result of a detile process in case there exists an edge of the left tile with a small side effect. It is a figure which shows the result of a detile process in case there exists an edge of the left tile with a small side effect. It is a figure which shows the result of a detile process in case there exists an edge of the left tile with a small side effect. It is a block diagram which shows the structural example of the image processing apparatus which concerns on this invention. It is a block diagram for demonstrating the form which implements this invention on a computer. It is a figure which shows the coefficient (interleaved state) of the decomposition level 2 after a dequantization in the case of 4 tile division | segmentation. It is a figure which shows the high frequency coefficient (interleaved state) of the decomposition level 2 used as correction object by the detiling process of a horizontal direction. It is a figure which shows the coefficient (interleaved state) after carrying out the inverse wavelet transform of the coefficient of FIG. 30 to a horizontal direction. It is a figure which shows the high frequency coefficient used as the correction object by the detilering process of the coefficient shown in the vertical direction shown in FIG. It is a figure which shows the coefficient (deinterleaving state) after carrying out the inverse wavelet transform of the coefficient of FIG. 31 to the orthogonal | vertical direction. It is a figure which shows the state which interleaved the coefficient of FIG. It is a figure which shows the high frequency coefficient used as the correction | amendment object of the horizontal direction detileing process of the decomposition level 1. FIG. It is a figure which shows the high frequency coefficient used as the correction | amendment object of the vertical direction detil process of a decomposition level 1. FIG. It is a flowchart which shows the whole processing flow of the detileing process in this invention. 3 is a flowchart of horizontal detiling processing in the first embodiment. 6 is a flowchart of vertical detiling processing in the first embodiment. 12 is a flowchart of horizontal detiling processing in the second embodiment. 10 is a flowchart of vertical detiling processing in the second embodiment. It is a figure which shows the increase estimation of the coefficient absolute value in 5x3 wavelet transformation and 9x7 wavelet transformation. It is a figure which shows the increase estimation of the component absolute value by component conversion.

Explanation of symbols

100 color conversion / inverse color conversion unit 101 two-dimensional wavelet transform / inverse conversion unit 102 quantization / inverse quantization unit 103 entropy encoding / decoding unit 104 code formation / tag processing unit 110 accumulation unit 111 coefficient correction unit 112 edge detection Part

Claims (20)

Image to remove the tile boundary distortion of the image obtained by dividing the image into tiles that do not overlap and frequency-converting the coefficient obtained by tile unit and then performing inverse frequency conversion after quantization and inverse quantization A processing device comprising:
Condition determining means for performing a condition determination on a coefficient near the tile boundary;
Coefficient correction means for performing correction processing of coefficients for removing tile boundary distortion according to the determination result by the condition determination means;
Have
When the condition determining means does not determine that the low frequency coefficient when tiling is equal to the low frequency coefficient when tiling is not satisfied (hereinafter referred to as a specific condition) , the coefficient correction is performed. The means performs a correction process for a specific high-frequency coefficient near the tile boundary that is affected by mirroring during frequency conversion,
When the specific condition is determined by said condition determining means, said coefficient correcting unit performs a correction process for a specific low-frequency coefficients around tile boundaries affected by mirroring the time frequency conversion, and then mirroring the time frequency conversion An image processing apparatus that performs correction processing on a specific high-frequency coefficient in the vicinity of a tile boundary that is affected by the above-described effect. The image processing apparatus according to claim 1.
In the correction process for the specific low-frequency coefficient by the coefficient correction unit, the value of the specific low-frequency coefficient is within the adjacent tile in the position closest to the specific low-frequency coefficient value and the specific low-frequency coefficient. An image processing apparatus that is corrected to a value between the low-frequency coefficient of each of the images. The image processing apparatus according to claim 1.
The frequency transform is a 5x3 wavelet transform,
In the correction process for the specific low frequency coefficient by the coefficient correction means,
In the interleaved coefficient sequence H1 L2 H2 L3 H3 L4 (where H1, H2, H3 are high frequency coefficients, L2, L3, L4 are low frequency coefficients, and the tile boundary is located between H2 and L3) An image processing apparatus, wherein the value of the low frequency coefficient L3 is corrected to a value of (L2 + L3) / 2. The image processing apparatus according to claim 1.
The image processing apparatus characterized by correction processing for said specific low-frequency coefficients by the factor correcting means when said specific condition is judged by the condition judging means is integrated into the correction processing for the specific frequency coefficients . The image processing apparatus according to claim 4 .
The frequency transform is a 5x3 wavelet transform,
In the correction process for the specific high frequency coefficient integrated with the correction process for the specific low frequency coefficient by the coefficient correction unit,
In the interleaved coefficient sequence H1 L2 H2 L3 H3 L4 (where H1, H2, H3 are high frequency coefficients, L2, L3, L4 are low frequency coefficients, and the tile boundary is located between H2 and L3) The value of the high frequency coefficient H2 is
H2 = 1/3 ・ H1−1 / 3 ・ L2 + 1/3 ・ L3
An image processing apparatus corrected by the above. The image processing apparatus according to claim 4 .
The frequency transform is a 5x3 wavelet transform,
In the correction process for the specific high frequency coefficient integrated with the correction process for the specific low frequency coefficient by the coefficient correction unit,
In the interleaved coefficient sequence H1 L2 H2 L3 H3 L4 (where H1, H2, H3 are high frequency coefficients, L2, L3, L4 are low frequency coefficients, and the tile boundary is located between H2 and L3) The value of the high frequency coefficient H3 is
H3 = L3-L2
An image processing apparatus corrected by the above. The image processing apparatus according to claim 4 .
The frequency transform is a 9x7 wavelet transform,
In the correction process for the specific high frequency coefficient integrated with the correction process for the specific low frequency coefficient by the coefficient correction unit,
Interleaved coefficient sequence H0 L1 H1 L2 H2 L3 H3 L4 H4 (where H0, H1, H2, H3, H4 are high frequency coefficients, L1, L2, L3, L4 are low frequency coefficients, and tile boundaries are H2 And located between L3)
The value of the high frequency coefficient H2 at
H2 = (-0.0535H0 + 0.15644H1 + 0.09127L1-0.295635L2 + 0.295635L3-0.09127L4) /0.60295
An image processing apparatus corrected by the above. The image processing apparatus according to claim 4 .
The frequency transform is a 9x7 wavelet transform,
In the correction process for the specific high frequency coefficient integrated with the correction process for the specific low frequency coefficient by the coefficient correction unit,
Interleaved coefficient sequence H0 L1 H1 L2 H2 L3 H3 L4 H4 (where H0, H1, H2, H3, H4 are high frequency coefficients, L1, L2, L3, L4 are low frequency coefficients, and tile boundaries are H2 And located between L3)
The value of the high frequency coefficient H3 at
H3 = -0.5L2 + 0.557545L3-0.05754L4 + 0.03372H4
An image processing apparatus corrected by the above. The image processing apparatus according to claim 1 or 4 ,
The condition determining means detects an edge component parallel to a tile boundary existing in the vicinity of a tile boundary, and determines the specific condition when the edge component exceeds a predetermined value. . The image processing apparatus according to claim 9 .
The image processing apparatus, wherein the condition determination unit excludes an edge component on a tile boundary from a detection target. The image processing apparatus according to claim 9 .
The frequency transform is a wavelet transform
The image processing apparatus according to claim 1, wherein the condition determination unit detects an edge component within a distance less than half of a tap length of a wavelet transform low-pass filter from a pixel in contact with a tile boundary. The image processing apparatus according to claim 9 .
The image processing apparatus according to claim 1, wherein the condition determination unit detects an edge component using a low frequency coefficient. The image processing apparatus according to claim 12 .
The condition determining means detects an edge component by taking a difference between adjacent low frequency coefficients, and an image processing apparatus. The image processing apparatus according to claim 12 .
The frequency transform is a 5x3 wavelet transform,
The condition determining means detects an edge component by taking a difference of low frequency coefficients in a right tile located at a distance of 0 and a distance of 2 from a tile boundary. The image processing apparatus according to claim 12 .
The frequency transform is a 5x3 wavelet transform,
The condition determining means obtains an edge edge component by taking a difference between a low frequency coefficient in the right tile located at a distance of 0 from the tile boundary and a low frequency coefficient in the left tile located at a distance of 1 from the tile boundary. An image processing apparatus characterized by performing detection. The image processing apparatus according to claim 14 .
The image processing apparatus according to claim 1, wherein the predetermined value related to the edge component is selected based on 24. The image processing apparatus according to claim 15,
The image processing apparatus according to claim 1, wherein the predetermined value related to the edge component is selected based on 6. To remove the tile boundary distortion of the image obtained by dividing the image into tiles that do not overlap and frequency-converting the frequency coefficient in units of tiles and then performing inverse frequency conversion after quantization and inverse quantization An image processing method comprising:
18. An image processing method comprising processing steps corresponding to the functions of the condition determination unit and the coefficient correction unit according to claim 1 . A program that causes a computer to function as the condition determination unit and the coefficient correction unit according to any one of claims 1 to 17 . A computer-readable information recording medium on which the program according to claim 19 is recorded.

Images