JP4012004B2 - Image processing apparatus and method, computer program, and computer-readable storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus that compresses an image using a discrete wavelet transform.
[0002]
[Prior art]
In recent years, image compression / encoding using discrete wavelet transform has attracted attention. For example, after the image to be encoded is divided into tiles of a certain size, the divided tiles are divided into a plurality of frequency bands (subbands) by performing discrete wavelet transform, and then the transform coefficients of each frequency band are quantized. Further, each quantization result is entropy-coded.
[0003]
As a method of performing discrete wavelet transform on the image data, for example, high-frequency and low-frequency separation processing by one-dimensional filtering is performed on the original image in the horizontal direction, and then the one-dimensional filter is also applied in the vertical direction. Is divided into four subbands LL, LH, HL, and HH, and a similar four-division process is repeated for LL corresponding to a low frequency component.
[0004]
For example, Japanese Patent Application Laid-Open No. 2000-69292 proposes a method and apparatus for optimizing the number of repetitions of the above four divisions according to the contents of an image.
[0005]
[Problems to be solved by the invention]
However, in this document, depending on the combination of the image size, tile size, number of discrete wavelet transforms, unit size to be encoded later, etc., there is a problem of increasing the generation of accompanying data generated every time encoding is performed. ing.
[0006]
The present invention has been made in view of the above conventional example, and accompanying data (code block attribute data) generated for each code block which is a unit for encoding the subband obtained by the discrete wavelet transform in the subsequent stage. To provide an image processing apparatus and method, a computer program, and a computer-readable storage medium capable of determining a good divided tile size in consideration of minimizing the amount of processing, time required for the processing, and reducing the amount of compressed data It is what.
[0007]
[Means for Solving the Problems]
In order to solve this problem, for example, an image processing apparatus of the present invention comprises the following arrangement. That is,
Input means for inputting the number of pixels in the horizontal direction H, the image data in the vertical direction number of pixels V,
The input image data is divided into tiles of x × y size defined by a horizontal tile size x where the maximum processable value is X and a vertical tile size y where the maximum processable value is Y. Dividing means to
Transform means for performing a two-dimensional discrete wavelet transform on the tiles to transform to a plurality of frequency components up to a predetermined level;
Entropy encoding means for entropy encoding each frequency component obtained by the conversion means for each preset M × N code block ,
The dividing means includes
X is subtracted if H ≧ X, H is the initial value of x if H <X, x is subtracted until x / M = 2 ^P (P is a natural number) if x ≧ 2M, and if x <2M X finally obtained by subtracting x until M / x = R (R is a natural number) is set as the horizontal tile size x to be applied to the division,
Y is subtracted if V ≧ Y, H is the initial value of y if H <Y, y is subtracted until y / N = 2 ^Q (Q is a natural number) if y ≧ 2N, and if y <2N The y obtained by subtracting y until N / x = S (S is a natural number) is set as the vertical tile size y applied to the division .
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings.
[0009]
FIG. 1 is a diagram showing the overall structure of an image processing apparatus applied to this embodiment. Reference numeral 1 denotes an image input unit, 2 denotes a discrete wavelet transform unit, 3 denotes a quantization unit, 4 denotes an entropy encoding unit, and 5 denotes a code output unit. Hereinafter, the operation of each unit will be described in order.
[0010]
[Image input section]
First, image data to be encoded (size is set to H × V) is input to the image input unit 1. Here, H represents the number of pixels in the horizontal direction, and V represents the number of pixels in the vertical direction. An input target in the image input unit 1 may be, for example, a digital camera or a scanner, an interface for receiving data, or an external storage device that stores an image file. The input image data is transferred to the discrete wavelet transform unit 2 at the subsequent stage.
[0011]
In the present embodiment, for simplicity of explanation, the image data to be encoded will be described as representing the multi-value image images monochrome (one pixel per example 8-bit grayscale image), the present invention is Not limited to this. For example, the present invention can be applied to a case where a plurality of color component data representing a color image is encoded. In this case, each color component may be compressed in multiple times as data or each of luminance and chromaticity component data as the single image data.
[0012]
[Discrete wavelet transform section]
The discrete wavelet transform unit 2 generates transform coefficients by performing a two-dimensional discrete wavelet transform process on the input image data.
[0013]
FIG. 2 simply shows the internal configuration of the discrete wavelet transform unit 2. The input image data is temporarily stored in the memory 201. Next, tile data having a size (x × y) determined by a predetermined algorithm by the processing unit 202 is sequentially read out from the memory 201, subjected to discrete wavelet transform, and written again in the memory 201 (this The predetermined algorithm will be described in detail after the operation of each unit is described).
[0014]
In addition, discrete wavelet transform processing is performed for each determined tile size in order from the upper left of the image data, but the size of the target image is not necessarily an integer multiple of the tile size. That is, a half-end tile may be generated on the right side or / and the lower side of the image. In this case, the tile is divided with the half size.
[0015]
In the case of the example in FIG. 3, the determined tile size is 160 × 120, and half-end tiles T3, T6, T7, T8, and T9 having a horizontal of 160 or less or a vertical of 120 or less are generated on the right and bottom sides of the image. Shows the case.
[0016]
The method of the discrete wavelet transform itself is known as disclosed in Japanese Patent Laid-Open No. 11-103460 and the like is not described in detail. For example, by performing discrete wavelet transform on input image data, FIG. ) Or a conversion coefficient as shown in FIG.
[0017]
First, a discrete wavelet transform is performed on tile data read from the memory 201 by using a one-dimensional (for example, horizontal) filter, and a high frequency component H and a low frequency component as shown in FIG. Get L. Then, a two-dimensional discrete wavelet transform coefficient as shown in FIG. 4B is obtained by applying a one-dimensional discrete wavelet to the obtained component in the other direction (vertical direction).
[0018]
In FIG. 4B, the high frequency component extracted in both the horizontal and vertical directions corresponds to HH1, the low frequency component extracted from the horizontal direction, and the high frequency component extracted from the vertical direction corresponds to LH1, and the horizontal A high frequency component extracted from the direction and a low frequency component extracted from the vertical component corresponds to HL1, and a low frequency component extracted from both the horizontal and vertical directions corresponds to LL1. In the encoding using the discrete wavelet transform, the coefficients in FIG. 4B obtained here are usually equal to the number of pixels constituting the original tile data.
[0019]
Note that since the LL1 is substantially a reduced image of the input tile data, it is known to perform a two-dimensional discrete wavelet transform on the LL1 only for the purpose of increasing the encoding efficiency. . As a result, a coefficient having a form as shown in FIG. 4C is obtained.
[0020]
In FIG. 4C, the high-frequency component extracted from both the horizontal and vertical directions of LL1 corresponds to HH2, the low-frequency component extracted from the horizontal direction of LL1, and the high-frequency component extracted from the vertical direction to LH2. Corresponding, the high frequency component extracted from the horizontal direction of LL1 and the low frequency component extracted from the vertical direction corresponds to HL2, and the low frequency component extracted from both the horizontal and vertical directions of LL1 is LL2 (original tile). The horizontal and vertical images are ¼ size images). The coefficients of FIG. 4C obtained here are usually the same as the number of pixels constituting the original tile data, as in FIG. 4B.
[0021]
Hereinafter, it is known that the low-frequency component LL2 is discrete wavelet transformed a predetermined number of times in the same manner, and the same processing may be repeated for LLn (n = 1, 2, 3, 4, 5,...). Are known.
[0022]
[Quantization unit]
The quantization unit 3 quantizes each frequency component data sequentially transferred from the discrete wavelet transform unit 2 and transfers the quantized data to the subsequent entropy encoding unit 4. The transfer unit of the frequency component data transferred from the discrete wavelet transform unit is a predetermined size M × N, but when the frequency component is half-end with respect to M × N, the same frequency component of the next tile is half-end. Carry the rest of the minutes.
[0023]
FIG. 5 shows LH1, HL1, and HH1 when the image data of FIG. 3 is applied when the number of discrete wavelet transforms is 3 and the code block that is a unit for encoding is 64 × 64. For simplicity, excluding half-end tiles). When a 160 × 120 pixel block is a tile, as shown in FIG. 5, in the frequency component of HL1 in the tile T1, there are 16 horizontal levels, so the remaining 64-16 = 48 is carried over to T2.
[0024]
[Entropy encoding unit]
The entropy encoding unit 4 decomposes the quantized frequency component data (quantization index) into bit planes, and outputs encoded data by performing binary arithmetic encoding on a bit plane basis.
[0025]
FIG. 6 shows a state where 4 × 4 certain frequency component data (quantization index) is encoded for each bit plane for the sake of simplicity. In the figure, the values other than the three quantization indexes (−6, +3, +13) are 0. In the case of the figure, it is decomposed into four bit planes (i) to (iv) (depending on the bit position that is “1” of the largest quantization index), and 2 in order from the MSB to the LSB. Value arithmetic coding is applied.
[0026]
For 1 bit indicating a positive / negative sign, when a non-zero value is detected for the first time at each position divided into 16 during encoding of each bit plane, that is, each quantization index is 0. Immediately after it is detected that it has a value other than 1 bit, “1 (positive)” or “0 (negative)” indicating a positive / negative sign is added, and at the same time, entropy encoding (arithmetic encoding) is performed. In the figure, immediately after “1” indicated by a thick frame in (i), “1” indicating positive is added and encoded, and “1” indicated by a thick frame in (ii) is encoded. Immediately after that, “0” indicating negative is added and encoded, and “1” indicating positive is immediately added immediately after “1” indicated by a thick frame in (iii). It will be encoded.
[0027]
[Code output section]
The encoded data corresponding to one image or a plurality of images obtained as described above is transmitted from the code output unit 5 to an external device via a transmission path, or transferred to an internal / external storage device.
[0028]
The present invention is characterized in that the size of tile division for the input image executed by the image input unit 1 is variably controlled so that the number of code blocks is reduced. Means for realizing this will be described below. In the following description, it is assumed that the control of the horizontal size of the tile and the control of the vertical size are independent. That is, since the principle of determining the tile size is the same, the principle of determining the tile size in the horizontal direction will be described here, and the description of the means for determining the tile size in the vertical direction will be omitted.
[0029]
FIG. 7A shows each subband when the discrete wavelet transform is performed n times with respect to the horizontal. a is the horizontal size of the subband Hn (Ln). FIG. 7B shows a state in which all tiles are arranged with respect to the horizontal with respect to an arbitrary subband H (nm) (T1 ′, T2 ′,..., T (l−1)). ', Tl'). At this time, a virtual image horizontal size is considered and it is set as H ′. When the horizontal size of the subband H (n−m) and the virtual image horizontal size H ′ are multiplied by (a · 2n) / (a · 2m−1), the horizontal tile size x and the horizontal image size H are obtained. For the horizontal size M of the data transferred from the discrete wavelet transform unit 2, M × 2n−m + 1 = Meq (m) multiplied by (a · 2n) / (a · 2m−1) is defined and equivalent M Treat as.
[0030]
The number of horizontal code blocks in the subband H (nm) corresponds to the number of rectangles delimited by a solid line and a broken line in FIG. That is, if the number of solid lines and broken lines is reduced, the number of code blocks decreases, and if the horizontal size x is increased, the number of horizontal code blocks in the subband H (nm) decreases.
[0031]
Further, when the number of code blocks in the subband H (nm) is Nsb (m) and the total number of code blocks is N, N = Nsb (m). The number decreases.
[0032]
FIG. 8 shows a simple procedure for reducing the number N of code blocks in the horizontal direction by increasing the tile size x in the horizontal direction.
[0033]
This process may be performed inside the discrete wavelet transform unit 2 or may be performed by a CPU (not shown) in the image processing apparatus having the configuration of FIG. Further, the horizontal tile size x is calculated in advance for the predetermined horizontal size H by the procedure of FIG. 8, and the result is stored in a ROM (not shown) in the image processing apparatus having the configuration of FIG. And it may be read out.
[0034]
First, it is determined whether the horizontal size H of the image to be processed is larger than the maximum horizontal tile size X that can be processed (step S801). When the horizontal size H of the image to be processed is larger than the maximum horizontal tile size value X that can be processed, the maximum horizontal tile size value X is set as the horizontal tile size x (step S802). Otherwise, the horizontal size H of the image is set as the horizontal tile size x (step S803).
[0035]
The above operation will be described using a specific example. When the horizontal size H of the input image is H = 256, the maximum value of the horizontal tile size is X = 160, the size of the horizontal code block is M = 16, and the number of discrete wavelet transforms is n = 3, FIG. FIG. 10B shows the state of the horizontal code block when x = 120. Each rightmost tile is a half-end tile that is less than the horizontal tile size x. According to the present embodiment, x = 160 and the number of code blocks in the horizontal direction is N = 19 by the procedure of FIG. This is a smaller number of horizontal code blocks than N = 24 at x = 120.
[0036]
The above is the content of determining the tile size in the horizontal direction, but the same may be done when determining the size y in the vertical direction.
[0037]
In accordance with the above processing, for example, when image compression using discrete wavelet transform is performed in an image processing apparatus such as a digital camera, 1600 × 1200 (2 million pixel class), 2048 × 1536 (3 million pixel class), 2272 A case where an image having a size of × 1704 (4 million pixel class) is divided into tiles will be described. Considering such an image processing apparatus having a code block size of 16 × 16 and a processable tile size of 160 × 128, for example, x = 160 and y = 128 are determined.
[0038]
<Second Embodiment>
In the first embodiment, the number of tiles is reduced by simply increasing the tile size and the number of code blocks is reduced. However, an example of further improving the efficiency will be described as the second embodiment.
[0039]
The number of code blocks in the horizontal direction in the subband H (nm) in FIG. 7 corresponds to the number of rectangles delimited by the solid line and the broken line in FIG. 7B. In addition, if the horizontal tile size x is selected so that the solid line and the broken line overlap, the number of substantial broken lines is reduced, and the number of horizontal code blocks is further reduced.
[0040]
FIG. 9 shows a simple procedure for optimizing the horizontal tile size x again so that the solid line and the broken line overlap in FIG. This process may be performed inside the discrete wavelet transform unit 2 or may be performed by a CPU (not shown) in the image processing apparatus having the configuration of FIG. Further, the horizontal tile size x is calculated in advance for a predetermined horizontal size H and horizontal code block size M by the procedure of FIG. 8, and the result is shown in an image processing apparatus having the configuration of FIG. You may memorize | store in ROM and read it.
[0041]
First, it is determined whether the horizontal size H of the image to be processed is larger than the maximum horizontal tile size X that can be processed (step S901). When the horizontal size H of the image to be processed is larger than the maximum value X of the processable horizontal tile size, the maximum value X of the horizontal tile size is set as the horizontal tile size x (step S902). Otherwise, the horizontal size H of the image is set as the horizontal tile size x (step S903). The process up to this point is the same as in the first embodiment.
[0042]
Next, it is determined whether x ≧ 2M (step S904). If x ≧ 2M, the process advances to step S905 to determine whether the horizontal tile size x is selected so that the solid line and the broken line overlap in FIG. If not, the horizontal tile size x is reduced by 1 (step S906) and the determination is repeated, and the horizontal tile size x is selected so that the solid line and the broken line overlap.
[0043]
On the other hand, if x <2M, it is determined in step S907 whether the horizontal tile size x is selected so that the solid line and the broken line overlap in FIG. If not, the horizontal tile size x is reduced by 1 (step S908) and the determination is repeated, and the horizontal tile size x is selected so that the solid line and the broken line overlap.
[0044]
The above operation will be described using a specific example. When the horizontal size H of the input image is H = 256, the maximum value of the horizontal tile size is X = 160, the size of the horizontal code block is M = 16, and the number of discrete wavelet transforms is n = 3, FIG. FIG. 10B shows the state of the horizontal code block when x = 120. Each rightmost tile is a half-end tile that is less than the horizontal tile size x. FIG. 11 also shows the case where the horizontal size H of the input image is H = 256, the maximum horizontal tile size is X = 160, the size of the horizontal code block is M = 16, and the number of discrete wavelet transforms is n = 3 and x = 128. Is shown. According to the present embodiment, x = 160 → 128 according to the procedure of FIG. 9, and the number of code blocks in the horizontal direction is N = 16. This value has fewer horizontal code blocks than N = 19 at x = 160 and N = 24 at x = 120.
[0045]
When image compression using discrete wavelet transform is performed in an image processing apparatus such as a digital camera by the means described above, 1600 × 1200 (2 million pixel class), 2048 × 1536 (3 million pixel class), 2272 × 1704 A case where an image having a size such as (4 million pixel class) is divided into tiles will be described. Considering such an image processing apparatus having a code block size of 16 × 16 and a processable tile size of 160 × 128, for example, x = 128 and y = 128 are determined.
[0046]
As described above, according to the present embodiment, the number of code blocks can be minimized by controlling the size of tile division by a simple determination method. As a result, it is possible to reduce the processing capacity / time and the amount of compressed data required for code block attribute generation. Furthermore, it is possible to reduce the processing capacity / time and the amount of compressed data required for code block attribute generation. In addition, the tile size can be selected without being restricted by the divisor of the image size.
[0047]
In the embodiment, the code block size has been described as 16 × 16. However, the present invention is not limited to this. For example, it may be 64 × 64.
[0048]
Further, it is not necessary to select a tile size so that the horizontal size and vertical size of each subband is a divisor or multiple of the encoding unit horizontal size M and unit vertical size N, and the tile size can be set without being restricted by the restrictions. It is possible to choose. Furthermore, by compressing each frequency component for each encoding size before entropy encoding, it is possible to improve compression efficiency by deleting information that is less important for human visual characteristics from the image data. It becomes.
[0049]
As is clear from the above embodiment, the main part of the present invention can be realized by software and can be operated by a general-purpose information processing apparatus (for example, a personal computer). In general, in order to operate a computer program on such an information processing apparatus, a readable storage medium (for example, a floppy (registered trademark) disk or a CDROM) storing the computer program is set and copied to the system. Such a computer-readable storage medium falls within the scope of the present invention because it is realized by installation.
[0050]
【The invention's effect】
As described above, according to the present invention, the size of tile division is controlled by a simple determination method to minimize the number of code blocks, and the processing capacity / time and the amount of compressed data required for code block attribute generation are suppressed. It becomes possible.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of an image processing apparatus according to an embodiment.
FIG. 2 is a diagram showing an internal structure of a discrete wavelet transform unit 2;
FIG. 3 is a diagram illustrating a state of tile division.
FIG. 4 is a diagram illustrating a state where a discrete wavelet transform is performed.
FIG. 5 is a diagram illustrating a state in which a subband HL1 in each tile of FIG. 3 is divided into code blocks.
FIG. 6 is a diagram illustrating a state of entropy encoding.
FIG. 7 is a diagram illustrating a state in which an arbitrary horizontal subband is divided into horizontal code blocks for arbitrary image data.
FIG. 8 is a flowchart for determining a horizontal size of a tile in the embodiment.
FIG. 9 is a flowchart for determining the horizontal size of an il in the second embodiment.
FIG. 10 is a diagram illustrating a relationship between a tile size and the number of code blocks when performing wavelet transform in the embodiment.
FIG. 11 is a diagram illustrating a relationship between a tile size and the number of code blocks when performing wavelet transform in the second embodiment.

Claims (8)

Input means for inputting the number of pixels in the horizontal direction H, the image data in the vertical direction number of pixels V,
The input image data is divided into tiles of x × y size defined by a horizontal tile size x where the maximum processable value is X and a vertical tile size y where the maximum processable value is Y. Dividing means to
Transform means for performing a two-dimensional discrete wavelet transform on the tiles to transform to a plurality of frequency components up to a predetermined level;
Entropy encoding means for entropy encoding each frequency component obtained by the conversion means for each preset M × N code block ,
The dividing means includes
X is determined if H ≧ X, H is determined as an initial value of x if H <X, x is subtracted until x / M = 2 ^P (P is a natural number) if x ≧ 2M, and x <2M Then, x finally obtained by subtracting x until satisfying M / x = R (R is a natural number) is set as a horizontal tile size x to be applied to the division,
If V ≧ Y, Y is determined. If V <Y, V is determined as the initial value of y. If y ≧ 2N, y is subtracted until y / N = 2 ^Q (Q is a natural number), and y <2N. Then , an image processing apparatus characterized in that y finally obtained by subtracting y until N / y = S (S is a natural number) is set to a vertical tile size y applied to the division . The dividing means, the image processing apparatus according to claim 1 if the odd occurs on the right side and lower side of the image, characterized in that applying a split remains odd. The transforming means arranges a low frequency component at the upper left of the tile and a high frequency component at the lower right , and the entropy coding means entropy encodes each frequency component from the upper left to the lower right for each M × N size, Encode the entropy coding with the left and right sides of the frequency component, and entropy coding for the same frequency component of the tile adjacent to the right and bottom and bottom right. the image processing apparatus according to any one of claims 1 or 2, characterized. The entropy encoding image processing apparatus according to any one of claims 1 to 3, characterized in that the arithmetic coding. Wherein prior to entropy encoding, the image processing apparatus according to any one of claims 1 to 4, characterized in that it has a quantizing means for quantizing each frequency component for each size of the coding. An input step of inputting the number of pixels in the horizontal direction H, the image data in the vertical direction number of pixels V,
The input image data is divided into tiles of x × y size defined by a horizontal tile size x where the maximum processable value is X and a vertical tile size y where the maximum processable value is Y. Splitting process,
A transforming step of performing a two-dimensional discrete wavelet transform on the tiles to transform to a plurality of frequency components up to a predetermined level;
An entropy encoding step of entropy encoding each frequency component obtained by the conversion step for each preset M × N code block ,
The dividing step includes
X is determined if H ≧ X, H is determined as an initial value of x if H <X, x is subtracted until x / M = 2 ^P (P is a natural number) if x ≧ 2M, and x <2M Then, x finally obtained by subtracting x until satisfying M / x = R (R is a natural number) is set as a horizontal tile size x to be applied to the division,
If V ≧ Y, Y is determined. If V <Y, V is determined as the initial value of y. If y ≧ 2N, y is subtracted until y / N = 2 ^Q (Q is a natural number), and y <2N. Then , an image processing method characterized in that y finally obtained by subtracting y until N / y = S (S is a natural number) is set as a vertical tile size y applied to the division . By computer executing narrowing read, the computer, a function causing a computer program as a system for compressing and encoding the image data,
Input means for inputting the number of pixels in the horizontal direction H, the image data in the vertical direction number of pixels V,
The input image data is divided into tiles of x × y size defined by a horizontal tile size x where the maximum processable value is X and a vertical tile size y where the maximum processable value is Y. Dividing means to
Transform means for performing a two-dimensional discrete wavelet transform on the tiles to transform to a plurality of frequency components up to a predetermined level;
Each frequency component obtained by the conversion means functions as entropy coding means for entropy coding for each preset M × N size code block ,
The dividing means includes
X is determined if H ≧ X, H is determined as an initial value of x if H <X, x is subtracted until x / M = 2 ^P (P is a natural number) if x ≧ 2M, and x <2M Then, x finally obtained by subtracting x until satisfying M / x = R (R is a natural number) is set as a horizontal tile size x to be applied to the division,
If V ≧ Y, Y is determined. If V <Y, V is determined as the initial value of y. If y ≧ 2N, y is subtracted until y / N = 2 ^Q (Q is a natural number), and y <2N. Then , a computer program characterized in that y finally obtained by subtracting y until N / y = S (S is a natural number) is the vertical tile size y applied to the division . Computer-readable storage medium characterized by storing a computer program according to claim 7.

Images