METHOD FOR RESIZING IMAGES IN BLOCK DCT DOMAIN
Technical Field of the Invention
The present invention relates to a method for reducing or increasing size of an image using symmetric convolution in block discrete cosine transform (hereinafter, "DCT") domain.
Background of the Invention
With wide spreading use of digital images the need for an efficient tool for manipulating such images grows as well. A fundamental and important tool for such manipulation is resizing of images. In particular, resizing of image is inevitable in cases where various imaging means shall be supported by a single image data. For example, a video on demand (hereinafter, "VOD") service shall be able to support various differently sized imaging means such as computer monitors, HDTVs, general TVs, as well as LCD screens of mobile phones. Since generating and storing separate images fitting to each of such imaging means would be almost impossible, it would be effective that a single image data is generated and stored, to be used in each of the above imaging means after an appropriate resizing thereof.
Generally, resizing of an image requires two operations: sampling rate change and low pass filtering. For reducing an image size, an input image is low pass filtered and then down-sampled (Decimation); while for increasing an image size, an input image is up-
sampled and then low pass filtered (Interpolation), which operations commonly are performed in the pixel domain.
However, since almost all still images and moving images are stored in compressed forms, a conventional image resizing process comprises three steps of decompressing, resizing, and compressing an image, requiring much time.
In order to overcome such shortcomings, different methods for directly resizing compressed images have been disclosed. Since compressed images are normally discrete cosine transformed (DCT), resizing an image means here resizing in the DCT domain.
One of such methods is disclosed in USP 5,708,732 (Fast DCT Domain Downsampling and Inverse Motion Compensation), wherein an image is downsampled by
1/n through generating one 8x8 DCT block from the average of n 8x8 DCT blocks.
However, a problem with such method is that, though it allows achieving a reduction rate of 1/n, it is incapable of achieving a reduction rate at random, of m/n.
Another method is disclosed in USP 5,845,015 (Method and Apparatus for Resizing Images Using the Discrete Cosine Transform), wherein a 4x4 DCT block is generated in a manner that low pass filtering is performed by multiplying the DCT coefficients with the transform coefficients of the low pass filter, and then, the low pass filtered coefficients thus calculated are folded over to be subsequently subtracted. However, a problem with this method is that the DCT block thus produced cannot be used immediately in the DCT domain, for it is not of 8x8 size as used in a general compression process.
Detailed Description of the Invention
The present invention, conceived to solve the above problems, aims to provide an image resizing method in the DCT domain, which method is capable of resizing an image in any size.
Another objective of the present invention is to provide an image resizing method in the DCT domain, wherein the resized DCT block is of 8x8 size as used in general compression standards.
Still another objective of the present invention is to provide an image resizing method requiring a reduced amount of calculations.
Brief Description of the Drawings
Fig. 1 is a diagram showing the concept of two-fold downsampling in accordance with the present invention.
Fig. 2 is a block diagram of an apparatus for performing two-fold downsampling in accordance with the present invention.
Fig. 3 is a diagram showing the concept of two-fold upsampling in accordance with the present invention.
Fig. 4 is a block diagram of an apparatus for performing two-fold upsampling in accordance with the present invention. Figs. 5a and 5b are concept diagrams showing a method for image resizing in a resizing rate at random.
Description of the Preferred Embodiments
In downloading method as per the present invention, zeros are inserted to the end of each sample of a block to be symmetrically convoluted with DCT coefficients of M window functions, the resulting values of such convolution are added together, and first eight samples thereof are downsampled by 1/M-time; while in upsampling method as per the present invention, eight samples of 0 are padded to eight samples of one block, which are symmetrically convoluted with DCT coefficients of L window functions, and then upsampled by L-time taking only even samples from the resulting values of the convolution. An image is resized by L/M after it has been upsampled by L-time and then downsampled by 1/M-time. The quantity of calculations can substantially be reduced by simplifying the window functions using a matrix equation.
Now, a discussion on the relational expressions of multiplication-convolution properties in the DCT domain follows, such properties being fundamental for embodying a filter for use in the above downsampling/ upsampling processes. A description of the multiplication-convolution in the DCT domain is given in the article, "Symmetric Convolution and the Discrete Sine and Cosine Transforms" by S. A. Martucci (In: IEEE Trans. Signal Processing, vol. 42, No. 5, May 1994).
A DCT generally used in reducing an image size is of type 2, the multiplication- convolution properties of which can be summarized as follows:
Type 2 DCT of a sequence x(n) having size N can be expressed in Equation 1 , while its inverse transformation is expressed in Equation 2.
[Equation 1]
X( k) , ioτO≤k, n<N
[Equation 2] πk n-
*(«) = CϊβHX k)} = -jj *J£Ϊ0u(k)X(k)cos JT •41-) ' for0≤/fe, «<N.
Here, the weighting function u(k) is as Equation 3, below.
[Equation 3]
u( k) _ = / f 11//22.. * k== 00 ] uK*} { 1, l≤k<N I
The multiplication-convolution properties of a DCT in Equation 1 through Equation 3 is defined by Equation 4, below.
[Equation 4]
x( n) x w(n) = C l { C2t{X( k)) <g)C2e{ uK «)} }
W(n), which is a random function here, can be a filtering function in a upsampling/downsampling process as per the present invention.
However, Θ s _ which represents a symmetric convolution, can be defined by
Equation 5, below.
[Equation 5]
X
e and W
e are anti-symmetric extensions of X(k) and W(k), respectively, wherein, e.g., X
e(k) is defined by X(k) by Equation 6.
[Equation 6]
( X(k) , Q≤k< N i
0. k= N
*, (*) = ' - X(2N- k) , N<k<2N}
Thus, if an arbitrary filter in the spatial domain is given, a counterpart filter in the DCT domain can easily be obtained using Equation 4.
Since a DCT adopted in actual image compression is a type 2 DCT in orthogonal form, the total calculation requires a scaling factor.
The relation between a type 2 DCT and a general DCT can be expressed as follows:
Equation 4 can be summarized as Formula 7 below, in view ofthe orthogonal form.
[Equation 7]
x(n) x tυ(n) = C2 ' 0r l Λo{ k)(F(k)C2θriho{X(k))<S>F(k)C2ortho{ u* n)))}
FXk) 2 N. k= 0 , J~2N, l≤k< N \
G(k) = l/F(k)
A description of the method for resizing images in accordance with the present invention is given below from the above described background.
Fig. 1 is a diagram showing the concept of two-fold downsampling in accordance with the present invention, wherein the left column shows examples of digital image signals in the spatial domain, while the right column shows the coefficients generated by applying a (forward) DCT to these signals. i (n)% and xi(n) in (a) are 8 sample signals ofthe adjacent blocks, while X\ (k)% and
Xι(k)%, are their respective DCT coefficients. » in (b) are symmetrical expansions in spatial domain of xι(n)$ and x∑fnjs, respectively, corresponding respectively
to . , which are generated by adding zeros to each end of the corresponding DCT coefficient samples in the DCT domain. In (c), the window functions, >ι(n)i
6 and
2(n)κ, for cutting off of desired parts from the signals in (b) as well as their corresponding DCT coefficients,
and W
2(k)ιβ are shown.
Signals indicated in (d) are generated by multiplying the signals in (b) with the window functions in the spatial domain, which correspond to the values obtained by symmetrical convolution of DCT coefficients in (b) with DCT coefficients of the window functions in the DCT domain.
As shown in (e), the sum of these two signals after the windowing, x(n)ιβ is the same as the original signal, i.e. the signal in (a), which can be expressed in the DCT domain by Equation 8 below.
[Equation 8 ]
X(k)l6 = ,(*)I6 ®5 Wx (k) 6 +X2(k)x6 <8>5 W2(k 6
Accordingly, since the 16 samples in (e) are the same as those obtained through an application of DCT to the adjacent two blocks, a two-fold downsampling can be achieved by taking eight coefficients from the low frequency side.
Fig. 2 shows a block diagram of an apparatus for performing the above two-fold downsampling.
The zero inserters 11, 12 output two 16-sample signals by inserting zeros at the end of each sample of the inputted 8-sample signals Xi and X2 of the two adjacent blocks. The symmetric convolution calculators 13, 14 perform a symmetric convolution of these 16-sample signals to the window function W| and W2, respectively. The adder 15 outputs a result as Equation 8 by adding results outputted by the symmetric convolution calculators 13, 14. The first 8-sample extractor 17 takes first 8 samples from the 16-sample signals resulting from Equation 8, i.e. 8 samples from the low frequency side, and then, outputs two-fold downsampled signal Z.
The window functions, Wι(k)ιβ and Wι(k)i6 can be obtained using Equation 1 , results of which calculation are shown in Table 1.
As shown in Table 1, since the coefficients contain a lot of zeros and the convolution is performed symmetrically, the window functions can be summarized in 8x8 matrices as follows:
[Equation 9]
XMtQsWM** for0≤*<8 * W,"Xl
[Equation 10]
01
[ς\ uoμenbg]
Λ['(ύ
lx
*(ς)
zx *(p)
zx *
τx *(z)
τx
°(o)
zx] =
[H uoμtmbg]
91
[ t uoμtmbg]
(
9*(π) -
,l<«!*>- "ti> >- ("(coin* "(i) »- ("tit)!* ♦ "to"*)-
"αi'j-
("(ci)1*-"**)'*)- (,,(tιι f-"U),jι)- <•'«!)!«♦ «<«!*>- (*,<it) *"(i>!*)- ("(«)'« + "(D ϊ- "(ι) >- *'tt) -
<*'t - " D - <"(ti) ;ΛV o
01
[31 uoμenbg]
»■(«>'_« "(si) -"<f>'- «l(tι)lΛ. «'(ι) "(tD t'Ot »l(ιι)!_ι ♦»'(*) 'a "<»>l- ♦ "ID1* "ω'V o o α "(o) α o
"<π)'.« -"<«>',- ι)|j-"ωlι 1(tι) »"(t) »'(cι>!-«* •'<.)!ιι»*'(ιl1-i «"((>'-« "«>'.-■
"(iD'Λ-'O 'Λ "<S|) |»"<6>'.. "(«>!« ♦»'<.)!• «l(ll)lΛ» "(t) "C6)'*»"(t) "ω' *"!!)1* *l(f>I-»« »,<ι) ι l(t) rpΛΛ o o o o o . o "(o) o
"(tl)lΛ*"(tl)'-l »'(CI)'j(*»'<ll) l »l(lll1Λ»"<6) "(SlU*"!.)1* »'(.)'*♦ •'(()!« ,l(fl +"<C>- »Hc>,-»**,(ι>l« "<l)!*r o o o o o o o "(o)
[π uoμenbg] g
:S OIJOJ SB si oi purj 6 suoμenbg ui xureui qoB
XSIP'Λ S = rxSΣ»ΛV = 8>;y50J°j " ^ε^® 'z(%)zx
C80Z0/Z0HX/I3d Z.C89S0/C0 OΛV
Consequently, a downsampling (zor) can be expressed in matrix form by Equation
16.
[Equation 16]
Z
c ■(3), Z° (4)
8 Z
or(5)
8 Z'(6)
t Z
or(7)
i]
T
}.
Here, the scaling factors, F and G can be expressed by Equations 17 and 18, respectively.
[Equation 17]
The quantity of calculations for an upsampling / downsampling can substantially be reduced in comparison to a conventional image resizing method, considering the zero coefficients in the symmetrical convolutions in matrix form as in the above Equation 16 and calculating the part in {}of Equation 16 in advance.
In the same manner, 8 samples of the adjacent three blocks can be three-fold downsampled by three window functions. Here, left 8 samples of first window function are 1 , while the remaining 16 samples thereof are 0; middle 8 samples of the second window function are 1, while the remaining 16 samples thereof are 0; and right 8 samples of the third window function are 1. while the remaining 16 samples thereof are 0. The DCT coefficients of these window functions can be obtained with Formula 1 above.
A 1/M-time downsampling of an image can be performed by expanded application of the above method, i.e. 8-samples of the adjacent M blocks can be downsampled by M window functions in the same manner.
To elaborate, 8xM samples are generated by inserting a zero sample at the end of each sample of 8-sample DCT blocks, and then, each of the 8xM samples of adjacent M- number DCT blocks is symmetrically convoluted with the DCT coefficients of M-number window functions. 1/M-time downsampling is completed when an 8-sample DCT is generated by adding first eight samples of values resulting from the above M-number symmetrical convolutions. Here, the M-number window function is a function having a
size of 8xM samples, and the n' window function thereof has 1 for its nth eight sample while the remaining samples thereof are all zeros.
In order to generate one 8-sample DCT by adding first eight samples of values resulting from the M-number symmetrical convolutions, one 8xM sample is generated by values resulting from the M-number symmetrical convolutions to each sample, to subsequently generate one 8-sample DCT block through taking first eight samples from the 8xM sample. Alternatively, one 8-sample DCT block can also be generated by adding each sample to one another after first eight samples of values resulting from the M-number symmetrical convolutions have been taken.
Next, the concept of two-fold upsampling in accordance with the present invention is explained referring to Fig. 3. In Fig. 3, the left column exemplifies digital image signals in the spatial domain, while the right column indicates DCT coefficients of these signals. x(n)s in (a) is an 8-sample signal of a block, while X(k)g is DCT coefficient of x(n)s. Y(k)ιβ in (b) represents a signal with added eight zeros at high frequency side of X(k)s, while y(n)i6 represents a signal appearing in the spatial domain at that time. It can be seen that the signal in the spatial domain is an upsampled 16-sample signal. In (c), window functions, w/(n)ι6 and W2(n)ιβ for cutting off desired parts from the signals in (b) as well as their respective DCT coefficients, Wι(k)κ, and W2(k)\(s are shown. A multiplication of (b) signal in the spatial domain by each the window function yields a (d) signal, which corresponds to a signal resulting from a symmetric convolution of the DCT coefficients of (b) with the DCT coefficients of each window function in the DCT domain. It can be seen here that the window functions used in the upsampling are the
same as that used in the downsampling. From the first 8-sample of
and the last 8-
sample of
as in (e) can be obtained, which procedure is the same as that of taking even samples of (d) signal in the DCT domain.
Fig. 4 shows a block diagram of an apparatus for performing the above process in accordance with the present invention.
A zero padder 21 pads zeroes to each 8-sample signal X in one block, at an 8- sample high frequency section, and outputs 16-sample signals, each of which is then symmetrically convoluted with the window functions Wi, W2 by the symmetrical convolution calculators 23, 24. After that, the even sample extractors 25, 26 extract even samples from outputs of the symmetrical convolution calculators 23, 24 and output two 8- sample signals Yi, Y , with which procedure a two fold upsampling process is completed.
The above upsampling process can be expressed in matrices by Equations 19 and 20.
[Equation 19] y°r = G W 2 Fx0f = Rx" ; R=G W,B, F
[Equation 20]
y? = GSWa α2 Fxor = GSW,uiSFxor = SG ^FSx"^ SRSxor
Here, ? > ? are results of the respective upsampled 8-samples, while the
other matrices are the same as those in case ofthe downsampling.
In the same manner, a three-fold upsampling can be preformed on each 8-sample signal in a block with three different window functions as in a three-fold downsampling.
To generalize: First, 8xL samples are generated by adding (L-l)x8 zero samples at the end of an 8-sample DCT block. Then, these 8xL samples are symmetrically convoluted with the DCT coefficients of the L window functions, to yield L symmetrical convolution values of 8xL samples. An L-fold upsampling is completed by generating L DCT blocks of 8 samples after every Lth sample has been taken from these values. Here, the L window functions are functions having a size of 8xL samples, and the nth window function thereof has 1 for its nth 8-sample while the remaining samples thereof are all zeros.
Since a window function as well as a DCT coefficient of an image for convolution in the DCT domain comprise zeroes, and convolution in the DCT domain is performed symmetrically, it can be easily expressed in matrices and a further simplification of the window functions within a prescribed image quality can be achieved.
For example, a matrix as Equation 21 can be obtained in case window functions for downsampling are employed in a manner that several coefficients thereof are substituted by zeros.
By using the simplified Equation 21, a 60% reduction in calculations can be reached in comparison to cases where Equations 11 and 12 are used.
As next, resizing of images at a resizing ratio at random is described below.
In order to achieve a L/M-times resizing of an image, the image is downsampled by 1/M after it has been upsampled by L-times, as described above. An example of such process is illustrated in Fig. 5a. Since the Equation 16 as well as Equations 19 and 20 can easily be transformed for an upsampling/downsampling to a desired size, corresponding window functions can also be easily obtained by the method of the present invention.
Fig. 5b illustrates resizing of an image by 3/2-times, i.e. 1.5 fold resizing. Here, each ofthe 8-sample signal is first upsampled to three 8-sample signals, and then, each two 8-sample signals are downsampled to form one 8-sample signal, with which procedure a 3/2-times resizing ofthe image is achieved.
To generalize: First, 8xL samples are generated by adding (L-l)x8 zero samples at the end of an 8-sample in a 8-sample DCT block. Then, these 8xL samples are symmetrically convoluted with the DCT coefficients of the L window functions, to yield L symmetrical convolution values of 8xL samples. An L-fold upsampling is completed by generating L DCT blocks of 8 samples after every Lth sample has been taken from these
values. After that, 8xM samples are generated by inserting a zero sample at the end of each sample of the DCT blocks upsampled, and each of 8xM samples of adjacent M DCT blocks are symmetrically convoluted with the DCT coefficient of the M window functions, to generate M results of the symmetrical convolution. A L/M-times resizing of an image is completed by generating one 8-sample DCT through addition of all the upper 8-samples of such results.
Here, the L window functions are functions having a size of 8xL samples, and the nth window function thereof has 1 for its n' 8-sample while the remaining samples thereof are all zeros. And, the M window functions are functions having a size of 8xM samples, and the mth window function thereof has 1 for its m' 8-sample while the remaining samples thereof are all zeros.
Industrial Applicability
As described above, since the present invention provides a resized DCT block in
8x8 size as used in general compression standards, no inverse DCT transform is required.
In addition, since the present invention provides an image resizing method capable of resizing an image in a size at random, such as resizing by L/M-times through upsampling an input image by L-times and then downsampling the same by 1/M-times. Furthermore, the present invention provides an image resizing method, wherein the amount of calculations is substantially reduced through simplification of the window functions.