KR100606357B1 - Method of decomposing and composing multimedia data based on discrete wavelet transformation and system for decomposing and composing the same - Google Patents

Abstract

Disclosed are a wavelet-based multimedia data decomposition and recovery method and system for use in a portable wireless terminal. The wavelet-based multimedia data decomposition and reconstruction method processes the multiplication and division between the coefficients of the wavelet filter and the image data by a simple combination of shift operations. As a result, image compression and reconstruction using wavelet transform can be processed very quickly, and the efficiency in image compression and reconstruction is high.

Description

Method and system for decomposing and restoring multimedia data based on wavelet {Method of decomposing and composing multimedia data based on discrete wavelet transformation and system for decomposing and composing the same}

1 is a flowchart illustrating a process of decomposing and restoring wavelet-based multimedia data according to an exemplary embodiment of the present invention.

FIG. 2 is a flowchart illustrating an image decomposition and reconstruction process of FIG. 1.

3 illustrates a structure of three levels of subband regions of a wavelet transform result of an image according to an exemplary embodiment of the present invention.

4 is a diagram illustrating a three-level two channel bi-orthogonal wavelet filter bank used in a three-level wavelet decomposition process of an image according to an exemplary embodiment of the present invention.

5 is a diagram illustrating a three-level two channel bi-orthogonal wavelet filter bank used in a three-level wavelet reconstruction process of an image according to an exemplary embodiment of the present invention.

FIG. 6 is a diagram illustrating a wavelet decomposition process of an image according to an exemplary embodiment of the present invention through a shift operation using a low pass filter.

FIG. 7 is a diagram for describing a wavelet reconstruction process of an image according to an exemplary embodiment of the present invention through a shift operation using a high pass filter.

8 is a flowchart specifically implementing the wavelet transform process of FIG. 2.

FIG. 9 is a flowchart specifically illustrating an inverse wavelet transform process of FIG. 2.

10 shows a data structure of an image file after image decomposition according to a preferred embodiment of the present invention.

11 illustrates a screen configuration of an animation authoring tool for PDA using wavelet transform according to an embodiment of the present invention.

 12A and 12B are separate views of the image and the metabolic portion of FIG. 11.

13A and 13B illustrate a display screen of an animation viewer dedicated for PDA using inverse wavelet transform according to an exemplary embodiment of the present invention.

The present invention relates to a wavelet-based multimedia data decomposition and recovery method and system. More specifically, the present invention relates to a wavelet-based multimedia data decomposition and recovery method and system for use in a portable wireless terminal.

Most video information is stored and transmitted in the form of digital images. When the size of an image becomes large, the storage of the digital image takes up a lot of storage capacity and the transmission of the digital image takes a lot of time and money. Accordingly, various techniques for compressing and reconstructing such digital images have been proposed.

Image compression technologies developed so far include JPEG (Joint Photographic Experts Group) and MPEG (Moving Picture Experts Group). JPEG is a compression standard for still image compression, and MPEG is widely used as a standard for video compression. The JPEG or MPEG has been developed based on the Discrete Cosine Transform (DCT) theory.

Recently, wavelet transforms have been developed to overcome the limitations due to the spatial and spatial characteristics of conventional image compression techniques and to perform transforms suitable for the characteristics of low and high frequency regions.

Among many discrete orthogonal transformations, the discrete wavelet transform has several useful properties, including locality and energy cohesion. In general, the compression effect of the wavelet transform is superior to that of the existing compression standard JPEG. For this reason, the next generation compression standard, JPEG-2000, has changed the compression method using the DCT based on the wavelet.

The result of the wavelet transform is that the low frequency region containing relatively more meaningful information that affects human visual perception is concentrated on one side, and the high frequency region containing relatively less is concentrated on one side. It has a subband region partition structure.

In order to improve the frame rate of the still image compression or the real-time video compression or the image quality, a fast and efficient calculation process is required. Particularly in a mobile environment such as a portable wireless terminal, for example a personal digital assistant (hereinafter referred to as a PDA), there is a need for an efficient and faster image compression and decompression method.

In conventional wavelet transformation, a convolution is performed with a digital filter made using a basis function that satisfies the wavelet condition. In the convolution, a multiplication operation is performed between the coefficients of the wavelet filter and the image data.

Therefore, in the case of using the conventional wavelet transform technique, the calculation process is complicated and the calculation speed is lowered during the wavelet transform. In particular, when a conventional wavelet transform technique is used for compressing and restoring an image in a limited environment such as a mobile wireless terminal, the efficiency of processing of compressing and restoring an image is inferior.

In addition, for example, in the case of providing an animation such as a cartoon in a mobile environment in the form of an e-book service, when a dialogue consisting of images and texts is compressed together using an existing image compression technique. Compression efficiency is lowered. Compared with image compression, the text is composed of lines. When compressed using the image compression technique, the compression efficiency is lowered, and the state after restoration is not satisfactory for the user.

A first object of the present invention for solving the above problems is a wavelet-based multimedia data compression method and the corresponding that can provide the optimum compression efficiency for each of the picture portion and the text portion when the animation is compressed in a mobile environment In providing a system to do so.

In addition, a second object of the present invention is to provide a wavelet-based multimedia data compression method and a corresponding system capable of providing optimal reconstruction efficiency for each of the picture portion and the text portion when the animation is restored in a mobile environment. .

According to an aspect of the present invention, a first multimedia data including image data and text data is received, the image data and the text data are separated from the first multimedia data, and the image data is input. Decompose an image frame corresponding to a plurality of levels in the first and second directions, and shift the binary data value of the image data for each level to correspond to the binary coefficients of the wavelet filter having binary coefficients. Brid transform generates compressed image data, codes the separated text data in a markup language for mobile content to generate second text data, and adds the second text data to the compressed image data to generate a second image data. Wavelet comprising generating multimedia data It provides half of the multimedia data compression method and the corresponding system.

According to an aspect of the present invention, a multimedia data including compressed image data and text data coded in a markup language for mobile content is input using a wavelet filter of a first binary coefficient. The image data and the text data are separated from the multimedia data, and the inverse wavelet is shifted by shifting the binary data value of the image data to correspond to the second binary coefficient of the inverse wavelet filter having a second binary coefficient. Generating a reconstructed image by converting, decoding the text data into a markup language for mobile content to generate a second text, and overlapping and displaying the reconstructed image on the reconstructed image. Wavelet-based multimedia data restoration method and this Provides a system equivalent to this.

In the wavelet-based multimedia data decomposition and reconstruction method according to the present invention, multiplication and division between coefficients of the wavelet filter and image data are processed by a combination of simple shift operations. Therefore, image compression and reconstruction using wavelet transform can be processed very quickly, and the efficiency in image compression and reconstruction is high.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a process of decomposing and restoring wavelet-based multimedia data according to an exemplary embodiment of the present invention.

Referring to FIG. 1, first, first multimedia data including image data and text data is received. (Step S10) In this case, the first multimedia data relates to image, sound, voice, and text information. Contains data. For example, in the case of an animation such as a cartoon, the first multimedia data may include text data such as picture data and dialogue, or may further include voice data for a sound or dialogue portion.

 In this case, for example, it is efficient to separate and compress text parts such as picture parts and dialogue. In particular, the image data and the text data are separated from the first multimedia image data in order to apply a wavelet transform technique suitable for a picture portion.

First, in the case of an image, the source image is decomposed by dividing the source image into a plurality of levels in a horizontal direction and a vertical direction of a source image frame. Image data of each decomposed block passes through a two channel filter bank of high frequency and low frequency. That is, the image data is compressed by performing a discrete wavelet transform by shifting a binary data value of the image data using a wavelet filter of binary integer coefficients (S30). do.

In the case of text, the separated text data is coded in a markup language for mobile content. (S50) The markup language for mobile content may be, for example, an extensible markup language (XML). . Detailed description will be described later.

The text data coded in the markup language is added to the compressed image data and stored in a memory or the like or transmitted over a network. For example, the network may be a wireless network. The compressed image data and the text data coded in the markup language undergo the following processing to restore the original image and text.

In the case of an image, an inverse wavelet filter satisfying the wavelet filter and a perfect reconstruction condition is used to shift the binary data value of the compressed image data to inverse wavelet transform to restore the original image data. (S40)

In the case of text, the text coded in the markup language is decoded and reconstructed (S60). The reconstructed image and the text overlap each other and are displayed to represent the original animation (S70). Hereinafter, a process of decomposing, compressing, and reconstructing an image will be described with reference to FIG. 2.

FIG. 2 is a flowchart illustrating an image decomposition and reconstruction process of FIG. 1.

Referring to FIG. 2, first, in order to decompose and compress an image, the source image is divided into a plurality of levels in a horizontal direction and a vertical direction of a source image frame, thereby decomposing the source image by subbands. decomposition). Image data of each decomposed block for each level is divided into a high frequency region and a low frequency region, and wavelet transforms each of the low frequency region and the high frequency region using the binary discrete wavelet filter (S31).

The wavelet transformed result is quantized (S33), and the quantized result is Huffman-coded. (S35) The wavelet transform is used to decompose each subband by increasing the correlation of the signal, and then Huffman compression is performed. To compress the data. Huffman compression is the most commonly used method of lossless compression, and it is a method of reducing the total amount of data by assigning the shortest code to the most frequently used information.

Then, Huffman decodes the compressed image data to restore the original image (S41), and dequantizes the Huffman decoded result (S43). The inverse quantized result is inverse wavelet transformed using the inverse wavelet filter of the binary coefficients for the low frequency region and the high frequency region (S45).

Hereinafter, wavelet transform and inverse wavelet transform according to an embodiment of the present invention will be described.

The wavelet transform may be represented as a process of convolution with a digital filter made using a basis function satisfying the wavelet condition. Since most digital filters have real number values, these coefficients must be converted to a finite number of bits through quantization. Distortion will occur during this process and full restoration will not be possible. To eliminate this condition, the coefficients of all filters can be represented by the sum and difference of multiples of two, including negative multipliers.

와 고대역 통과 필터(highpass filter)

을 이용한 완전 복원 조건은 다음의 수학식 1과 같다. Lowpass filter in 2-channel filterbank

And highpass filters

The complete recovery condition using is as shown in Equation 1 below.

G ₀ (z) H ₀ (z) + G ₁ (z) H ₁ (z) = 2z ^-l where l in z ^-l represents delay

G ₀ (z) H ₀ (-z) + G ₁ (z) H ₁ (-z) = 0

For example, if G ₀ (z) is selected as Equation 2 below,

G₀(z)=(z^-1 + 2 + z)

G ₀ (z) = (z ^-1 + 2 + z)

The resulting scale function is a linear B-spline function. Then, H ₀ (z) is selected as in Equation 3 below.

H₀(z)=(-z^-2 + 2z^-1 + 6 + 2z - z²)

H ₀ (z) = (-z ^-2 + 2z ^-1 + 6 + 2z-z ² )

을 없애기 위하여 분해 필터에서는

을 곱해주고, 복원 필터에서는

을 곱해주어, 모든 필터의 계수들을 2의 배수 형태로 나타내도록 한다. 이와 같은 방법으로 모든 필터링 계산을 실수를 사용하지 않는 정수만의 연산으로 가능하게 할 수 있다. 분해와 복원의 필터들이 단순히 시간축의 역순인 직교 웨이브릿의 경우와는 대비하여, 위와 같은 쌍직교의 경우 분해와 복원 필터가 서로 교차하여 양 음의 기호가 바뀌는 시간축 상의 역순인 관계를 가진다.Therefore, this filter combination satisfies the above perfect recovery condition. Coefficient in the expression of the above equation

In order to eliminate the decomposition filter

Multiply by the

Multiply by so that all filter coefficients are in multiples of two. In this way, all filtering calculations can be made with integer only operations that do not use real numbers. In contrast to the case of orthogonal wavelets in which the decomposition and reconstruction filters are simply the reverse order of the time axis, in the bi-orthogonal manner, the decomposition and reconstruction filters intersect each other and have a reverse relationship on the time axis in which the positive and negative symbols are changed.

3 illustrates a structure of three levels of subband regions of a wavelet transform result of an image according to an exemplary embodiment of the present invention.

Referring to FIG. 3, wavelet transformation is performed on a source image at a plurality of levels in a horizontal direction and a vertical direction of a source image frame.

HL _i (HL ₁ , HL ₂ , HL ₃ ) is a subband located at the upper right of each defined area, and the high frequency component in the horizontal direction is displayed, and LH _i (LH is a subband located at the lower left. ₁ , LH ₂ , LH ₃ ) region is a high frequency component in the vertical direction, and the subband located at the lower right, that is, HH _i (HH ₁ , HH ₂ , HH ₃ ) region is a diagonal component appears.

The number of levels may be, for example, three levels, and the number of levels may be four or more levels depending on the application purpose as well as three levels. The image data of each decomposed block is transformed into a frequency domain through two channel filter banks of high frequency and low frequency and divided into a plurality of subbands.

The wavelet transform using the discrete signal is as follows.

s _m,k = h ₀(n-2k) s _m-1,n, m=0, ... ,M

s _{m, k} = h ₀ ( n -2 k ) s _{m-1, n} , m = 0, ..., M

d _m,k = h ₁(n-2k) s _m-1,n, m=0, ... ,M

d _{m, k} = h ₁ ( n -2 k ) s _{m-1, n} , m = 0, ..., M

와

는 각각 웨이브릿 분해를 위한 저대역 통과 필터와 고대역 통과 필터이며, 원 신호는 반복적으로 다음 수학식 6을 이용하여 복원할 수 있다.here

Wow

Are respectively a low pass filter and a high pass filter for wavelet decomposition, and the original signal may be reconstructed using Equation 6 repeatedly.

s _m-1,n = g ₀(n-2k) s _m,k + g ₁(n-2k) d _m,k

s _{m-1, n} = g ₀ ( n -2 k ) s _{m, k} + g ₁ ( n -2 k ) d _{m, k}

와

는 각각 복원용 저대역 통과 필터와 고대역 통과 필터이다. here

Wow

Are respectively a low pass filter and a high pass filter for restoration.

4 is a diagram illustrating a three-level two-channel bi-orthogonal wavelet filter bank used in a three-level wavelet decomposition process of an image according to an exemplary embodiment of the present invention, and FIG. 5 is a preferred embodiment of the present invention. Is a diagram illustrating a three-level two-channel bi-orthogonal wavelet filter bank used in a three-level wavelet reconstruction process of an image according to the present invention.

은 3-레벨 분해를 통하여 고대역 통과 신호들인

로 분해되고, 최하위 레벨 3에서는 저대역 통과 신호

을 얻는다. 도 5를 참조하면,

신호들은 다시 3-레벨 웨이브릿 필터 뱅크를 통하여 복원되어진다.4, the input signal

Is a high-pass signal with 3-level decomposition

Low pass signal at lowest level 3.

Get 5,

The signals are then recovered through a three-level wavelet filter bank.

FIG. 6 is a diagram illustrating a wavelet decomposition process of an image according to an exemplary embodiment of the present invention through a shift operation using a low pass filter.

In a preferred embodiment of the present invention, in order to use the characteristics of the filter coefficients defined above more effectively to perform wavelet transformation at a faster calculation speed, a calculation method using only a shift operation and a superposition operation is used.

Hereinafter, the wavelet decomposition process will be described with an example using a low pass filter having five coefficients. The coefficients of the low pass filter h _{0 with} the image data sequence {..., a, b, c, d, e, f, g, h, i, ...} and 5 binary coefficients are _equal to {- 1/8, 1/4, 3/4, 1/4, -1/8}.

In general, each time one image data value needs to be multiplied with the coefficients of the low pass filter, all five multiplications are performed, and then the sum is calculated. This process continues until all image data has been processed. This low pass decomposition filtering may be represented by Equation 7 as follows.

In a preferred embodiment of the present invention, the equation (7) is more effectively calculated using the shift and superposition operations as follows.

It scales using a data point (eg a) as long as the index of the low pass filter mask is at zero. In digital binary representation, it is 1/4 if you shift-right twice, 1/8 for three shift-rights, and 3/4 if you add one shift-right and two shift-rights.

Using this combination of shift operations, the first intermediate calculation values {-a / 8, a / 4, 3a / 4, a / 4, -a / 8} are obtained. It appears that the data value located at index 0 spreads to both ends of the filter by the weight of the filter values (the coefficient value of the filter).

This calculation is performed for five clock cycles, and after five clock cycles all five intermediate calculated values of one particular column (e.g., the corresponding column of data c in FIG. 6) are added. Then, the first calculated value l _b0 (−a / 8 + b / 4 + 3c / 4 + d / 4−e / 8) is obtained as shown in the vertical line at the position of data c of FIG. 7. This is the same as l ₀ (-a / 8 + b / 4 + 3c / 4 + d / 4-e / 8) of Equation 7, wherein the first calculated value l _a0 (-a / 8 + b / 4 + 3c / 4 + d / 4-e / 8). The difference is that the image data value and the filter coefficient value are conventionally processed through a multiplication operation, which makes the calculation more complicated and slows down the calculation. However, in the present invention, the image data value is processed using only the shift operation and the sum operation. This is simple and speeds up computation.

Then, after 2 clock cycles after the first clock (third clock), the next intermediate data, l _b1 (-c / 8 + d / 4 + 3e / 4 + f / 4-g / 8), due to the down-sampling characteristic during wavelet conversion Can be obtained. This is equal to l ₁ (−c / 8 + d / 4 + 3e / 4 + f / 4 − g / 8) of Equation 7. Then, after another two clock cycles (the fifth clock), the next intermediate data, l _b2 (-e / 8 + f / 4 + 3g / 4 + h / 4-i / 8), can be obtained. . This is the same as l ₂ (-e / 8 + f / 4 + 3g / 4 + h / 4-i / 8) of Equation 7, and the second calculation value l _a1 obtained by a general calculation method using a conventional multiplication operation same as (-c / 8 + d / 4 + 3e / 4 + f / 4-g / 8). This calculation continues until all image data sequences have been processed. As such, the calculation process using only shift-superposition can be performed very quickly by eliminating unnecessary calculation processes. Therefore, the image decomposition and compression efficiency is higher than the conventional convolution method.

The high pass filtering can be calculated in a similar manner to the low pass filtering described above. This high pass decomposition filtering may be represented by Equation 8 as follows.

Since the high pass decomposition filtering is performed in a similar manner to the low pass filtering described above, the following description is omitted.

Data passing through the high pass filter is stored for reconstruction, and data passing through the low pass filter is used for the processing of the next level wavelet decomposition.

FIG. 7 is a diagram for describing a wavelet reconstruction process of an image according to an exemplary embodiment of the present invention through a shift operation using a high band filter.

For image reconstruction, again the data sequence is {..., a, b, c, d, e, f, ...} and the coefficient of the high pass filter g ₀ with five binary coefficients is {-1 / 4, -1/2, 3/2, -1/2, -1/4}.

The first process of wavelet reconstruction according to an embodiment of the present invention is an interpolation process that doubles the data sequence by adding zeros between image data. As with the horizontal arrows in FIG. 7, the convolution between the conventional general image data and the filter coefficient is the same as the method described in the image decomposition and compression process.

The high pass recovery filtering may be represented by Equation 9 below.

The inverse wavelet transform according to the preferred embodiment of the present invention is more efficiently calculated using the shift and sum operations as follows.

As long as the index of the filter mask is at 0, scaling is performed using a data point (for example, a). In digital binary representation, once shift-right is 1/2, twice shift-right is 1/4, and once shift-right is added to the original value, 3/2 is obtained. As a result, the first clock obtains the first intermediate calculation values {-a / 4, a / 2, 3a / 2, a / 2, -a / 4} of FIG. This seems to spread the image data value located at index 0 to both ends of the filter by the weight (filter coefficient value) of the filter values.

(After four clock cycles after the first clock, adding all four intermediate calculations in one particular column adds the first calculation h _b0 (-a / 2-b /, as shown in the vertical row at the position of data b in Figure 7). 2) is obtained, which corresponds to h ₀ (−a / 2 − b / 2) of Equation 9 and is equal to the first data h _a0 (−a / 2 − b / 2) obtained by the general calculation method described above. )

In the second clock, as shown in FIG. 7, since the input data value is 0, all calculation processes can be omitted. In the third clock, the shift operation is performed in the same manner to calculate the next image data b (-b / 4). At the fourth clock, add the two data calculated earlier, as shown in the vertical line at the position of input data b as shown by the vertical arrow in FIG. Get the first output h _b0 (-a / 2-b / 2). This value coincides with h ₀ (−a / 2 − b / 2) of Equation 9 and is the same value as the first data h _a0 (−a / 2 − b / 2) obtained by the general calculation method described above. The fifth clock adds all three intermediate calculations except the two zeros to get the second output h _b1 (-a / 4 + 3b / 2-c / 4). This value corresponds to h ₁ (-a / 4 + 3b / 2-c / 4) of Equation 9, and the second data h _a1 (-a / 4 + 3b / 2-c / obtained by the general calculation method described above. Same value as 4). Repeat the same process until you have processed all the data.

As described above, the calculation method using only the shift operation can speed up the calculation because the calculation method is simple. Using the normal convolution in the wavelet transform requires all calculations to be performed even in high-band filtering with many coefficients of zero, which is clearly an unnecessary waste of calculation, while in the present invention, since zero data is not calculated, unnecessary calculations other than shift operations are required. You can speed up the computation by removing the.

The low pass recovery filtering may be calculated in a manner similar to the high pass recovery filtering described above. This low pass decomposition filtering may be expressed as Equation 10 below.

Since the low pass decomposition filtering is performed in a similar manner to the high pass filtering described above, the following description is omitted. When the low pass data and the high pass data are added together, the restoration process at one level is completed.

A pseudo code that specifically implements a wavelet transform and an inverse wavelet transform process for image data according to an embodiment of the present invention described above is as follows.

<Pseudo code of compression algorithm>

procedure Decomp (level, xsize, ysize, re [], im [])

for (level = 0; level <num_levels; level ++)

   for (i = 0, j = 0; j <xsize; i = i + 2, j ++)

      re [j] =-im [i-2] >> 3 + im [i-1] >> 2 + im [i] >> 1 + im [i] >> 2 + im [i + 1] >> 2-im [i + 2] >> 3

; Horizontal low pass decomposition filtering processing part

      re [j + xsize / 2] =-im [i-1] >> 2 + im [i] >> 1-im [i + 1] >> 2

; Horizontal high pass decomposition filtering processing part

   endfor

   for (i = 0, j = 0; j <ysize; i = i + 2, j ++)

      im [j] =-re [i-2] >> 3 + re [i-1] >> 2 + re [i] >> 1 + re [i] >> 2 + re [i + 1] >> 2-re [i + 2] >> 3

; Vertical low pass decomposition filtering processing part

      im [j + ysize / 2] =-re [i-1] >> 2 + re [i] >> 1-re [i + 1] >> 2

; Vertical high pass decomposition filtering processing part

   endfor

xsize = xsize / 2;

ysize = ysize / 2;

endfor

<Pseudo code of restoration algorithm>

procedure Recons (level, xsize, ysize, re [], im [])

for (level = num_levels-1; level> = 0; level--)

   for (i = 0, j = 0; j <ysize; i ++, j ++)

      re [j] = im [i];

      re [j + 1] = im [i] >> 1 + im [i + 1] >> 1

; Vertical Low Pass Recovery Filtering Part

      re [j + ysize / 2] =-im [i-1] >> 2 + im [i] + im [i] >> 1-im [i + 1] >> 2;

      re [j + ysize / 2 + 1] =-im [i] >> 1-im [i + 1] >> 1

; Vertical high pass recovery filter processing part

   endfor

   for (i = 0, j = 0; j <xsize; i ++, j ++)

      im [j] = re [i];

      im [j + 1] = re [i] >> 1 + re [i + 1] >> 1

; Horizontal low pass recovery filtering part

      im [j + ysize / 2] =-re [i-1] >> 2 + re [i] + re [i] >> 1-re [i + 1] >> 2;

      im [j + ysize / 2 + 1] =-re [i] >> 1-re [i + 1] >> 1

; Horizontal high pass recovery filtering part

   endfor

xsize = xsize * 2;

ysize = ysize * 2;

endfor

Hereinafter, an image compression and decompression algorithm will be described with reference to the pseudo code with reference to FIGS. 8 and 9.

8 is a flowchart specifically implementing the wavelet transformation process of FIG. 2, and FIG. 9 is a flowchart specifically implementing the inverse wavelet transformation process of FIG. 2.

Referring to FIG. 8, first, variable values such as level, image horizontal size (xsize) and image vertical size (ysize) are initialized.

The level variable is checked to determine whether it has a final level value (num_levels) (S803). In the case of the final level, the decomposition process is terminated. Direction, horizontal direction) and vertical direction (or column direction, vertical direction) to perform the wavelet transform.

It is determined whether the processing in the horizontal direction of the image is completed (S805). When the processing in the horizontal direction is not completed, S807 to S809 are repeatedly performed. That is, first, the horizontal low-band decomposition filtering process is performed by shifting and summing image data values for the low frequency region (S807), and then the horizontal high-band by shifting and summing image data values for the next high frequency region. Pass-through decomposition filtering processing is performed (S809).

When the horizontal direction processing is completed, it is determined whether the vertical direction processing of the image is completed (S811). When the vertical direction processing is not completed, S813 to S815 are repeatedly performed. In other words, first, by performing a shift operation on the low frequency region and summing the image data values, the vertical low pass reconstruction filtering process is performed (S813). The reconstruction filtering process is performed (S815). When the vertical direction processing is completed, the level value is increased by one, and then the process returns to S803 to process the next level image.

Referring to FIG. 9, first, variable values such as level, image horizontal size (xsize), and image vertical size (ysize) are initialized (S901).

The level variable is checked to determine whether it is at level 0 (S903). If the level is 0, the restoration process is terminated. If the level is not at level 0, the restoration process is sequentially performed. And perform an inverse wavelet transform in the vertical direction (or column direction, vertical direction).

It is determined whether the horizontal processing of the image is completed (S905), and when the horizontal processing is not completed, steps S907 to S909 are repeated. That is, the horizontal low pass recovery filtering process is performed by first shifting and summing image data values for the low frequency region (S907), and the horizontal high band by shifting and summing image data values for the next high frequency region. Pass-through recovery filtering is performed (S909).

When the horizontal direction processing is completed, it is determined whether the vertical direction processing of the image is completed (S911). When the vertical direction processing is not completed, steps S913 to S915 are repeatedly performed. That is, first, by performing a shift operation on the low frequency region and summing the image data values, the vertical low band reconstruction filtering process is performed (S913). The reconstruction filtering process is performed (S915). When the vertical direction processing is completed, the level value is decreased by one, and then the process returns to S903 to process the next level image.

10 shows a data structure of an image file after image decomposition according to a preferred embodiment of the present invention.

Referring to FIG. 10, an 8 byte header (File ID information tag 1001, Number of decomposition level 1003, Image size (row, 1005), Image size (column, 1007), Initial quantization factor (1009)), 3-byte quantum and related information (Quantization information tag 1011, Quantization factor for lowpass 1013), Lowpass image data (1017) and Lowpass information tag (1015) after 3-level image decomposition ), Quantization tags for each high-band of the level (Quantization information for highpass level 1, level 2 (1019, 1023)), quantization factor (Quantization factor (1021, 1025), Huffman coding information tag (Huffman coding information) tag 1027, Huffman tree data 1029, Encoded highpass data 1033 after three-level image decomposition, and a related information tag.

On the other hand, the dialogue portion of the animation data consisting of pictures and dialogues is coded in a markup language such as XML, and an example of an XML tag for the dialogue portion is shown next.

<Example of meta tag XML tag>

<ID> Specify ID </ ID>

<cut> Sequencing cuts </ cut>

<position> Position of metabolism </ position>

<font> Specify metabolic fonts </ font>

<color> Specify metabolic color </ color>

<typing> Specify the effect of letter by letter as if typing a metabolism </ typing>

<voice> Specify the effect of reading metabolism </ voice>

<text> Metabolism </ text>

<image> Specify a picture </ image>

11 illustrates a screen configuration of an animation authoring tool for PDA using wavelet transform according to an embodiment of the present invention. 12A and 12B are separate views of the image and the metabolic portion of FIG. 11.

Referring to FIGS. 11, 12A and 12B, after adjusting the configuration, size, etc. of a picture by using an animation authoring tool for PDA, the dialogue is separated and inserted. First, a picture 1101a is produced using graphic software such as Photoshop, and then a dialogue part 1101b is added using the animation tool. The dialogue part 1101b may select the location, font, and color of the dialogue part 1101b using the location selection areas 1107 and 1109, the font selection area 1111, and the color selection area 1113. That is, as shown in FIG. 11, after adjusting the position of the speech balloon, which is the area 1110c for displaying dialogue, with a cursor or the like, writing the dialogue in the caption window causes the dialogue to be simultaneously recorded in the dialogue display region 1101c. . After all animation authoring work is completed using the animation authoring tool, the picture portion 1101a is compressed using a wavelet transform according to an exemplary embodiment of the present invention, and the dialogue and other information are, for example, in XML format. It is stored in the header portion of the stored and compressed image to form one file as a whole. Since all information including the metabolism is stored in XML format, it is possible to use the search function efficiently.

13A and 13B illustrate a display screen of an animation viewer dedicated for PDA using inverse wavelet transform according to an exemplary embodiment of the present invention.

Referring to FIGS. 13A and 13B, the PDA-only animation viewer decompresses and compresses a compressed picture image as described above by inverse wavelet conversion, and reads text data together with position, font, and color information about a dialogue part. Display overlapping picture and dialogue on PDA screen. In FIG. 13B, two cut cartoons are displayed on one PDA screen. In one PDA screen, it can be set to show from one basic cut to maximum 4 cuts considering the small screen of PDA.

By using the animation authoring tool and a dedicated viewer as described above, an e-book service for a cartoon can be provided to a portable wireless terminal, for example, a PDA.

The wavelet transform and the inverse wavelet transform process may be performed through software calculation, but may be implemented to be processed faster by implementing hardware consisting of digital logic circuits such as shift operation and logical sum.

In addition, since the wavelet-based multimedia data decomposition, compression and decompression method can be used for wavelet transform based still image codec and video codec, it can be applied to JPEG-2000 or MPEG-4.

According to the present invention as described above, the wavelet-based multimedia data compression and decompression method solves the multiplication and division between the coefficients of the wavelet filter and the image data by a combination of simple shift operations, thereby multiplying all the calculations. The calculation speed can be improved by processing only the shift operation of the integer coefficient without division.

In addition, the computation process using only the shift-superposition operation eliminates unnecessary computation and can very quickly process image compression and reconstruction using wavelet transform. As a result, the efficiency of image decomposition, compression and reconstruction is higher than that of the conventional convolution method.

Therefore, the present invention can be effectively applied to a portable wireless terminal having a processor limitation and requiring a high processing speed.

In addition, by compressing and restoring an animation provided to a portable wireless terminal such as a cartoon by separating the image and the text, it is easy to modify the text or translate it into a foreign language.

In addition, by compressing and restoring an animation provided to a portable wireless terminal such as a cartoon by separating the image and the text, the compression ratio and the resolution can be increased, and the text portion can be searched.

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art will be variously modified and changed within the scope of the invention without departing from the spirit and scope of the invention described in the claims below I can understand that you can.

Claims (22)

Receiving first multimedia data including image data and text data; Separating the image data and the text data from the first multimedia data; Decompose an image frame corresponding to the image data into a plurality of levels in the first and second directions, and shift the binary data values of the image data for each level to correspond to the binary coefficients of the wavelet filter having binary coefficients. Generating compressed image data by performing wavelet transform on the result; Coding the separated text data in a markup language for mobile content to generate second text data; And Generating second multimedia data by adding the second text data to the compressed image data. 2. The method of claim 1, wherein the number of levels is three levels. The method of claim 1, wherein the wavelet filter is a bi-orthogonal wavelet filter. The method of claim 1, wherein generating the compressed image data By dividing each level into a low frequency region and a high frequency region, and shifting and summing binary data values of the image data by a predetermined number of times to correspond to the binary coefficients of the wavelet filter in each of the low frequency region and the high frequency region. Converting wavelets;  Quantizing the wavelet transformed result; And And Huffman coding the quantized result. The method of claim 4, wherein the second multimedia data further comprises Huffman coding information, quantization information for the low frequency region, and quantization information for the high frequency region. The method of claim 1, wherein the markup language for mobile content is an extensible markup language (XML). The wavelet-based multimedia data compression method of claim 1, wherein the second text data includes at least one of position, font, and color information on a display area of the text. The method of claim 1, wherein the second text data includes at least one of a tag related to an effect of displaying the text by one letter and a tag related to a sliding effect of scrolling the text at a predetermined time interval. Wavelet-based multimedia data compression method. The wavelet-based multimedia data compression method of claim 1, wherein the wavelet-based multimedia data compression method is used in a portable wireless terminal including at least one of a personal digital assistant (PDA) and an IMT 2000 phone. Multimedia data compression method. The method of claim 1, wherein the image is a picture portion of an animation including a cartoon provided to an e-book service, and the text is a dialogue portion of the animation. Receiving multimedia data including compressed image data and text data coded in a markup language for mobile content using a wavelet filter of first binary coefficients; Separating the image data and the text data from the multimedia data; Generating a reconstructed image by performing an inverse wavelet transform on the binary data value of the image data to correspond to the second binary coefficient of the inverse wavelet filter having a second binary coefficient; Generating second text by decoding the text data into a markup language for mobile content; And And overlapping the second text on the reconstructed image and displaying the second text. 12. The method of claim 11, wherein the second binary coefficients are selected together with the wavelet filter to satisfy a complete reconstruction condition. The method of claim 11, wherein generating the reconstructed image Dequantizing the image data; Inversely Huffman coding the dequantized result; And Performing a reverse wavelet transform by summing the inverse Huffman coded results by shifting the binary data values of the image data by a predetermined number of times to correspond to the second binary coefficients of the inverse wavelet filter in a low frequency region and a high frequency region; Wavelet-based multimedia data recovery method comprising the. 12. The method of claim 11, wherein the markup language for mobile content is an extensible markup language (XML). The wavelet-based multimedia data recovery method of claim 11, wherein the wavelet-based multimedia data recovery method is used in a portable wireless terminal including at least one of a personal digital assistant (PDA) and an IMT 2000 phone. How to restore multimedia data. The cartoon image of claim 15, wherein the displaying comprises overlapping the second text on the restored image to provide an e-book service. Wavelet based multimedia data restoration method characterized in that the step of displaying. The cartoon image of claim 15, wherein the displaying comprises overlapping the restored text on the restored image to provide an e-book service. Wavelet based multimedia data restoration method characterized in that the step of displaying. The method of claim 15, wherein the displaying comprises displaying the restored image and the restored text in a sliding manner so as to scroll one scene at a predetermined time interval on the screen of the personal digital assistant (PDA). Wavelet-based multimedia data recovery method characterized in that. The method of claim 15, wherein the second text is output as voice from the personal digital assistant. delete Memory in which the program is stored; A processor coupled to the memory to execute the program Including but not limited to: The processor is in accordance with the program Receiving first multimedia data including image data and text data; Separating the image data and the text data from the first multimedia data; Decompose an image frame corresponding to the image data into a plurality of levels in the first and second directions, and shift the binary data values of the image data for each level to correspond to the binary coefficients of the wavelet filter having binary coefficients. Generating compressed image data by performing wavelet transform on the result; Coding the separated text data in a markup language for mobile content to generate second text data; And And generating second multimedia data by adding the second text data to the compressed image data. Memory in which the program is stored; A processor coupled to the memory to execute the program Including but not limited to: The processor is in accordance with the program Receiving multimedia data including compressed image data and text data coded in a markup language for mobile content using a wavelet filter of first binary coefficients; Separating the image data and the text data from the multimedia data; Generating a reconstructed image by performing an inverse wavelet transform on the binary data value of the image data to correspond to the second binary coefficient of the inverse wavelet filter having a second binary coefficient; Generating second text by decoding the text data into a markup language for mobile content; And And overlapping the second text on the restored image to display the second text.

Images