KR19990043414A - A method for estimating hierarchical motion through reconstruction of candidate subcarriers and candidate vectors in wavelet transform domain - Google Patents

Abstract

The present invention relates to a method of hierarchical motion estimation through a candidate vector and re-synthesis of a sub-image using a wavelet transform property having a multi-resolution structure.

The present invention estimates a motion vector from a plurality of candidate vectors to a vector having a minimum distortion for a wavelet transformed low-frequency sub-picture of the highest hierarchical layer, and transforms the estimated motion vector to a low- Compensated sub-image, motion estimation / compensation of the high-frequency sub-image of the layer by the motion vector of the same layer, and inverse wavelet transform of the compensated high-frequency sub-image and the compensated low- The re-synthesized sub-image is obtained. The reconstructed sub-image is applied to the low-frequency sub-image of the corresponding layer to obtain a motion vector. The process is repeated until the reconstructed sub-image has the same resolution as that of the original image. .

The present invention improves the disadvantages of a large amount of computation, a large number of bits, an accumulation of errors, and the like, which are problematic when using the conventional motion estimation method, and accurate motion vector estimation is possible.

Description

In this paper, we propose a new method for estimating motion in a wavelet transform domain using sub-reconstruction and candidate vectors.

The present invention relates to hierarchical motion estimation of a digital image encoding system, and more particularly, to a motion estimation method for reconstructing a sub-image using a wavelet transform property having a multi-resolution structure and obtaining a motion vector through a candidate vector.

Generally. The purpose of digital image coding is to compress image data in order to transmit or store as high quality image as possible within a limited channel capacity. Image compression removes the redundancy existing in a video signal. In the video signal, spectral redundancy between color components, temporal redundancy between the picture and the screen, and spatial redundancy between pixels in the picture exist do. In particular, temporal redundancy in moving picture compression is eliminated using motion estimation and compensation techniques. However, since the conventional motion estimation method using the block matching method and the space-time gradient based method estimates only the component of the translational motion of each local block, There is a disadvantage that this occurs. In order to solve this problem, a motion estimation method using a hierarchical block matching method has been studied. In a hierarchical block matching method, a motion vector estimation structure is hierarchically configured, And the local motion.

Among wavelet transform techniques, Wavelet transform can represent signals with locality with respect to time and frequency, which is advantageous for interpreting image signals having a non-stationary process. Which is similar to the visual characteristics of the human eye.

1 is an octave tree distribution diagram in which an image is divided into bands using wavelet transform. In wavelet transform, a wavelet transform is performed by passing the original image through the low-pass filter and the high-pass filter in the vertical and horizontal directions, respectively, and the wavelet transform is repeatedly performed.

1 (a) shows that the converted region is divided into four bands W ⁰ ₂ (LL), W ¹ ₂ (LH), W ² ₂ (HL), and W ³ ₂ (HH). LL is a low-frequency component in both the horizontal and vertical directions, and LH is a low-frequency component in the horizontal direction and a high-frequency component in the vertical direction. The HL has a high frequency component in the horizontal direction, a low frequency component in the vertical direction, and a high frequency component in the diagonal direction. Thus, the W ⁰ ₂ (LL) sub-image is similar to the original image with DC components of the original image. When the W ⁰ ₂ (LL) sub-image is re-wavelet transformed, the second wavelet transformed sub-images can be obtained. When the low-frequency component W ⁰ ₄ (not shown) obtained in the second conversion is subjected to the wavelet transform again, a band-divided image is obtained as shown in FIG.

As shown in FIG. 1, the wavelet-transformed image is decomposed into a plurality of sub-images having different resolutions and frequency band components. At this time, sub-images with the same resolution are treated as the same level, and these levels form a pyramid structure composed of several layers. This is shown in FIG.

FIG. 2 is a pyramid structure in which images are hierarchically decomposed using wavelet transform, and is represented by 10 sub-images (three levels) as shown in FIG. 1 (b).

Level 1 is composed of three sub-images W ¹ ₂ , W ² ₂ and W ³ ₂ , level 2 is composed of three sub-images W ¹ ₄ , W ² ₄ and W ³ ₄ , Three sub-images W ¹ ₈ , W ² ₈ , W ³ ₈ and one sub-image S ₈ .

As described above, the wavelet transform divides the original image into multiple resolutions having different frequency bands, and the cross-correlation between sub-images having the same directionality at each resolution is very large. Therefore, if the two-dimensional image information is hierarchically divided using the wavelet transform and the motion is estimated in the upper layer containing the global information of the image, the search area is greatly reduced, so the calculation amount can be drastically reduced. Also, since the upper layer contains only low-frequency components, there is an advantage that the estimation error due to the high-frequency noise can be reduced.

Multiresolution Motion Estimation (MRME), which is a representative method for estimating motion in a wavelet transformed image, is a method of estimating and transmitting motion in all wavelet-transformed subbands. Since MRME requires a large number of bits and a calculation amount, there is a method of compensating the motion and estimating the motion using only low-frequency component images for each level in the wavelet-transformed image.

The multi-resolution motion estimation method and the motion estimation method using low-frequency components will be described with reference to FIGS. 3 and 4. FIG.

① Multi-resolution motion estimation

The multi-resolution motion estimation scheme starts with estimating a motion vector (block size P) in a block matching manner at S ₈ of the highest level (level 3), as shown in FIG. The estimated motion vector is transitioned to the sub-picture W ⁱ ₈ (i = 1, 2, 3) of the same level or transitioned to the lower level. Since the resolution increases twice as much as the sub-image at the previous level, the size of the motion estimation block is defined as 'P · 2 ^Mm ' when the level to which the sub-image belongs is m. Where M represents how many levels the image has been decomposed (M = 3 in FIG. 3). Therefore, since the number of motion estimation blocks in all levels is the same, a motion vector at a lower level is scaled by a resolution of a motion vector shifted from a higher level by a resolution twice, The motion is estimated. In this case, it is defined that a more detailed motion vector is estimated near the motion at the lower level based on the higher-level motion vector (refine).

② Motion estimation method using only low frequency components

A motion estimation method using only a low-frequency component image is a method of estimating motion using only low-frequency component images of each level among wavelet-transformed sub-images as shown in FIG.

After wavelet decomposition of the original image into 10 sub-images, only low-frequency images of each level are stored. In step S8, which is a negative low-frequency component of the highest level (level 3), a motion vector is estimated by a block matching method with a size P of the motion estimation block. The estimated motion vector is transferred to the S4 sub-picture of the higher level. At this time, since the resolution of the S4 sub-image is twice as high as that of the low-level S8 sub-image, the motion vector is also shifted after scaling it twice. Then, the tablets (refine) the motion vector transition from the S ₈ and the size of the motion estimation block to 2P. In this way, motion estimation for low-level low-frequency images is performed repeatedly. This process is performed until the resolution of the lower-level low-frequency image becomes equal to the resolution of the original image.

Since the MRME estimates a motion vector for all sub-images at each level, the number of motion vectors to be transmitted is large and a considerable amount of calculation is required. To solve these drawbacks, the method of estimating motion using only low-frequency components of each level in the wavelet transformed image is not to estimate the motion in all the 3M + 1 eigens when decomposing to M level like MRME, Since the motion vector is estimated only for +1 sub-images, the amount of calculation and the number of motion vectors to be transmitted are small. However, if the initial motion vector estimated from the low-level component of the high-level component is erroneously estimated, the error accumulates as the error is lowered to the lower level, resulting in an error that estimates the motion of the image in a completely different direction It is possible. That is, if the image is wavelet-transformed and decomposed into three levels (10 sub-images), the resolution of the S8 sub-image is 1/64 of the original image, while on the energy side, the sum of the energy of the remaining sub- Energy. As a result, the S8 sub-image contains a lot of energy, but the size of the image is considerably small and the motion estimation block becomes smaller accordingly. The QCIF image has a 2 × 2 motion estimation block on the S8 sub-image. In this case, the size of the block is small, which makes it difficult to perform accurate motion estimation using the block matching method.

In the pyramid structure, the MRME technique or the method of estimating the motion only with the image of the low-frequency component estimates the motion in the TOP-DOWN manner. Therefore, the motion vector of the lower level is sensitive to the motion vector of the highest level . That is, when a motion estimation error occurs at the highest level, the error accumulates continuously and causes a larger error at the lower level.

In order to solve the above problems, the present invention has been made to solve the above problems, and it is an object of the present invention to provide a motion vector estimation method and a motion vector estimation method in which a motion vector of each layer is selected from several candidate vectors, And then performing motion compensation on the synthesized image after synthesis to estimate a motion vector for the original image.

According to an aspect of the present invention, there is provided a hierarchical motion estimation method in a wavelet transform domain, the method comprising: (a) generating a first original image and a second original image temporally adjacent to the first original image, Transforming each layer into a hierarchical multiresolution sub-image composed of one low-frequency sub-image and three high-frequency sub-images; (b) dividing the low-frequency sub-image of the uppermost layer of the first original image and the low-frequency sub-image of the upper layer of the second original image into P-sized search blocks and generating a vector having a minimum distortion among a plurality of candidate vectors for each search block To a motion vector; (c) obtaining a motion compensated low frequency sub-image using the motion vector estimated in step (b) and the low frequency sub-image for the first original image; (d) obtaining a synthesized sub-image of a lower layer by performing motion estimation, compensation processing, and inverse wavelet transform of a high-frequency sub-image of the same level as the motion-compensated low-frequency sub-image obtained in step (c); (e) using the synthesized sub-image of the step (d) and the sub-image of the second original image corresponding to the synthesized sub-image, to synthesize the low-frequency sub-image for estimating a motion vector in the step (b) And repeating the step (b) and the following steps until the combined sub-image has the same resolution as the original image.

1 is an octave tree distribution diagram in which an image is divided into bands using wavelet transform,

FIG. 2 is a pyramid structure in which images are hierarchically decomposed using a wavelet transform,

FIG. 3 is a diagram for explaining a multi-resolution motion estimation technique in a wavelet transform domain; FIG.

4 is a diagram for explaining a motion estimation technique using only a low-frequency component image in a wavelet transform region,

FIG. 5 is a block diagram of a hierarchical motion estimation apparatus through re-synthesis of a sub-image according to the present invention.

FIG. 6 is a pyramid structure diagram for explaining the hierarchical structure of a sub-image subjected to wavelet transformation by the hierarchical motion estimation apparatus shown in FIG. 5;

FIG. 7 is a diagram for explaining a procedure for estimating motion through a hierarchical candidate vector in a wavelet transform domain according to the present invention; FIG.

8 is a diagram for explaining a motion estimation process by sub-image re-synthesis in the wavelet transform domain according to the present invention.

DESCRIPTION OF THE REFERENCE NUMERALS

100, 200, 300, 110, 210, 310:

120, 220, 320: candidate vector motion estimation /

130, 230, 330: a motion estimation /

140, 240, 340: Inverse wavelet transform unit 400:

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings.

FIG. 5 is a block diagram of a hierarchical motion estimation apparatus through re-synthesis of a sub-image according to the present invention. FIG. 6 is a pyramid structure diagram for explaining a hierarchical structure of a sub-wavelet transformed by the hierarchical motion estimation apparatus shown in FIG.

Referring to FIG. 5, the wavelet transform unit 100, 200, 300 divides the pixel block S1 of the previous frame original image into a hierarchical multi-resolution image. As shown in FIG. 6, the multi-resolution layer partitioning includes one low-frequency sub-image (layer 1: S2), (layer 2: S4), (layer 3: S8), and three high- 1, S2-2, S2-3) (Layer 2: S4-1, S4-2, S4-3), (Layer 3: S8-1, S8-2, S8-3). Among the sub-images generated in the respective layers, the high-frequency sub-images are provided to corresponding motion estimation / compensation units 130, 230, and 330 of the corresponding layers.

The wavelet transform units 110, 210 and 310 perform wavelet transform on the pixel blocks S1 'of the current original image in a hierarchical manner, and convert wavelet-transformed low-frequency sub-images in the respective layers into corresponding corresponding layers To the candidate vector motion estimation / compensation units 120, 220, and 320 of FIG.

The uppermost layer motion estimation / compensation unit 120 among the candidate vector motion estimation / compensation units 120, 220, and 320 estimates a motion vector using low-frequency sub-images of the wavelet transformed current and past images, The compensating units 220 and 320 perform motion estimation using the sub-image of the wavelet-transformed past image of the layer and the sub-image synthesized from the upper layer of the layer. Motion estimation is performed using a plurality of candidate vectors, and a vector having a minimum distortion among the candidate vectors is estimated as a motion vector for the synthesized sub-image. Here, the candidate vector set includes a motion vector on the upper layer in the same position as the motion vector to be estimated (a block existing at the same position when the image is divided into several blocks) and neighboring vectors around the motion vector.

FIG. 7 is a diagram for explaining a motion estimation process using a hierarchical candidate vector in a wavelet transform domain according to the present invention. The S8 sub-image of the highest layer is divided into blocks of P size and a motion vector is estimated by a block matching method. The motion vector estimated in the subpicture of S8 is scaled by 2 times to the S4 subimage of the lower layer, and the block size is 2P. A new motion vector is obtained by stabilizing the transferred motion vector. The motion vector that transitions on the subpixel S8 transitions along with the motion vector at the same position as the block to be estimated on the S4 subpixel as well as the neighboring motion vector estimated temporally and spatially. These vectors are called candidate vectors. Motion is estimated from the transition candidate vector, and a motion vector having the minimum distortion among the motion vectors is set as a motion vector on S4 sub-picture. This process is repeated until the resolution of the lowest layer becomes equal to the original image. This means that the motion estimation error does not propagate from the upper layer to the next lower layer.

Hereinafter, the operation and effect of the embodiment of the present invention will be described in detail with reference to FIGS. 5 to 8. FIG.

Referring to FIG. 5, the candidate vector motion estimator / compensator 320 for the third layer, which is the highest layer, includes a low-frequency sub-image S8 that is wavelet-transformed with respect to a past frame, After dividing the image S8 'into a search block for a 2 × 2 size motion estimation, for example, a P size, a motion vector for each search block in which motion is currently occurring is estimated using a block matching method for past sub-images . The motion vector estimation is determined as a vector having a minimum distortion using a plurality of candidate vectors as shown in FIG. The block matching method is a process for finding a matching block having the smallest mean square error while setting a search area from a past image and moving a search block for the current frame in one pixel unit within the search area. The detailed description is omitted. In addition, the motion estimation / compensation unit 320 generates a motion compensated image C_S8 using the motion vector estimated by the block matching method. The motion compensated image C_S8 is input to the inverse wavelet transform unit 340 of the layer 3.

The second motion estimation / compensation unit 330 for the third layer, which is the highest layer, performs wavelet-transformed high frequency sub-images S8-1, S8-2, S8-3 and the first motion estimation / Compensated sub-image for the high-frequency sub-image using the motion vector estimated by the compensating unit 320 of the S8 sub-image. At this time, the motion estimation / compensation on the high frequency sub-image at the same level as the S8 sub-image can be implemented by the following two methods. (1) a method of applying the motion vector obtained from the S8 subtraction as it is, and (2) a method of estimating a new motion vector through a block matching method in the shifted peripheral region after shifting by the motion vector based on the motion vector obtained in S8. As shown in (2), a refine process is performed to estimate a more detailed motion vector near the motion space.

The inverse wavelet transform unit 340 transforms the motion compensated image C_S8 for the layer 3 S8 and the motion compensated sub-image for the high-frequency sub-images S8-1, S8-2, and S8-3 of the layer 3 And performs inverse wavelet transform on the synthesized image. The image obtained as a result of the inverse wavelet transform is equal to the size of the S4 sub-image of the layer 2 and is called the combined S4 image (R_S4) (refer to FIG. 8). The synthesized S4 image R_S4 is provided to the candidate vector motion estimator / compensator 220 of the layer 2.

The layer 2 candidate vector motion estimator / compensator 220 multiplies the synthesized S4 image by P × 2 ^Mm, that is, a 2 × P search, similar to the motion performed in the layer 3 candidate vector motion estimator / And estimates a new motion vector using the block matching method with the S4 'sub-image of the current image provided from the wavelet transform unit 210. [

That is, the motion vector estimated on the S8 sub-image is transmitted to the combined S4 sub-image R_S4. Since the synthesized S4 sub-image R_S4 has a resolution twice as large as that of the S8 sub-image, the motion vector on the S8 sub-image is also scaled and transmitted twice. At this time, not only the motion vectors at the same position but also candidate vectors including the motion vectors of the blocks adjacent to the block are transmitted together. After each candidate vector is moved by a vector amount, a motion vector is re-estimated around the vector (refine the motion vector). In the synthesized S4 sub-image R_S4, a motion vector at the same position, which is obtained by scaling the motion vector estimated on the S8 sub-image twice and a neighboring motion vector thereof, is a candidate vector. The estimated motion vectors, that is, the motion vector having the minimum distortion among the candidate vectors, are used to generate the motion compensated S4 sub-image C_S4. The compensated S4 sub-image is provided to the inverse wavelet transform unit 240 of the same layer. The estimated motion vector is provided to the motion estimation / compensation unit 230. [

The motion estimation / compensation unit 230 of the layer 2, like the motion estimation / compensation unit 330 of the layer 3, performs a wavelet-transformed high frequency sub- S4_4, S4-2, and S4-3 and a motion vector-compensated new S4 sub-image C_S4 using the estimated motion vectors of the candidate vector motion estimator / compensator 220. [ At this time, the motion estimation / compensation on the high-frequency sub-image at the same level as the sub-image S4 can be implemented by the two methods (1) and (2)

The inverse wavelet transform unit 240 of the layer 2 performs motion compensation on the motion compensated image C_S4 for the layer 2's S4 and the high frequency sub-images S4-1, S4-2, and S4-3 of the layer 2 Synthesizes the sub-images, and performs inverse wavelet transform on the synthesized image. The image obtained as a result of inverse wavelet transform is equal to the size of the S2 sub-image of the layer 1 and is called the synthesized S2 sub-image (R_S2). The synthesized S2 image is provided to the candidate vector motion estimator / compensator 120 of layer 1.

The candidate vector motion estimator / compensator 120 of the layer 1 performs a similar operation on the S2 image synthesized as in the operation performed by the candidate vector motion estimator / compensator 220 of the layer 2 to a P × 2 ^Mm, that is, a search of 4 × P And estimates a new motion vector having a minimum error rate among a plurality of candidate vectors using the block matching method with the current sub-image S2 'provided from the wavelet transform unit 110. [ The estimated motion vector is used to generate a motion compensated S2 sub-image C_S2 for the combined S2 image R_S2.

The motion estimation / compensation unit 130 of the layer 1 performs a motion compensation / compensation operation on the high-frequency sub-image (i. E. S2-1, S2-2, and S2-3 and a motion vector compensated new S2 sub-image C_S2 using the estimated motion vectors of the candidate vector motion estimator / compensator 120. [ At this time, the motion estimation / compensation on the high-frequency sub-image at the same level as the sub-image S2 can be implemented by the two methods described previously.

The inverse wavelet transform unit 140 of the layer 1 transforms the motion compensated image C_S2 for the layer-1 S2 and the motion-compensated sub-image for the layer-1 high-frequency sub-images S2-1, S2-2, And performs inverse wavelet transform on the synthesized image. The synthesized image is the same size as the original image. The inverse-wavelet-transformed image and the estimated motion vector of the combined image are provided to a candidate vector motion estimator 400, and motion estimation is performed on the reconstructed image to output a predicted motion vector. This motion vector is used as a motion vector of the original input image.

In another embodiment of the present invention, the motion estimation / compensation units 130, 230, and 330 of each layer perform motion estimation / compensation for the high-frequency sub-image of each layer by using a kind of control signal, May not be performed. When the motion estimation / compensation for the high-frequency sub-image is not performed, the high-frequency sub-image is directly supplied to the inverse wavelet transformers 140, 240 and 340, and the remaining operations are the same as those described above. If the motion estimation / compensation is not performed for the high frequency sub-image, the calculation amount may be reduced and the processing speed may be faster, but the accuracy may be lowered.

As described above, according to the present invention, it is possible to improve disadvantages such as a large amount of computation, a large number of bits, and accumulation of errors, which have been a problem in using a conventional motion estimation method for moving image compression using wavelet transform, Motion vector estimation is possible. Also, when the motion estimation and compensated image is compared with the conventional image using the present invention, it can be expected that the performance such as PSNR and DFD energy can be improved remarkably.

Claims (4)

A hierarchical motion estimation method in a wavelet transform domain, (a) wavelet-transforming each of the first original image and the second original image temporally adjacent to the first original image to generate a hierarchical multiresolution sub-image composed of one low-frequency sub-image and three high-frequency sub- Dividing; (b) generating a plurality of candidate vectors for each search block by dividing the low-frequency sub-image of the upper layer of the first original image and the low-frequency sub-image of the upper layer of the second original image into P-sized search blocks; (c) motion estimation using a plurality of candidate vectors for the low-frequency sub-image of the upper layer estimated in step (b) to generate a motion vector having a minimum distortion among the candidate vectors as a motion vector, ; (d) obtaining a synthesized sub-image of a lower layer by performing motion estimation, compensation processing, and inverse wavelet transform of a high-frequency sub-image of the same level as the motion-compensated low-frequency sub-image obtained in step (c); (e) generating a low frequency sub-image for estimating a motion vector in the step (b) using a reconstructed sub-image of the lower layer obtained in step (d) and a low- To the re-synthesized sub-image, and repeating the step (b) and the following steps until the re-synthesized sub-image has the same resolution as the original image. And a hierarchical motion estimation method using a candidate vector. 2. The method of claim 1, wherein in step (c), the candidate vector includes a motion vector for a sub-picture on an upper layer at the same position as a motion vector to be estimated and neighboring vectors around the motion vector. The candidate vector delivered to the synthesized sub-image of the layer is a scaled-by-2-fold scale of the compensated sub-vector of the upper layer; Estimating a motion vector of a synthesized sub-image using a plurality of candidate vectors, and estimating a vector having a minimum distortion among a plurality of candidate vectors as a motion vector for the synthesized sub-image, A method for estimating a hierarchical motion over a vector. The method of claim 1, wherein, in the step (d), the motion vector for motion estimation / compensation processing of the high-frequency sub-image of the layer is applied to the low- Estimates and applies a new motion vector in the neighboring area through a block matching method; In the case of the remaining layer, the motion vector estimated from the synthesized sub-picture of the layer provided from the upper layer is applied or the motion vector estimated from the synthesized sub-image of the layer is used as the motion vector, Wherein a new motion vector is estimated and applied through a block matching method in a wavelet transform domain. The method of claim 1 wherein searching the low-frequency sub-image is for the split, and wherein the movement of the compensation sub-picture P × 2 movement of ^Mm amplitude vector estimate as a search block for motion vector estimation in the P size, in the step (e) Into blocks; Wherein M represents the number of wavelet transform layers, and m represents a layer to which the sub-image belongs, in the wavelet transform domain.