KR100440567B1 - A method for forming binary plane for motion search and a motion estimating apparatus using the same - Google Patents

Abstract

The present invention relates to a multi-resolution motion estimation method using a binary plane that indicates whether a motion is searched in a wavelet transform region. After splitting an input image into various resolutions by wavelet transform, it is possible to use temporal redundancy between successive frames. Comprising a binary plane representing a motion region, and performing a multi-resolution motion estimation based on this.

According to the present invention, a large amount of computation and number of bits generated when using a conventional motion estimation method by adaptively distributing a motion search region according to a layer in consideration of energy distribution between bands, human visual characteristics, and motion search computation amount In addition, problems such as accumulation of motion search errors can be improved by the method proposed in the present invention, and thus the execution time can be improved without deterioration of image quality.

Description

Binary plane construction method for motion search and motion estimation device using the same {A METHOD FOR FORMING BINARY PLANE FOR MOTION SEARCH AND A MOTION ESTIMATING APPARATUS USING THE SAME}

The present invention relates to a binary plane configuration method for a motion search and a motion estimation device using the same. More particularly, the present invention is applied to fields such as signal and image processing, video compression, and the like. A binary plane construction method and a motion estimation apparatus using the same.

In general, a video has both spatial and temporal correlation. Image compression is the elimination of the correlation, i. In more detail, an image signal includes spectral correlation between color components, temporal correlation between frames and frames, and spatial correlation between pixels in a screen. Among these, since temporal correlation is the most important factor, a large amount of data can be compressed through encoding between successive frames.

The encoding method between frames includes a motion estimation process for estimating motion between successive frames using time correlation, a motion compensation process using estimated motion information, and a motion compensated difference encoding a difference between the motion compensated frame and the current frame. It consists of three parts of the video encoding process.

In video coding, a block matching algorithm (BMA) is generally used as a motion estimation method because the algorithm is simple and hardware implementation is easy. However, it is difficult to determine the appropriate search area and the size of the block with respect to the motion of the target image, and the partitioning phenomenon occurs because the motion is estimated by dividing the block into a predetermined size block.

In order to remove these drawbacks, methods for estimating the hierarchical motion of the split image after dividing the input image into multi resolution using wavelet transformation have been studied.

The wavelet transform, which has recently been widely applied in the fields of signal and image processing and video compression, can express various types of signals much more efficiently, and thus can improve the compression ratio. Therefore, the wavelet transform is based on Fourier transformation. Compared to the conventional signal processing algorithms, it is faster and can implement localization of signals efficiently in time domain and frequency domain.

The motion estimation method in wavelet transformed images known to date is representative of multi-resolution motion estimation (MRME). In this method, wavelet transforms a reference and a current frame and moves between subbands. Using the correlation of, the motion of the high resolution layer is predicted as a base band motion vector.

In addition, the block matching algorithm is executed by varying the size of the motion estimation block in proportion to the resolution in consideration of the energy concentration between the bands. Thus, an even motion vector is obtained between the blocks, which significantly reduces the blocking effect compared to the conventional motion estimation methods. have.

Hereinafter, a wavelet transform and a multi-resolution motion estimation method using the same according to the present invention will be described with reference to FIGS. 1 and 2.

FIG. 1 illustrates a hierarchical structure of a wavelet transformed image in which an original image S1 is decomposed to three layers of resolution and each layer is composed of a plurality of sub-images. Once the original image is wavelet-converted, it has a sub-image (S2) that is 1/4 the size of the original image and has a low frequency component of the original image, and has high frequency components in the horizontal, vertical, and diagonal directions of the original image. It is divided into three sub-pictures W12, W22, and W32 to form layer 1. When the sub-image S2 which is a low frequency component is wavelet-converted again, the sub-image S4 having the low-frequency component of the sub-image S2 and the high-frequency components in the horizontal, vertical and diagonal directions of the sub-image S2 The sub-pictures W14, W24, and W34 having the subfields are subdivided to form layer 2. When the sub-image S4 is further wavelet transformed, the sub-image S8 having the low frequency component of the sub-image S4 and the high-frequency components in the horizontal, vertical and diagonal directions of the sub-image S4 are included. It is divided into three sub-pictures W18, W28, and W38 to form layer 3. Here, these sub-images are correlated with each other among sub-images having the same directionality of each layer, and the sub-image S8 of the low frequency component in the lowest resolution region has the most energy, and the high frequency component toward the high resolution region The energy of the sub-image becomes smaller.

2 illustrates a multi-resolution motion estimation method. In the multi-resolution motion estimation method, the motion is estimated using the baseband image, that is, the motion vector V _s8 estimated from the sub-image S8 of the highest layer, as an initial motion vector of the sub-images of the remaining high-resolution layers. After the original image is wavelet transformed into M (M = 3 in FIG. 1) layers as shown in FIG. 1, each sub-image of the M resolution layer is divided into a unit block having a predetermined pixel size, Motion estimation is performed on each block of the belonging sub-image S8 by a block matching algorithm. The estimated motion vector V _s8 is transferred to a high frequency sub-image of the same layer and also to a lower layer. In addition, the size of the search block increases twice as the high resolution layer, that is, the lower layer, and the number of motion search blocks of each sub-picture is the same regardless of the resolution, and the motion vector of the high resolution layer is the baseband of the top layer. Using the scaled motion vector V _s8 of the image S8 by 2 times as the base vector, the micro-movement displacement? (X, y) is estimated in the vicinity of the motion space of the vector.

This multi-resolution motion estimation method requires a large amount of computation and a number of bits since a motion vector must be obtained from all sub-pictures and transmitted to the receiver, and a problem such as error accumulation in a lower layer when the initial motion estimation is wrong It implies That is, in the conventional multi-resolution motion estimation method, a method for reducing the amount of computation and the bit amount of the motion search has been requested.

Meanwhile, an apparatus and method for accurately detecting a motion vector by repeating a procedure of synthesizing a low-frequency image of a motion estimation higher layer with a wavelet transformed high-frequency component image of the same layer in a hierarchical motion estimation apparatus to produce a compensated sub-image This is a hierarchical motion estimation apparatus and method through the resynthesis of wavelet transformed sub-pictures. Korean Patent Publication No. 10-252347 (January 18, 2000) (hereinafter referred to as "prior patent 1") It is.

The preceding patent 1 includes a plurality of wavelet transform units, an equal number of hierarchical inverse wavelet transform units, and motion estimation / compensation units, each inverse wavelet transform unit being provided from a corresponding wavelet transform unit. High frequency sub-pictures and motion-compensated sub-pictures having the same size as the high-frequency sub-pictures, and reconstructed by inverse wavelet transforming the synthesized video, wherein each motion estimation / compensation unit The motion vector of the reconstructed image transmitted from the inverse wavelet transform unit is estimated from the low frequency sub-picture of the past image wavelet transform unit of the corresponding layer, and motion compensated to generate a motion compensated sub-image.

However, although the prior patent 1 has some effect on improving the accuracy of motion vector extraction, it is not related to reducing the calculation amount of motion search.

In addition, a motion estimation apparatus having a variable block size in a wavelet transform region of a two-dimensional image to reduce the amount and calculation amount of motion search information by varying the size of the motion search block in the wavelet transform region is disclosed in Korean Patent Publication No. 10- 156958 (July 27, 1998) (hereinafter referred to as "prior patent 2").

In order to reduce the amount of motion search calculation amount and information, the prior patent 2 uses a method of dividing a sub-picture into large blocks, and repeatedly dividing the blocks according to prediction error values of each block to vary the size of the motion search block. have.

However, the method according to the prior patent 2 performs a motion search for all areas of the sub-picture, and only changes the size of the block, so that all layers have the same motion search block. Therefore, there is a problem that the efficiency of the motion search is not high because the motion search is performed for all regions of the sub-image belonging to each layer.

In addition, the block size is determined variably using the wavelet transform and the properties of the wavelet coefficients, and the computation time is reduced by performing motion estimation compensation using only the lowest four frequency bands. "Effective Motion Estimation / Compensation Method and System in Video Compression / Restoration System" for Increasing the number of Korean Patent Publication No. 10-0223647 (July 10, 1999) (hereinafter referred to as "prior patent 3") Known.

The prior patent 3 is a method of searching for a motion by changing the size of a motion search block to a rectangle or a large size according to a sub-picture. Therefore, similar to the above Patent 2, the motion search is performed for all regions of the sub-picture, and only the size of the block is changed. Accordingly, even in the above-described Patent 3, there is a problem that the efficiency of the motion search is not high because the motion search is performed for all regions of the sub-image belonging to each layer. In addition, in the prior patent 3, by using the motion vector used for the sub-pictures of the upper layer as it is in the lower layer, there is a problem that may accumulate errors in the lower layer.

The present invention is to solve the above problems, a binary plane for determining whether to search for motion using a wavelet transformed current input image and the reference image of the previous frame is configured, the binary plane is a higher layer image By searching for motion in the lower layer image only in the region of motion in, the motion search area is adaptively distributed according to the input image layer to reduce the amount of computation and the number of bits, and the motion search error is accumulated. An object of the present invention is to provide a motion estimation apparatus capable of solving the problem.

1 illustrates a hierarchical pyramid of a wavelet transformed image.

2 is a diagram illustrating a general multi-resolution motion estimation method.

3 is a block diagram of a motion estimation apparatus according to the present invention;

Figure 4 is a flow chart of the binary plane configuration method according to the present invention.

5 illustrates an interlayer transition in a binary plane according to the present invention.

Explanation of symbols on the main parts of the drawings

100: wavelet transform unit 200: binary plane configuration unit

300: motion search unit 400: predictive image synthesis unit

500: frame storage unit 600: reference image synthesis unit

700 subtractor

According to an aspect of the present invention, there is provided a motion estimation apparatus, including: a wavelet converter configured to convert an input image of a current frame into a sub-image having a hierarchical structure having a multi-resolution; A frame storage unit which stores an image of a previous frame as a reference image; The absolute average difference between the baseband image of the wavelet transformed current frame and the co-located block of the corresponding image of the previous frame is obtained, compared with a predetermined threshold value, and the binary plane is determined by setting a potential motion region according to the result. A binary planar configuration constituting part; A motion search unit which receives an image of the wavelet transformed current frame, an image of the previous frame, and the binary plane, and searches for motion of each layer of the current frame image by the binary plane to obtain a motion vector; A prediction image synthesizer configured to obtain a prediction image using the motion vector and the binary plane; And synthesizing a reference image using the difference image between the wavelet-converted current frame image and the predicted image and the predicted image, and providing the synthesized reference image as a reference image of a previous frame. It is characterized by including a wealth.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings.

First, FIG. 3 illustrates a motion estimation apparatus according to an embodiment of the present invention. As shown in FIG. 3, the motion estimation apparatus of the present invention includes a wavelet converter 100 that receives an input image; A frame storage unit 500 which stores an image of a previous frame as a reference image; A binary plane configuration unit 200 which receives an output of the wavelet converter 100 and the frame storage unit 500 and outputs a binary plane; A motion search unit 300 which receives the output of the wavelet transform unit 100, the binary plane configuration unit 200, and the frame storage unit 500 and outputs a motion vector; A predictive image synthesizer 400 which receives the outputs of the binary plane configuration unit 200, the motion search unit 300, and the frame storage unit 500 and outputs a predicted image; A subtractor 700 which receives the output of the wavelet converter 100 and the predictive image synthesizer 400 and outputs a difference image between the wavelet converter 100 and the predicted image synthesizer 400; The reference image synthesizer 600 receives the outputs of the prediction image synthesizer 400 and the subtractor 700, synthesizes a reference image, and outputs the reference image to the frame storage 500.

When the operation is started, a video signal constituting a current frame, which is a target of motion estimation, is input to the wavelet converter 100 as an input video. The wavelet transform has been described above with reference to FIG. 1, and the wavelet transform unit 100 converts the input image into sub-images having a hierarchical structure having a multi-resolution as shown in FIG. 1. These sub-images have a large energy in the sub-image having a low frequency component, and the energy decreases toward the sub-image having a high frequency component. Sub-images having high frequency components of each layer are composed of sub-images having horizontal, vertical, and diagonal components, and these sub-images have deep correlations with sub-images having the same directionality of different layers. In addition, if the mean absolute difference (MAD) between the same position block is smaller than a predetermined threshold value in the sub-image having the low frequency component of the current frame and the low-frequency component of the previous frame, the entire image. In the other sub-images of the decomposed hierarchical structure, the absolute mean difference corresponding to the same direction and the same position does not have a significant meaning. That is, in video encoding, many regions coincide with each other between two consecutive frames, and according to the present invention, a binary plane for determining whether there is motion is formed, and the motion is searched for only the region where there is motion. Contribute.

The wavelet-converted images are input to the binary planar component 200 and the motion searcher 300, respectively. Meanwhile, the frame storage unit 500 stores the image of the previous frame as a reference image. The binary plane configuration unit 200 configures a binary plane for determining whether to search for motion using the wavelet transformed image of the current frame and the reference image of the previous frame stored in the frame storage unit 500.

The binary plane configuration unit 200 first configures a baseband binary plane. The baseband binary plane is a binary plane for a sub-image having a low frequency component of the uppermost layer, and is represented by M8 (x, y) in the present invention. The binary plane is set to " 1 " or " 0 " depending on potential movement in any position or block represented by plane coordinates x, y. More specifically, the absolute average difference between the same position block of the sub-image S8 having the highest hierarchical low frequency component of the current frame and the corresponding sub-picture of the previous frame is calculated, and if the absolute average difference is larger than the threshold, the corresponding position is potential. By judging by the motion region, the baseband binary plane is constructed. The binary plane for the sub-images of the other layers except for the baseband sub-picture is based on the transition of the binary plane M8 in the baseband, and is determined as a potential motion region in the binary plane M8. With respect to the blocks only, the absolute average difference between the current frame and the corresponding sub-picture Wij (x, y) of the reference frame is compared with a predetermined threshold value, and it is configured to determine whether to search for motion based on the comparison result. That is, it is determined whether the binary planes of other sub-images except the baseband sub-image S8 and the sub-layers of the remaining layers are potential motion regions only for blocks determined to be potential motion regions in the binary plane of the base-band sub-image. Since the motion search, which will be described later, is based on the binary plane configured as described above, it can be seen that the calculation amount for the motion search is greatly reduced by the present invention.

Next, the motion search unit 300 includes an image of the current frame output from the wavelet converter 100, an image of a previous frame stored as a reference image in the frame storage unit 500, and the binary plane component ( A binary plane configured at 200 is input, and a motion vector is obtained by searching for the motion of each layer of the current frame image by the binary plane. In this case, the motion search unit 300 obtains a motion vector for a potential motion region of the baseband, and a motion vector for a motion vector of another layer only for a potential motion region of the corresponding binary plane. That is, in any lower layer, the baseband motion vector is scaled only twice for the potential motion region of the corresponding binary plane, and then used as the base vector to estimate the micro motion displacement near the motion space of the vector. Find the motion vector.

The prediction image synthesizer 400 obtains a prediction image by using the motion vector and the binary plane, and the prediction image is input to the subtractor 700 and wavelet-converted image of the current frame by the subtractor 700; Difference image is obtained. The reference image synthesizer 600 synthesizes a reference image by using the difference image and the prediction image. The reference image is stored in the frame storage unit 500 as a synthesized reference image, and an image of a next frame is input. Is provided as a reference image of the immediately preceding frame.

Next, the configuration method of the binary plane will be described in more detail with reference to FIGS. 4 and 5.

Referring to FIG. 4, in the method of configuring a binary plane, a process of constructing a baseband binary plane is largely performed (10), and a subpicture having a high frequency region of the same layer as a binary plane obtained from the baseband and a subpicture of another layer. A process of transitioning to a binary plane (Mij (x, y)) (20), a process of reconstructing a binary plane for each sub-image (30), and a process of determining whether a binary plane is configured for all sub-images (40) )

When the operation starts, the variables (i, j) used are initialized, and the absolute mean difference (MAD S8 (x, y)) between the baseband image of the current frame and the same position block of the reference image of the previous frame is calculated ( 11). Next, it is determined whether the obtained absolute average difference MAD S8 (x, y) is larger than a predetermined threshold value TH1 (12). In step 12, if the absolute mean difference MAD S8 (x, y) is greater than the predetermined threshold value TH1, it is determined that it is a potential motion area and the binary plane M8 (x, y)) is set to "1" (13). If the absolute mean difference MAD S8 (x, y) in step 12 is less than the predetermined threshold value TH1, it is determined that the potential average area is not the potential motion area, and the binary plane M8 (for the block at that position) is determined. x, y)) is set to " 0 " (14). The above process 10 is performed for all blocks of the baseband image to form a binary plane M8 (x, y) for the baseband image.

Next, the binary plane M8 (x, y) for the baseband image obtained as described above is the binary plane Mij (x, y) for the sub-image having the high frequency region of the highest layer and the sub-image of the other lower layer. (20). Where i represents the resolution hierarchy and j represents the horizontal, vertical and diagonal directions. FIG. 5 illustrates a process in which the binary plane M8 (x, y) for the baseband image is transferred in horizontal, vertical and diagonal directions.

Next, it is determined whether there is a latent motion region in the binary plane (Mij (x, y)) of the hierarchy indicated by the variables (i, j) (31). That is, if the corresponding binary plane Mij (x, y) is set to "1", the potential motion region exists in the binary plane Mij (x, y). If the corresponding binary plane Mij (x, y) is set to "1" in step 31, the binary plane Mij (x) is used to reconstruct the corresponding binary plane Mij (x, y). and y)), the absolute mean difference (MAD Wij (x, y)) with respect to the sub-image wij (x, y) of the layer to which it belongs is calculated (32). If the corresponding binary plane (Mij (x, y)) is set to " 0 " in step 32, it can be used as it is without reconstruction of the binary plane, and jumps to step 41. Next, it is determined whether the obtained absolute mean difference MAD Wij (x, y) is smaller than a predetermined threshold value TH2 (33). In step 33, the absolute mean difference MAD Wij (x, If y)) is smaller than the threshold value TH2, the corresponding binary plane Mij (x, y) is determined to be not a potential motion area and is set to " 0 " (34). If the absolute average difference MAD Wij (x, y) is larger than the threshold TH2, it is correct that the corresponding binary plane Mij (x, y) is set to the latent motion region, and jumps to step 41. That is, in step 30, the binary plane Mij (x, y) for the remaining sub-images other than the baseband is reconstructed, and the binary plane Mij (x, y) is set to “1”. That is, only the case where there is a latent motion region in any block of the corresponding binary plane (Mij (x, y)), the absolute mean difference is calculated and compared with the threshold to reconstruct the binary plane of the corresponding block. This is based on the temporal correlation of video compression, that is, between successive frames, that sub-pictures having the same directionality among the sub-pictures of each layer of the wavelet transform have a deep correlation and so-called redundancy.

Next, it is determined whether the reconstruction process of the binary plane has been performed for all the sub-images (41). Otherwise, the variables (i, j) are sequentially increased, and then jumps to just before step 20, the above process (20). , 30), and if the reconstruction of the binary plane has been performed for all the sub-images in step 41, the binary plane construction operation is terminated.

It will be apparent to those skilled in the art that the binary planar construction method described above may be embodied as a computer program or the like and sold or used in a form stored in a recording medium such as a CD-ROM.

As described above, according to the present invention, in constructing a binary plane for determining whether to search for motion in a multi-resolution motion estimation method using wavelet transform, the consecutive frames are the same among the sub-images of each layer of the wavelet transform. Using the property that the directional sub-images have redundancy with each other, after constructing a binary plane for the baseband image, the binary plane of the lower layer sub-image is reconstructed only for the potential motion region in the binary plane of the baseband image. Then, the motion search calculation amount can be effectively reduced by performing the motion search using this binary plane. In addition, problems such as a large amount of computation, a number of bits, and a cumulative motion search error that occur when using the conventional method can be improved, and thus the execution time can be improved without deterioration of image quality.

Claims (8)

A wavelet converting unit converting the input image of the current frame into a sub-image having a hierarchical structure of multiple resolutions; A frame storage unit which stores an image of a previous frame as a reference image; The absolute average difference between the baseband image of the wavelet transformed current frame and the co-located block of the corresponding image of the previous frame is obtained, compared with a predetermined threshold value, and the binary plane is determined by setting a potential motion region according to the result. A binary planar configuration constituting part; A motion search unit which receives an image of the wavelet transformed current frame, an image of the previous frame, and the binary plane, and searches for motion of each layer of the current frame image by the binary plane to obtain a motion vector; A prediction image synthesizer configured to obtain a prediction image using the motion vector and the binary plane; A subtractor for obtaining a difference image between the image of the wavelet transformed current frame and the prediction image; And And a reference image synthesizer configured to synthesize a reference image by using the difference image and the predicted image, and to provide the synthesized reference image as a reference image of a previous frame. The method of claim 1, The wavelet transformed image of the current frame includes a baseband image having a low frequency component of a top layer, a subpicture having a high frequency component of a top layer, and a subpicture having a low frequency component and a high frequency component of a remaining lower layer, respectively. Motion estimation apparatus. The method according to claim 1 or 2, The binary planar component A binary plane for the baseband image of the wavelet transformed current frame is configured, and a binary plane for the remaining sub-images transitions the binary plane for the baseband image, and then a potential motion region is defined in the transitioned binary plane. Motion reconstruction apparatus characterized in that the reconstruction only for the case. delete The absolute average difference between the baseband image of the wavelet transformed current frame and the co-located block of the corresponding image of the previous frame is obtained, compared with a predetermined threshold value, and then the binary plane is set by setting whether or not it is a potential motion region. A first process of constructing; A second process of transitioning a binary plane obtained in the baseband to a binary plane for a sub-image having a high frequency region of the same layer and a sub-image for another layer, Reconstructing a binary plane only for the case where there is a latent motion region in the binary plane for each sub-image; And Determining whether the binary plane is configured for all the sub-images, and if the binary plane for all the sub-images is not configured, a motion search comprising a fourth process for repeating the second and third processes above Binary Plane Configuration Method The method of claim 5, The first process described above Calculating an absolute mean difference between the baseband image of the wavelet transformed current frame and the co-located block of the corresponding image of the previous frame; A second step of comparing the obtained absolute average difference with a predetermined threshold value; According to the comparison result, if the absolute average difference is greater than a predetermined threshold value, and set as a potential motion area, if the absolute average difference is less than a predetermined threshold value is set to a third step of setting the non-potential motion area Binary plane configuration method for the movement search, characterized in that. The method of claim 5, The third process described above A fourth step of determining whether there is a latent motion region in each binary plane transitioned in the second process; A fifth step of calculating an absolute mean difference between the current frame and the immediately preceding frame of the sub-picture, if there is a latent motion region in the fourth step; A sixth step of determining whether the absolute average difference obtained in the fifth step is greater than a predetermined threshold value; In the sixth step, if the absolute mean difference is smaller than a predetermined threshold, the potential moving area is set not to exist in the corresponding binary plane. If the absolute mean difference is larger than the predetermined threshold, the set value of the corresponding binary plane is set. Method of constructing a binary plane for the motion search, characterized in that comprises a seventh step to use as is. The absolute average difference between the baseband image of the wavelet transformed current frame and the co-located block of the corresponding image of the previous frame is obtained, compared with a predetermined threshold value, and then the binary plane is set by setting whether or not it is a potential motion region. A first process of constructing; A second process of transitioning a binary plane obtained in the baseband to a binary plane for a sub-image having a high frequency region of the same layer and a sub-image for another layer, Reconstructing a binary plane only for the case where there is a latent motion region in the binary plane for each sub-image; And It is determined whether a binary plane has been configured for all sub-images, and if a binary plane for all sub-images is not configured, a binary plane for motion search including a fourth process for repeating the second and third processes above. Recording medium in which the configuration method is stored in the form of a computer program.