KR100294895B1 - Motion compensation moving-picture compressing method using filter - Google Patents

Abstract

PURPOSE: A motion compensation moving-picture compressing method using a filter is provided to obtain excellent picture quality without increasing transmission rate of an encoder. CONSTITUTION: A motion compensation moving-picture compression is carried out in such a manner that motion vectors by blocks are calculated and the current image is estimated through the motion vector of the current frame and the image of the previous frame. To remove blocking effect in the moving picture compression, the motion vectors by blocks are transformed into pixels through one-time operation to calculate motion vectors by pixels. Then, the current image is estimated through the motion vectors by pixels and the image of the previous frame.

Description

Motion Compensation Video Compression Method Using Filter

The present invention relates to a motion compensation video compression method using a filter, by converting one motion vector per block into a pixel unit using a filter and removing a blocking phenomenon by using a motion vector through an optimization step. Image quality is achieved, and this technology is widely used in industries such as video compression systems and high definition television.

In general, video information compression techniques (eg, MPEG, H.261, H.263, etc.) use several methods to reduce the large amount of information in a video. Among them, predictive coding based on block motion compensation (hereinafter referred to as BMC) that reduces information by using correlation between screens is widely used.

This method is based on predicting the current screen using the previous screen and the motion vector based on the current screen showing a high correlation with the previous screen in most cases where there is no sudden change of the screen.

The motion vector is calculated by dividing the current image and the previous image by the block at the transmitter and finding a block with high correlation in the previous image using a method such as a block matching algorithm (hereinafter referred to as BMA). .

The calculated motion vector is used to predict the current image, and the difference between the predicted image and the current image, that is, an error image, is transmitted together with the motion vector.

The receiver predicts the current image in units of blocks by using the entire screen and the motion vector, and then restores the current image by combining the received error image.

Here, the motion vector is calculated in units of blocks, which is to roughly model the image motion, resulting in blocky artifacts in the reconstructed image.

This blocking phenomenon is a phenomenon in which the boundary between adjacent blocks is discontinuous in the reconstructed image, which serves as a major cause of the deterioration of the image quality in the reconstructed image.

In order to remove such blocking phenomenon, the conventional method is to increase the amount of information to be transmitted or to use a method such as overlapped BMC (OBMC) temporarily.

However, these methods also had a problem that can not be a fundamental way to remove the block phenomenon.

In the present invention, in order to solve the conventional problems as described above, one motion vector per block is used by using a filtered motion compensation (FMC) method without using the BMC method. It aims to provide improved image quality in decoding by fundamentally eliminating the cause of the blocking phenomenon by converting to units and using motion vectors through an optimization step.

1 is a block diagram of a video encoder to which the present invention is applied.

2 (a) is a view showing the original image of the eighteenth frame of the 'deadline' sequence.

FIG. 2 (b) is a diagram showing the result of compressing the image of FIG. 2 (a) using the BMC scheme. FIG.

FIG. 2 (c) is a diagram showing the result of compressing the image of FIG. 2 (a) using the OBMC method. FIG.

FIG. 2 (d) shows the result of compressing the image of FIG. 2 (a) by the FMC method according to the present invention. FIG.

Figure 3 (a) is a graph showing the PSNR results for each scheme in the 'deadline'.

Figure 3 (b) is a graph showing the PSNR results for each method in 'claire'.

Figure 3 (c) is a graph showing the PSNR results for each scheme in the 'salesman' sequence.

* Explanation of symbols for main parts of the drawings

1: digital part 2: motion estimation part

3: motion compensation unit 4: DCT / quantization unit

5: Inverse quantization / inverse DCT part 6: Variable length coding unit

7: buffer 8: rate control unit

A motion estimation process of calculating a motion vector in units of blocks to achieve the above object;

In the method for compressing a motion compensation video including a motion vector of the current frame input in the motion estimation process and a motion compensation process for predicting the current video through the image of the previous frame,

To remove a blocking phenomenon in the video compression;

A motion estimation process of converting the motion vector of the block unit into the pixel unit by one operation and calculating the motion vector of the pixel unit;

And a motion compensation process for predicting the current image based on the pixel-based motion vector and the image of the previous frame.

In addition, in order to achieve the above object, a motion estimation process for calculating a motion unit of the block unit;

In the method for compressing a motion compensation video including a motion vector of the current frame input in the motion estimation process and a motion compensation process for predicting the current video through the image of the previous frame,

To remove blocking effects in the video compression and to prevent degradation of the peak signal-to-noise ratio (PSNR);

A motion estimation process of converting a motion vector in a block unit into a pixel unit through an iterative operation and calculating a motion vector in a pixel unit;

And a motion compensation process for predicting the current image based on the pixel-based motion vector and the image of the previous frame.

The above objects, features, and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The basic principle of the present invention is as follows.

The blocking phenomenon in BMC is caused by using one motion vector per block. Therefore, in the case of synthesizing the prediction image by spatially smoothing the motion vector of the block unit by a method such as interpolation, the occurrence of blocking phenomenon is theoretically impossible.

Therefore, in the FMC of the present invention, the predictive image is synthesized after converting the motion vector in the unit of block into the unit of pixel. In this case, when the motion vector obtained from the BMC is used, the blocking phenomenon is eliminated, but the peak signal-to-noise-ratio (hereinafter referred to as PSNR) is reduced. Accordingly, the transmitter can obtain a higher quality reconstructed image only by calculating and transmitting a motion vector through an optimization process.

The motion vector through the optimization process is a vector that minimizes the norm of the error image or the “displaced frame difference (DFD)” image, and can be calculated through a procedure such as successive linearization. .

As a result of evaluating the FMC, the FMC method was able to completely remove the blocking phenomenon from the reconstructed image, and provided a higher PSNR value than the conventional method. As a result, FMC is ahead of many existing video compression standards (e.g., H.261, H.263, MPEG-1, MPEG-2, MPEG-4) and adopted as the next generation video compression standard. There is a big sense that there is a possibility.

The present invention aims to eliminate the blocking phenomenon and ultimately minimize the size of the DFD. Looking at the expression development as follows.

The optimized motion vector minimizes the size of the DFD, for example, EMSE (estimated mean square error). EMSE can be written as follows without considering unnecessary constants.

In Equation 1, r denotes a range of an image, r = (x, y) denotes pixel coordinates, and u (r) denotes a motion vector at position r. DFD is defined as follows.

In Equation 2, I _n (r) and I _n-1 (r + u (r)) mean current (n) and previous (n-1) frames, respectively. As can be seen from the equation, DFD is the difference between the current frame and the previous frame moved. The predicted frame, I _n-1 (r + u (r)), is synthesized using the previous frame, I _n-1 (r) and the corresponding motion vector, u (r).

The optimization of EMSE can be obtained by approximating numerical solution by nonlinear problem or linearization. To use continuous linearization, assume an optimized motion vector, a motion vector close to u (r), u ^* (r).

Since this error is defined as δ = u (r) -u * (r) and DFD is nonlinear with respect to u (r), Equation 2 is developed as follows by the Taylor expansion around u ^* (r). .

In Equation 3 above, ∇ is a gradient operator, Superscript T means the transpose of the matrix, and hot means the higher order term for δ. Using this equation (1) is written as follows.

If the above EMSE is discontinuous and aligned in lexicographical form, the vector form of the DFD matrix, b (mn × 1) and the improvement vector δ _p (2mn × 1), are expressed as follows.

In Equation 5, n and m are pixel unit sizes of the frame, n is vertical and m is horizontal. And δ _p is the motion consists of two components (δ _h , δ _v ) of the motion vector, and the subscripts (h, v) mean (horizontal and vertical) directions, respectively.

For example, the Common Intermediate Format (CIF) frame is (m, n) = (288,352), and is divided into 22 macro blocks and 18 macro blocks. By the definition of Equation 5, the EMSE is discontinuous as follows.

In Equation 6 above Discrete representation of, A (u) and discrete representation of DFD, b are both functions of u (r) as shown in Equation 6, and A (u) is (mn × 2mn) in size This is from the first term in Taylor's deployment.

δ _p is defined in all pixels of the image and is modeled as a linear transformation of a sparse motion vector field (for example, a vector field in macroblock units). In addition, when the lexical terms of the macroblock unit motion vector are expressed as δ _b , the two motion vector fields are connected by the following linear transformation.

In other words, H in Equation 7 denotes a linear transformation that maps a motion vector field sampled in units of macro blocks to a motion vector field sampled in units of pixels.

Using the definitions in equations (5) and (7), the discontinuous EMSE is written as

The optimization of Equation 8 above for all δ _B and H results in a decrease in EMSE. The problem of optimizing δ _b and H at the same time is a difficult nonlinear problem that can be solved easily by fixing H first.

For example, assuming that the motion vector field is band-limited, H may be used as a band-limited interpolation filter.

However, even if H is fixed, the optimization of EMSE for δ _b is still a difficult problem because the solution of EMSE is a solution of the nonlinear equation problem. However, if a close initial value for u (r) is available, the numerical solution can be obtained by the following iterative method.

The above algorithm aims to start with the initial vector u ^(O) (r) and converge to the optimal vector u (r) suitable for the two-dimensional filter H. The name of this algorithm will be called FMC (filter motion compensation).

In the algorithm FMC (Equation 9), iterate until the solution converges. To obtain the initial vector, u ^(O) (r), we use the existing BMA. In this case, it is assumed that the vector obtained by the BMA is close to the optimal vector and can converge quickly through iteration. In fact, convergence has been done three times or less, for example, in CIF images.

To implement / test the above algorithm, H used a Parks-McClellan equiripple finite impulse response (FIR) filter as an interpolation filter.

In order to calculate the matrix in Equation 9, we used a conjugate gradient algorithm with fast speed and good convergence in sparse matrix.

One optimized vector u (r) per macroblock obtained through the FMC scheme described above is coded along with the error image and transmitted to the decoder. The encoding process is complicated due to the large amount of computation for each iteration, but the decoding process is relatively simple. First, the decoder interpolates the optimized vector received using the same band-limited interpolation filter used in the encoder. The decoder then simply synthesizes the predicted image using the reconstructed motion vector in pixel units. The remaining operations are the same as the existing BMC operations.

An example of operating an actual video encoder using the above principle is as follows.

FIG. 1 is a block diagram of a video encoder to which the present invention is applied, and receives input video signals Y (luminance signal), Cr (color difference signal), and Cb (color difference signal) to receive each signal according to a synchronization signal (Sync). A digital unit 1 for digitizing;

A motion estimator (2) for calculating a pixel-by-pixel motion vector through interpolation / filtering from a block-by-motion vector of the digitalized image through the digital unit (1); A motion compensator (3) for predicting the current image from the previous frame image due to the frame delay and the motion vector of the current frame input from the motion estimator (2); A DCT / quantization unit 4 for aligning the difference between the predicted image and the current image for each frequency component through a DCT transformation (the lower frequency component represents a larger value), and then dividing the integer into a large integer to take only an integer part; An inverse quantization / inverse DCT unit 5 for restoring the original image compressed by the DCT / quantization unit 4 and outputting the original image compressed to the motion compensation unit 3; A short length code is assigned to a DCT coefficient or a motion vector output from the DCT / quantization unit 4, and a long code is assigned to a value having a low probability of occurrence, thereby reducing an average code length. A variable length encoding unit 6; A buffer (7) for controlling the flow so that no overflow or underflow occurs in the encoding as a kind of buffer for adjusting the overall bit rate of the transmitted information input from the variable length encoding unit 6; And a rate control section 8 for maintaining a constant data rate by adjusting the encoding according to the amount of information transmitted through the buffer 7 being large and small.

In this case, the amount of information to be transmitted in the motion estimation unit 2 does not increase, and the portion where the image is actually compressed is performed by the DCT / quantization unit 4.

Referring to the operation of the encoder as described above, the input video signal is cut into a certain unit (G0P: group of picture) so that the first frame of each unit is left as the original image and the remaining frames are compared with the first frame based on this frame. Output only the difference to reduce the amount of information to be sent.

In the above process, the core of the present invention is a technology applied to the motion estimator 2 and the motion compensator 3. Specifically, Equations 7 to 9 according to the present invention are the motion estimator 2. Equation 7 is embedded in the motion compensation unit 3 and used to estimate motion of the image. The equation 7 is used to perform motion compensation using the estimated value calculated by the motion estimation unit 2. Equations 1 to 6 are equations used to derive Equations 7 to 9. Using the above equation is to convert the motion vector of the existing block unit to the motion vector of the pixel unit by the introduction of digital filtering, to remove the blocking phenomenon and to enhance the video quality.

The result of the compression using the motion encoder to which the present invention is applied is as follows.

The test evaluated the FMC algorithm as CCITT's test sequences 'deadline', 'claire' and 'salesman'.

The sequence has a size of (352 × 288) in pixels and (16 × 16) in blocks. 21 frames were used for each of the three sequences. The removal of the blocking phenomenon was visually evaluated and compared with the conventional methods (BMC and OBMC), and the PSNR value was also compared with the conventional method.

2 (a) to 2 (d) is a diagram comparing the results of the present invention with the naked eye, Figure 2 (a) shows the original image of the 18th frame of the 'deadline' sequence.

FIG. 2 (b) shows the result of compressing the original image using the conventional BMC method, and FIG. 2 (c) shows the result of compressing the original image using the existing method OBMC. ) Is a view showing the result of compressing the original image by the FMC method according to the present invention.

As shown in each figure, block diagrams are observed in the hand and mouth in FIG. 2 (b) synthesized with BMC, and ghosting artifacts are observed near the finger in FIG. 2 (c) synthesized with OBMC.

However, in FIG. 2D synthesized with FMC, such a phenomenon of lowering the image quality did not occur.

On the other hand, the same evaluation was performed in the claire and salesman sequences. Similar results were obtained in claire and very few movements in salesman.

However, the following PSNR values of Figs. 3 (a) to 3 (c) show the superiority of FMC in all sequences.

FIG. 3 (a) shows the PSNR results for each scheme in 'deadline', and FIG. 3 (b) shows the PSNR results for each scheme in 'claire', and FIG. 3 (c) shows A diagram showing the PSNR result for each scheme in the 'salesman' sequence.

In each drawing, the BMC method is indicated by a dotted line, the OBMC method is indicated by a thick dotted line, and the FMC method of the present invention is indicated by a solid line.

As shown in the figure, the PSNR value of FMC was the highest overall, followed by OBMC and BMC.

As described in detail above, in the present invention, a motion predictor generated in a conventional video information compression technique uses a filter for one motion vector per block to remove a blockage phenomenon in which a boundary between adjacent blocks is discontinuous. By eliminating the blocking phenomenon by using the motion vector through the optimization step and the optimization step, it is effective to obtain better image quality in the predicted video than the conventional method without increasing the bit rate of the encoder, and to all video transmission / reception and storage systems. Application is possible.

In addition, since the FMC method of the present invention can also be processed in real time, by designing a codec that is an image compression / restorer by applying this technology, the advantage of implementing a competitive video decompressor by providing a high quality and at the same time real-time processing Entails.

In addition, preferred embodiments of the present invention are disclosed for the purpose of illustration, and various modifications, changes, additions, etc. will be possible to those skilled in the art within the spirit and scope of the present invention, such modifications and modifications belong to the following claims You will have to look.

Claims (6)

A motion estimation process of calculating a motion vector in block units; A method of compressing a motion compensation video including a motion vector of a current frame input in the motion estimation process and a motion compensation process for predicting a current video through an image of a previous frame, wherein the blocking phenomenon in the video compression is removed. In order to; A motion estimation process of converting the motion vector of the block unit into the pixel unit by one operation and calculating the motion vector of the pixel unit; And a motion compensation process of predicting the current image based on the pixel-based motion vector and the previous frame image. The motion compensation video compression method as claimed in claim 1, wherein the motion vector is converted by Equation 1 below. [Equation 1] In the above formula, H means any transformation (all transformation schemes), and δ _b means a vector that lexicographically creates a motion vector field in macroblock units. The method of claim 1, wherein a linear filter is applied when the motion vector in the block unit is converted into the pixel unit. [Equation 2] In the above formula, H means a linear transformation (matrix), and δ _b means a vector that sequentially creates a motion vector field in macroblock units. A motion estimation process of calculating a motion vector in block units; A method of compressing a motion compensated video including a motion vector of a current frame input in the motion estimation process and a motion compensation process for predicting a current video based on an image of a previous frame, wherein the blocking phenomenon in the video compression is eliminated. To prevent degradation of the peak signal-to-noise ratio (PSNR); A motion estimation process of converting a motion vector in a block unit into a pixel unit through an iterative operation and calculating a motion vector in a pixel unit; And a motion compensation process of predicting the current image based on the pixel-based motion vector and the previous frame image. The motion compensation video compression method as claimed in claim 4, wherein the motion vector is converted by Equation 3 below. [Equation 3] In the above formula, H means any transformation (all transformation schemes), and δ _b means a vector that sequentially creates a motion vector field in macroblock units. The motion compensation video compression method as claimed in claim 4, wherein a linear filter is applied when the motion vector in the block unit is converted into the pixel unit. [Equation 4] In the above formula, H means a linear transformation (matrix), and δ _b means a vector that sequentially creates a motion vector field in macroblock units.