KR100870554B1 - Motion compensated temporal filtering method for efficient wavelet-based scalable video coding and record-medium for executing method thereof - Google Patents

Abstract

Disclosed are a motion compensation temporal filtering method for wavelet based efficient scalable video encoding, and a recording medium on which a program for executing the method is recorded. A motion compensation temporal filtering method for wavelet based scalable video encoding includes determining a search candidate for search points in a reference frame for calculating a motion vector of an arbitrary current block in a current frame using PMVFAST; Computing a block matching error (SAD) between the current block and the comparison block of the reference frame by the search points according to the search candidates, searching for a point having the least SAD among the search points and selecting the optimal point. step; And calculating a motion vector of the current block by using the found optimal point. According to the present invention, the number of multiple connection pixels can be minimized by flattening the motion vector field so that the motion vectors do not overlap each other.

Motion vector, wavelet, PMVFAST, variable size block, motion compensation

Description

Motion compensated temporal filtering method for efficient scalable video coding based on wavelet and recording medium recording a program for executing the method for efficient wavelet-based scalable video coding and record-medium for executing method }

1 is a diagram for explaining conventional wavelet based scalable video coding.

FIG. 2 is a diagram illustrating a motion vector setting for motion estimation between a reference frame and a non-connection pixel in a reference frame and a current frame by MCTF (Motion Compensated Temporal Filtering) according to the related art. FIG.

3 is a functional block diagram showing a schematic configuration of a codec for performing a wavelet-based MCTF according to each embodiment of the present invention.

4 is a diagram illustrating an example of a reference frame and a current frame according to a motion compensation temporal filtering method using a predictive motion vector field adaptive search technique (PMVFAST) and a variable size block according to an embodiment of the present invention.

5 is a diagram illustrating a motion vector calculation method according to a conventional global search method.

6 is a diagram illustrating a motion vector calculation method using PMVFAST according to an embodiment of the present invention.

7 is an exemplary view showing an example of a variable size block having a variable size in accordance with an embodiment of the present invention.

8 is a flowchart illustrating a motion compensation temporal filtering process for efficient scalable video encoding based on wavelet according to an embodiment of the present invention.

The present invention relates to motion estimation, and more particularly, to a motion compensation temporal filtering method for wavelet based efficient scalable video encoding, and a recording medium on which a program for executing the method is recorded.

As information and communication technology including the Internet is developed, not only text and voice but also video communication are increasing. Conventional text-based communication methods are not enough to satisfy various needs of consumers, and accordingly, multimedia services that can accommodate various types of information such as text, video, and music are increasing. The multimedia data has a huge amount and requires a large storage medium and a wide bandwidth in transmission. For example, a 24-bit true-color image with a resolution of 640 * 480 would require a capacity of 640 * 480 * 24 bits per frame, or about 7.37 Mbits of data. When transmitting it at 30 frames per second, a bandwidth of 221 Mbit / sec is required, and about 1200 G bits of storage space is required to store a 90-minute movie.

Therefore, in order to transmit multimedia data including text, video, and audio, it is essential to use a compression coding technique. The basic principle of compressing data is the process of eliminating redundancy. Spatial overlap, such as the same color or object repeating in an image, temporal overlap, such as when there is almost no change in adjacent frames in a movie frame, or the same note over and over in audio, or high frequency of human vision and perception Data can be compressed by removing the psychological duplication taking into account the insensitive to. In general, intraframe compression is used to remove spatial redundancy and interframe compression is used to remove temporal redundancy.

Conventional video coding, such as MPEG-1, MPEG-2, H.263 or H.264, removes temporal redundancy by motion compensation and spatial redundancy by transform coding based on motion compensation prediction. These methods have good compression ratios, but the recursive approach in the main algorithm does not provide the flexibility for true scalable bitstreams. Accordingly, research on wavelet-based scalable video coding has been actively conducted in recent years. Scalable video coding means video coding with scalability. Scalability refers to a feature capable of partially decoding from one compressed bitstream, that is, playing various videos. Scalability means spatial scalability, which means that the resolution of the video can be adjusted, and signal-to-noise ratio (SNR), which means that the quality of the video can be adjusted, and temporal scale that can adjust the frame rate. It is a concept including the capability and the combination of each of them.

Among many techniques used for wavelet-based scalable video coding, Motion Compensated Temporal Filtering (hereinafter referred to as MCTF) is a key technique for removing temporal redundancy and temporally flexible scalable video coding. In the MCTF, coding is performed in units of group of pictures (GOP), and the pair of the current frame and the reference frame is temporally filtered in the direction of movement.

1 is a diagram for describing conventional wavelet based scalable video coding, and FIG. 2 is a diagram illustrating a motion vector setting for motion estimation between a reference frame and an unconnected pixel in a current frame by an MCTF according to the related art. The illustrated figure.

First, the MCTF will be briefly described with reference to FIG. 2, and the video sequence is decomposed according to a plurality of time levels. According to this temporal level, temporal scalability can be implemented. Although not shown in the figure, at time level 1, the video sequence is decomposed into low frequency frames and high frequency frames. First, a high frequency frame is generated by performing temporal prediction on three adjacent input frames. At this time, the temporal prediction uses both forward prediction and backward prediction. Next, the low frequency frame is generated by temporally updating the input frame using the generated high frequency frame. In this case, the input frame is updated using two adjacent high frequency frames. At temporal level 2, temporal prediction and temporal update are performed again from the updated low frequency frame. By repeating the decomposition process according to the plurality of time levels, one low frequency frame and one high frequency frame at the highest time level can be obtained.

The encoder stage sends one low frequency frame and 15 generated high frequency frames to the decoder stage that exist at the highest temporal level, and the decoder stage that receives it can obtain 16 decoded frames as a whole by reversing the time level. .

The connection pixel refers to a corresponding pixel when any pixel of the current frame is connected to the pixel of the reference frame, and the non-connection pixel refers to a pixel that is not connected between the current frame and the reference frame. Of course, as the number of non-connected pixels increases, various problems may occur due to deterioration of image quality.

Referring to FIG. 1, a problem of a conventional MCTF caused by generation of a plurality of unconnected pixels is as follows. According to the global search method used in the conventional motion compensation temporal filtering, a block whose minimum block matching error value (SAD) with the current block is minimum is searched by examining all search areas in the reference frame. In this case, some or all of the blocks connected by other blocks in the reference frame are selected as the blocks having the smallest SAD, so that duplication of arbitrary pixels may occur. Such overlapping selected pixels are called multiple connection pixels, and among the multiple connections to one pixel of the reference frame, only pixels of the current frame having a minimum Displayed Frame Difference (DFD) between the reference frame and the corresponding pixel of the current frame This connection is maintained and all remaining multiple connections are treated as unconnected pixels. At this time, the pixels of the other frame in the reference frame may remain as unconnected pixels without being connected at all, and the pixels of the current frame with multiple disconnects are excluded without the possibility of being connected with these unconnected pixels. Are all treated as unconnected pixels. As a result, it can be seen that the more unconnected pixels are generated as the number of multiple connected pixels connected in duplicate.

Therefore, according to the prior art, a plurality of unconnected pixels may be generated, which may cause various problems due to deterioration of image quality.

Accordingly, the present invention has been made to solve the above-described problem, and the motion compensation temporal for efficient scalable video coding that can minimize the number of multiple-connected pixels by flattening the motion vector field so that the motion vectors do not overlap each other. A filtering method and a program for executing the method are provided to provide a recording medium.

In addition, the present invention provides a motion compensation temporal filtering method and a method for performing the method for efficient scalable video coding that can increase the accuracy of motion compensation temporal filtering by using a variable size block for motion prediction along with the smoothing of the motion vector field To provide a recording medium on which the program is recorded.

In addition, according to the present invention, as a result of efficient motion-compensated temporal filtering, it is possible to encode an image having a smaller bit rate while maintaining the same image quality, and improve the performance of motion-compensated temporal filtering by using a variable sized block to increase the accuracy of motion prediction. This is to improve the subjective image quality of the image after performing filtering.

Other objects of the present invention will become more apparent through the preferred embodiments described below.

According to an aspect of the present invention for achieving the above object, in the motion compensation temporal filtering method for wavelet-based scalable video encoding, (a) a reference for calculating a motion vector of any current block in the current frame Determining search candidates for search points in the frame by using Predictive Motion Vector Field Adaptive Search Technique (PMVFAST); (b) calculating a Sum Absolute Difference (SAD) between the current block and the comparison block of the reference frame by the search points according to the search candidates to determine a point having the minimum SAD among the search points. Searching and selecting as an optimal point; And (c) calculating a motion vector of the current block by using the searched optimal point, and a recording medium on which a motion compensation temporal filtering method and a program for performing the method are recorded.

Here, the current block may be set to a plurality of blocks having different sizes, and the step (b) searches for a point having a minimum SAD for each of the plurality of current blocks, and SAD of each found point. Can be compared with each other to select the point having the smallest SAD as the optimum point.

The current block may be set to four main blocks having sizes of 32 * 32, 32 * 36, 16 * 32, and 16 * 16.

Further, if the point searched by the main block having the size of 16 * 16 among the available size of the current block has the minimum SAD, the step (b) is 16 * 16, 16 * 8, 8 * 16. , 4 subblocks having a size of 8 * 8 may search for a point having a minimum SAD for the selection of the optimal point.

In addition, the current block may be set to seven blocks having sizes of 32 * 32, 32 * 36, 16 * 32, 16 * 16, 16 * 8, 8 * 16, and 8 * 8.

In addition, the order in which the main blocks are used for searching may be set to correspond to the number of search candidates.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, and in describing the present invention with reference to the accompanying drawings, the same or corresponding elements are denoted by the same reference numerals regardless of the reference numerals, and duplicates thereof. The description will be omitted.

3 is a functional block diagram showing a schematic configuration of a codec for performing a wavelet-based MCTF according to each embodiment of the present invention.

Referring to FIG. 3, the codec according to the present embodiment includes a motion processor 310, a spatial analyzer 320, an entropy coding unit 330, a packetizer 340, and a motion vector that perform MCTF and motion estimation. The field coding unit 350 is included. Of course, it will be apparent to those skilled in the art that each component of the codec does not necessarily need to be implemented in hardware, but may be implemented in software such as a program.

To describe the schematic functions of each component of the codec, the MCTF 312 of the motion processor 310 estimates motion, sets the two estimated blocks as the connection block, and performs pixel-to-pixel filtering between the two blocks. .

When the motion estimation unit 314 performs motion estimation in the MCTF 312, the motion estimation unit 314 performs a function of pairing the corresponding two blocks of the reference frame and the current frame.

When the spatial scalability is implemented through the MCTF 312, the spatial analysis unit 320 spatially analyzes each temporal level (desired level) wavelet filter. ) To reduce the spatial resolution.

The entropy coder 330 adjusts temporal resolution and spatial resolution and then performs entropy coding according to the distribution of each pixel value. Due to the nature of entropy coding, the more dense the distribution is in one value (the smaller the variance is), the higher the coding efficiency. Temporal low frequency frames are difficult to reduce variance as desired since the original image appears, but since temporal high frequency frames are similar in color (originally 0), that is, close to black and similar to the difference between two frames, positive and negative values around zero Are distributed (for example, {0, -1,1,0,2, -1,0,0,0,0, -1,2,1,..., Etc.)).

Therefore, if the motion estimation is optimized, the variance can be further lowered, and the lower variance can increase the efficiency of entropy coding. This means that higher image quality can be obtained with the same number of bits. It also has the same meaning as using fewer bits in the same picture quality. (A typical example of entropy coding is Huffman Coding)

The motion vector field coding unit 350 encodes (encodes) a pixel value through motion estimation, so that the motion vector (MV: motion vector) used for motion estimation is also encoded and included in the coded bit string. Function.

The packetizer 340 functions to encode each pixel value of an image, that is, encodes a motion vector as described above. In addition, the packetizer unit 340 collects all the pieces of additional information such as a mode used for the encoding, and performs a packetization operation for transmission or storage.

Although the function of each component of the codec has been briefly described, each function of the codec component of the wavelet-based MCTF will be apparent to those skilled in the art, and thus a detailed description thereof will be omitted.

In particular, the present invention relates to the function of the motion processor 310 among the components of the codec described above.

Hereinafter, the processing method according to the motion prediction in the motion processing unit 310 according to the present embodiment will be described.

4 is a diagram illustrating an example of a reference frame and a current frame according to a motion compensation temporal filtering method using a predictive motion vector field adaptive search technique (PMVFAST) and a variable size block according to an embodiment of the present invention, and FIG. 6 is a diagram illustrating a motion vector calculation method according to a global search method, and FIG. 6 is a diagram illustrating a motion vector calculation method using PMVFAST according to an embodiment of the present invention.

Referring to FIG. 4, according to the present embodiment, by using PMVFAST, which is one of algorithms used as a fast motion prediction method in a DCT (Discrete Cosine Transform) based video encoder such as H.264 and MPEG-4, It can be seen that the motion vectors of the current frame are more flattened compared to FIG. 1 of the technique.

PMVFAST uses motion vectors of blocks having temporal and spatial correlation, that is, motion vectors of blocks at the same position in a reference frame and motion vectors of blocks in a neighbor of the current block in the current frame. Locality can be saved by determining and searching for candidates.

Referring to FIG. 5, the conventional global search method has more search ranges for searching for a search point having a minimum SAD by using the corresponding block of the reference frame 510 as an origin, and thus, each of the current frames 520. The motion vector for the blocks is likely to overlap.

In contrast, referring to FIG. 6, the motion vector field is further flattened by determining a search candidate in the reference frame 610 using PMVFAST according to the present embodiment, and determining a motion vector by searching for a minimum SAD for only the search candidate. You can.

 Therefore, according to the present embodiment, more smooth motion vectors can be calculated as compared with the conventional global search method.

Referring back to FIG. 4, according to the present embodiment, it is possible to have fewer multiple connection pixels by using a variable size block. In other words, motion prediction using blocks of smaller size is performed in large or complex motions, and motion prediction using blocks of larger size in motionless or flat parts is performed to increase the accuracy of motion prediction, thereby temporal filtering. The pixel connectivity of the city can be improved.

7 is an exemplary diagram illustrating an example of a variable size block having a variable size according to an embodiment of the present invention.

Referring to FIG. 7, in performing the prediction, 4 * main block modes of 32 * 32, 32 * 36, 16 * 32, and 16 * 16 and 16 * 16 are set to 32 * 32 as a basic macroblock. It can be divided into four subblock modes: 16 * 16, 16 * 8, 8 * 16, and 8 * 8. Of course, each size of the main block mode and the sub block mode is only one embodiment, it is natural that it can be set differently and applied differently. In the case described above, a basic macroblock larger than 16 * 16, which is a basic macroblock size used in a conventional DCT-based hybrid encoding method, is used. Therefore, in the case where there is almost no movement such as an image or a background having a large real spatial resolution, the effect of merging existing macroblocks is merged (that is, combining two blocks of 16 * 16 size). In small images or motions or complex parts, there is an effect of dividing existing macroblocks (i.e., a 16 * 16 block is divided by a plurality of subblock modes).

That is, according to the present embodiment, since a motion vector is almost absent in a portion having little or no motion, a block of a larger size is used, and a block of a smaller size is used in a place where a larger motion vector exists due to a lot of motion. Detailed motion estimation is possible.

8 is a flowchart illustrating a motion compensation temporal filtering process for efficient scalable video encoding based on wavelet according to an embodiment of the present invention.

In describing the motion compensation temporal filtering process according to the present embodiment with reference to FIG. 8, the performing agent may be the codec shown in FIG. 1, and the following description will be omitted for convenience of description.

Referring to FIG. 8, a search candidate by PMVFAST is determined in step 810. That is, the search candidate is set using the motion vector of the corresponding block (comparison block) of the reference frame and the motion vector of neighboring blocks of the corresponding block (current block) of the current frame.

In operation 820, the search points of the search candidates may be searched by using blocks corresponding to each mode of the main block mode (that is, blocks of 32 * 32, 32 * 36, 16 * 32, and 16 * 16 size). Select a point (ie, the point with the least SAD). The order of each block used for the search of the main block mode may be randomly selected or preset. Or, it may be determined according to the number of search candidates. For example, when the number of points according to the search candidates is less than or equal to the predetermined number, the number of search candidates is greater than or equal to the predetermined number of blocks from 32 * 32 blocks. It can be used from 16 * 16 block to the largest block.

Here, all the search candidates may be searched, but only a part of the search candidates may be searched using a fast motion estimation method such as a diamond search method or a cross search method. Fast motion estimation techniques, such as diamond search and cross search, are still in use and will not be described in detail.

In operation 830, it is determined whether the mode in which the minimum SAD is searched among the optimal SADs searched according to each mode of the main block mode is a 16 * 16 block.

If the result of the determination in step 830 is a 16 * 16 block, the block according to each mode of the sub-block mode (that is, each block of 16 * 16, 16 * 8, 8 * 16, 8 * 8 size) in step 840 Search for the minimum SAD again. Of course, the SAD having the minimum value is selected by comparing each SAD of the points searched by blocks of each size according to the sub-block mode. In addition, since the search by the 16 * 16 block is performed in step 820, the search according to the sub-block mode may be omitted.

In contrast, if the determination result in step 830 is not a 16 * 16 block, the process proceeds to step 850 without performing step 840.

Here, according to another embodiment, without having to perform step 830, in step 820, each block having a minimum SAD is searched for blocks of all sizes according to the main block mode and the sub block mode, and one minimum SAD is determined according to the result. You can also choose a point to have.

Link each pixel according to the corresponding block of the current frame with the reference frame using the corresponding block mode (that is, one of eight according to the main block mode and the sub block mode) in which the point having the minimum SAD selected in step 850 is searched. do. That is, a motion vector is calculated between the corresponding block (or pixel) of the current frame and the block (or pixel) corresponding to the point having the selected minimum SAD of the reference frame.

Therefore, according to the present embodiment, by introducing the PMVFAST motion prediction algorithm rather than the global search method used in the conventional motion compensation temporal filtering, the time required for the motion prediction, which is the purpose of the original PMVFAST, can be greatly reduced, and the initial setting value of the PMVFAST is reduced. Due to this, the distribution of the obtained motion vector is relatively flat, thereby reducing the case of overlapping selection as a reference block. This, in turn, reduces the number of unconnected pixels in the motion compensation temporal filtering, which makes the decomposition of energy into the low / high frequency subband more efficient by increasing the number of pixels applied to the motion compensation temporal filtering. Since the amount of energy remaining in the high frequency subband is directly proportional to entropy and directly related to the bit rate in the entropy encoding stage, efficient motion compensation temporal filtering results in a video encoding having a smaller bit rate while maintaining the same image quality. . In addition, by using a variable sized block to increase the accuracy of motion prediction, the performance of motion compensation temporal filtering is improved, and the subjective picture quality of the image after filtering is improved.

The method of the present invention as described above may be implemented in a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

As described above, according to the present invention, the motion compensation temporal filtering method and the method for an efficient scalable video coding that can minimize the number of multi-linked pixels by flattening the motion vector field so that the motion vectors do not overlap each other as possible It is possible to provide a recording medium on which a program for executing the recording is recorded.

In addition, the present invention can increase the accuracy of motion compensation temporal filtering by using a variable size block for motion prediction together with the flattening of the motion vector field.

In addition, according to the present invention, as a result of efficient motion-compensated temporal filtering, it is possible to encode an image having a smaller bit rate while maintaining the same image quality, and improve the performance of motion-compensated temporal filtering by using a variable sized block to increase the accuracy of motion prediction. It is possible to improve the subjective image quality of the image after performing filtering.

Although described above with reference to a preferred embodiment of the present invention, those skilled in the art that various modifications of the present invention without departing from the spirit and scope of the invention described in the claims below And can be changed.

Claims (12)

A motion compensated temporal filtering method for wavelet based scalable video encoding, (a) determining search candidates for search points in a reference frame for calculating a motion vector of any current block in the current frame using a predictive motion vector field adaptive search technique (PMVFAST); (b) calculating a Sum Absolute Difference (SAD) between the current block and the comparison block of the reference frame by the search points according to the search candidates to determine a point having the minimum SAD among the search points. Searching and selecting as an optimal point; And (c) calculating a motion vector of the current block by using the found optimal point; The current block may be set to a plurality of blocks having different sizes, and step (b) may search for a point having a minimum SAD with each of the plurality of current blocks, and search each other for the SAD of each found point. And comparing the point having the least SAD as the optimum point. delete The method of claim 1, The current block may be set to four main blocks having sizes of 32 * 32, 32 * 36, 16 * 32, and 16 * 16. The method of claim 3, wherein If the point searched by the main block having the size of 16 * 16 among the available size of the current block has the minimum SAD, the step (b) is 16 * 16, 16 * 8, 8 * 16, 8 A motion compensation temporal filtering method characterized by searching for a point having a minimum SAD for selecting the optimum point with four sub-blocks having a size of * 8. The method of claim 1, The current block can be set to 7 blocks having sizes of 32 * 32, 32 * 36, 16 * 32, 16 * 16, 16 * 8, 8 * 16, and 8 * 8. . The method according to claim 3 or 5, The order in which the current blocks are used for searching is set to correspond to the number of search candidates. In order to perform a motion compensation temporal filtering method for wavelet-based scalable video encoding, a program of instructions that can be executed in a digital processing apparatus is implemented, and in a recording medium on which a program that can be read by a digital processing apparatus is recorded. , (a) determining search candidates for search points in a reference frame for calculating a motion vector of any current block in the current frame using a predictive motion vector field adaptive search technique (PMVFAST); (b) calculating a Sum Absolute Difference (SAD) between the current block and the comparison block of the reference frame by the search points according to the search candidates to determine a point having the minimum SAD among the search points. Searching and selecting as an optimal point; And (c) calculating a motion vector of the current block by using the found optimal point; The current block may be set to a plurality of blocks having different sizes, and step (b) may search for a point having a minimum SAD with each of the plurality of current blocks, and search each other for the SAD of each found point. And a point having a minimum SAD in comparison as the optimum point. delete The method of claim 7, wherein And the current block may be set to four main blocks having sizes of 32 * 32, 32 * 36, 16 * 32, and 16 * 16. The method of claim 9, If the point searched by the main block having the size of 16 * 16 among the available size of the current block has the minimum SAD, the step (b) is 16 * 16, 16 * 8, 8 * 16, 8 A recording medium having a program recorded therein, wherein four sub-blocks having a size of 8 are searched for a point having a minimum SAD for selecting the optimum point. The method of claim 7, wherein The current block can be set to seven blocks having sizes of 32 * 32, 32 * 36, 16 * 32, 16 * 16, 16 * 8, 8 * 16, and 8 * 8. media. The method according to claim 9 or 11, And the order in which the current blocks are used for searching is set to correspond to the number of the search candidates.

Images