WO2011071347A2

WO2011071347A2 - Image encoding/decoding method and apparatus using selective filtering of a motion-compensated frame, and inter-prediction method and apparatus for same

Info

Publication number: WO2011071347A2
Application number: PCT/KR2010/008868
Authority: WO
Inventors: 임정연; 김수년; 최재훈; 이규민; 한종기; 이영렬; 문주희; 김해광; 전병우; 유영조
Original assignee: 에스케이텔레콤 주식회사
Priority date: 2009-12-11
Filing date: 2010-12-10
Publication date: 2011-06-16
Also published as: WO2011071347A3

Abstract

Embodiments of the present invention relate to an image encoding/decoding method and apparatus using selective filtering of a motion-compensated frame, and to an inter-prediction method and apparatus for same. One embodiment of the present invention provides an image-encoding apparatus which selectively filters, for each of a plurality of filtering blocks, the motion-compensated frame generated by estimating and compensating for the motion of the current frame of an image, so as to generate a filtered motion-compensated frame, subtracts the filtered motion-compensated frame from the current frame to generate a residual frame, transforms and quantizes the residual frame, and encodes the transformed and quantized residual frame. According to the present invention, the motion-compensated frame is filtered to remove a large amount of the temporal redundancy remaining in the residual frame, thus improving video compression efficiency.

Description

Image encoding / decoding method and apparatus using selective filtering of motion compensation frame and inter prediction method and apparatus therefor

An embodiment of the present invention relates to a method and apparatus for image encoding / decoding using selective filtering of a motion compensation frame, and an inter prediction method and apparatus therefor. More specifically, the present invention relates to a method and apparatus for improving the compression efficiency of a video by filtering a motion compensation frame generated through motion estimation and motion compensation during video encoding and decoding.

The content described in this section merely provides background information on the embodiments of the present invention and does not necessarily constitute prior art.

In order to efficiently store or transmit video data, it is necessary to compress and compress the data. Techniques for compressing video data include H.261, H.263, H.264, MPEG-2, and MPEG-4. In the video compression technique, the luminance component and the chrominance component of each frame image of the video are predicted temporally through motion estimation and motion compensation, and the prediction residuals are transformed, quantized, and entropy encoded to be transmitted in the form of a bitstream.

H.264 / AVC, the latest video compression technology, was developed jointly by Moving Picture Experts Group (MPEG) and Video Coding Experts Group (VCEG). Macroblocks are 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, Motion estimation and motion compensation are performed by dividing into smaller blocks having a size of 4x4. In this case, up to 16 frames may also be used for the reference frame for motion estimation and motion compensation. In addition, the H.264 / AVC adopts a 1/4 pixel interpolation method, and the compression rate can be improved while maintaining the image quality compared to MPEG-4, which is a conventional video compression technology.

As such, in the conventional video compression technology, high compression ratio can be achieved by removing temporal redundancy in the image using motion estimation and motion compensation, but the motion compensation frame generated through motion estimation and motion compensation is still temporal. Since a lot of redundancy remains, the prediction residual to be transformed, quantized, and entropy encoded becomes large, and thus a large amount of residual data needs to be encoded.

In order to solve this problem, an embodiment of the present invention is to improve the compression efficiency of a video by filtering a motion compensation frame generated through motion estimation and motion compensation during video encoding and decoding, and encoding the remaining frame. There is this.

In order to achieve the above object, according to an embodiment of the present invention, in an apparatus for encoding / decoding an image, a motion filtered by selectively filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame for each filtering block An image encoder for generating a compensation frame and subtracting the current frame and the filtered motion compensation frame to generate, transform, quantize, and encode a residual frame; And restoring the motion information, the transformed and quantized residual frames by decoding motion information data and the image coded data extracted from the bitstream, and restoring the residual frames by inverse quantization and inverse transform. Compensating for motion to generate a motion compensation frame and selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame, and adding the reconstructed residual frame and the filtered motion compensation frame to recover the current frame. An image encoding / decoding apparatus comprising an image decoder is provided.

Further, to achieve another object of the present invention, an embodiment of the present invention, in the apparatus for encoding an image, selectively filtering the motion compensation frame generated by estimating and compensating for the motion of the current frame of the image for each filtering block A predictor for generating a filtered motion compensation frame, a subtractor for generating a residual frame by subtracting the current frame and the filtered motion compensation frame, a transformer and a quantizer for transforming and quantizing the residual frame, and encoding the transformed and quantized residual frame. It provides an image encoding apparatus comprising an encoder.

In addition, in order to achieve another object of the present invention, an embodiment of the present invention, in the apparatus for decoding a video, the motion information and transformed and quantized residual by decoding the motion information data and the video encoded data extracted from the bitstream Decoder for reconstructing frame, Inverse quantizer and inverse transformer for reconstructing residual frame by inverse quantization and inverse transform of reconstructed transform and quantized residual frame, Motion compensation by compensating for motion of current frame based on reconstructed motion information A predictor for generating a frame and selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame, and an adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame. Providing an image decoding device do.

In addition, to achieve another object of the present invention, an embodiment of the present invention, a motion estimator for determining the motion information for the block of the image, generates a motion compensation block based on the determined motion information and includes a motion compensation block A motion compensator for generating a motion compensation frame and a motion compensation filter for selectively filtering the motion compensation frame for each filtering block are provided.

In addition, in order to achieve another object of the present invention, an embodiment of the present invention generates a motion compensation block based on motion information extracted from the bitstream, decoded and reconstructed, and generates a motion compensation frame including the motion compensation block. It provides a motion compensator and a motion compensation filter for selectively filtering the motion compensation frame for each filtering block.

In addition, to achieve another object of the present invention, an embodiment of the present invention, in a method for encoding / decoding an image, generates a motion compensation frame by estimating and compensating for the motion of the current frame of the image and the motion compensation An image encoding step of selectively filtering a frame for each filtering block to generate a filtered motion compensation frame, and converting, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame; And generating a motion compensation frame by compensating for the motion of the current frame based on the motion information reconstructed by decoding the bitstream, and selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame. And an image decoding step of reconstructing the current frame by adding the residual frame to be decoded and the filtered motion compensation frame.

According to another aspect of the present invention, there is provided a method of encoding an image, the method comprising: estimating and compensating for a motion of a current frame of an image to generate a motion compensation frame; Selectively filtering the filtering blocks to generate a filtered motion compensation frame, and converting, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame. .

The image encoding method may further include encoding filtering application information for each block indicating whether the motion compensation frame is selectively filtered for each filtering block.

The generating of the filtered motion compensation frame may include generating the motion compensation frame when the coding cost for the motion compensation frame selectively filtered for each filtering block is smaller than the coding cost for the motion compensation frame that has been filtered. The compensation frame may be selectively filtered for each filtering block.

The generating of the filtered motion compensation frame may include determining whether an encoding cost when filtering a filtering block for each filtering block of the motion compensation frame exceeds an encoding cost when the filtering block is not filtered. You can decide whether to filter by filter.

The filtered motion compensation frame may include an unfiltered filtering block and a filtered block of the motion compensation frame.

The generating of the filtered motion compensation frame may be performed by using different filters or different filter coefficients for each filtering block or for each pixel position of the filtering block.

The image encoding method may further include encoding filtering identification information indicating whether to filter each filtering block.

The image encoding method may further include encoding filtering identification information indicating whether to filter by the filtering block or a bundle or a combination unit of the filtering blocks.

The image encoding method may further include encoding the filtering identification information using the basic coding block.

The image encoding method may further include encoding whether the filtering block is encoded using the segmentation identification information and the filtering identification information by using the hierarchical filtering block structure.

The image encoding method may further include encoding filtering block identification information indicating a size or a unit of the filtering block.

In addition, in order to achieve another object of the present invention, an embodiment of the present invention, in a method of decoding an image, a motion compensation frame by compensating for the motion of the current frame based on the motion information that is reconstructed by decoding the bitstream Generating a filtered motion compensation frame by selectively filtering a motion compensation frame for each filtering block; and restoring a current frame by adding a residual frame to be recovered by decoding the bitstream and the filtered motion compensation frame. It provides a video decoding method comprising a.

The generating of the filtered motion compensation frame may selectively filter the motion compensation frame for each filtering block based on the filtering application information for each block, which is reconstructed by decoding the bitstream.

The generating of the filtered motion compensation frame may select a filtering block to be filtered among the filtering blocks of the motion compensation frame based on the filtering identification information reconstructed by decoding the bitstream.

The filtered motion compensation frame may include an unfiltered filtering block and a filtered filtering block of the motion compensation frame.

The generating of the filtered motion compensation frame may determine, as the filtering block, a block having a size identified by the filtering block identification information reconstructed by decoding the bitstream.

The restoring of the filtering identification information may include filtering in units of a filtering block that is restored by decoding the bitstream. The identification information may determine a filtering unit to be used for decoding among the frame unit and the filtering block unit.

Restoring the filtering identification information in units of the filtering block may include restoring size information of the filtering block by the filtering block size information restored by decoding the bitstream.

In the restoring of the filtering identification information in the filtering block unit, the filtering information may be determined by restoring the identification information of the filtering block in the basic coding block unit.

Restoring the filtering identification information in units of the filtering block may determine whether to filter the filtering block having a hierarchical structure in consideration of various filtering block sizes by the filtering block identification information restored by decoding the bitstream.

In addition, to achieve another object of the present invention, an embodiment of the present invention, in the inter prediction method, determining the motion information for the block of the image, generating a motion compensation block based on the determined motion information And generating a motion compensation frame including a motion compensation block and selectively filtering the motion compensation frame for each filtering block.

In addition, to achieve another object of the present invention, an embodiment of the present invention, in the inter prediction method, generates a motion compensation block based on motion information extracted from the bitstream and decoded and includes a motion compensation block; And generating a motion compensation frame and selectively filtering the motion compensation frame for each filtering block.

As described above, according to the exemplary embodiment of the present invention, the compression efficiency may be further improved by removing temporal redundancy remaining in the motion compensation frame generated through the motion estimation and the motion compensation. In addition, since the quantization error can be further reduced by adaptively selecting and filtering the filter type and the filter coefficient according to the characteristics of the pixels of the block in the motion compensation frame, the compression efficiency can be further improved, and the encoded and decoded reconstruction is restored. The deterioration of the image quality of the image can be further reduced.

In addition, in the case of filtering the motion compensation frame for each block, compression efficiency may be further improved by adaptively selecting and filtering the filtering block for each filtering block.

1 is a block diagram schematically illustrating a video encoding apparatus according to an embodiment of the present invention;

2 is a block diagram schematically illustrating an inter prediction apparatus for image encoding according to an embodiment of the present invention;

3 is an exemplary view illustrating a process of generating a motion compensation frame according to an embodiment of the present invention;

4 is an exemplary view illustrating a process of filtering a motion compensation frame according to an embodiment of the present invention;

5 is a flowchart illustrating a video encoding method according to an embodiment of the present invention;

6 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention;

7 is a block diagram schematically illustrating an inter prediction apparatus for image decoding according to an embodiment of the present invention;

8 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

9 is a flowchart illustrating a video encoding method according to another embodiment of the present invention;

FIG. 10 is a flowchart for exemplarily describing a process of determining whether to filter the entire motion compensation frame or selectively filter the motion compensation frame for each filtering block based on an encoding cost according to another embodiment of the present invention; FIG.

11 is a flowchart for exemplarily describing a process of determining whether to perform filtering for each filtering block according to another embodiment of the present invention;

12 is an exemplary view showing a method of encoding filtering identification information according to another embodiment of the present invention;

13 is a flowchart illustrating an image decoding method according to another embodiment of the present invention.

A video encoding apparatus (Video Encoding Apparatus), a video decoding apparatus (Video Decoding Apparatus) to be described below is a personal computer (PC), notebook computer, personal digital assistant (PDA), portable multimedia player (PMP) It may be a user terminal such as a portable multimedia player (PSP), a PlayStation Portable (PSP), a wireless communication terminal, a smart phone, a TV, or a server terminal such as an application server or a service server. A communication device such as a communication modem for communicating with various devices or a wired / wireless communication network, a memory for storing various programs and data for encoding or decoding an image or inter or intra prediction for encoding or decoding, and executing a program And a microprocessor for controlling and the like. It can mean a variety of devices.

In addition, an image encoded by a video encoding device into a bitstream (encoded data) may be transmitted in real time or non-real time through a wired or wireless communication network such as the Internet, a local area wireless communication network, a wireless LAN network, a WiBro network, or a mobile communication network, or a cable or universal serial bus. The image decoding apparatus may be transmitted to a video decoding apparatus through various communication interfaces such as a universal serial bus (USB), and may be decoded by the video decoding apparatus to restore and reproduce the video.

In general, a moving picture is composed of a series of pictures, and each picture is divided into a predetermined area such as a block. When a region of an image is divided into blocks, the divided blocks are largely classified into intra blocks and inter blocks according to encoding methods. An intra block refers to a block that is encoded by using an intra prediction coding method. An intra prediction coding is performed by using pixels of blocks that are previously encoded, decoded, and reconstructed in a current picture that performs current encoding. A prediction block is generated by predicting pixels of a block, and a difference value with pixels of the current block is encoded. An inter block refers to a block that is encoded using inter prediction coding. Inter prediction coding generates a prediction block by predicting a current block within a current picture by referring to one or more past or future pictures, and then generates a current block. This is a method of encoding a difference value with. Here, a frame referred to for encoding or decoding the current picture is referred to as a reference frame.

1 is a block diagram schematically illustrating a video encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus 100 according to an embodiment of the present invention is an apparatus for encoding an image, and includes a predictor 110, a subtractor 120, a transformer and quantizer 130, and an encoder. The encoder may include an encoder 140, an inverse quantizer and an inverse transformer 150, an adder 160, a deblocking filter 170, and a memory 180.

The predictor 110 filters a motion compensation frame generated by estimating and compensating for a motion of a frame to be encoded in the input image (hereinafter referred to as a current frame). That is, the predictor 110 estimates the motion of each block in the current frame according to the inter prediction mode in the reference frame stored in the memory 180 after it is already encoded, decoded, and reconstructed, and then is a motion vector. ) And generate a motion compensation frame in which the motion of the current frame is compensated by compensating the motion of each block using the motion vector, and filtering the generated motion compensation frame.

To this end, the predictor 110 determines a motion vector and a reference frame index for a block in which the block mode is inter mode in the current frame, and uses the determined motion vector and reference frame index. The motion compensation block may be generated, and the motion compensation frame may be filtered by generating a motion compensation frame including the generated motion compensation block.

Here, the predictor 110 may filter the motion compensation frame using another filter according to the characteristics of the pixels of the block in the motion compensation frame, and the motion compensation frame may be a Winner filter or a deblocking filter. Filtering may also be performed using various filters such as the above.

Also, the predictor 110 may filter the motion compensation frame using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame.

The subtractor 120 subtracts the current frame and the filtered motion compensation frame to generate a residual frame. Here, the residual frame is a frame including a residual generated by subtracting the current frame and the filtered motion compensation frame, and refers to a frame including a residual signal that is a difference between pixels of the current frame and pixels of the filtered motion compensation frame.

Converter and quantizer 130 transforms and quantizes the remaining frames. That is, the transformer and quantizer 130 converts the residual signal of the residual frame generated by the subtractor 120 into a frequency domain to generate a transformed residual frame having a transform coefficient and transforms the transformed signal. The transform coefficients of the residual frame are quantized to produce transformed and quantized residual frames. In this case, as a transform method used, a method of transforming an image signal in a spatial domain into a frequency domain such as a Hadamard transform or a discrete cosine transform based integer transform is used. As a scheme, various quantization techniques such as Dead Zone Uniform Threshold Quantization (DZUTQ) or quantization weighted matrix (DZUTQ) may be used.

The encoder 140 encodes the transformed and quantized residual frames. That is, the quantized transform coefficients of the residual frame transformed and quantized by the transformer and the quantizer 130 are encoded to generate image encoded data. The generated image coded data is included in the bitstream. As the encoding technique, an entropy encoding technique may be used, but various encoding techniques may be used without being limited thereto.

In addition, the encoder 140 may generate filter coefficient data by encoding filter coefficient information indicating filter coefficients of a filter used to filter the motion compensation frame. The generated filter coefficient data is included in the bitstream.

In addition, the encoder 140 may generate block mode data by encoding block mode information indicating a block mode of each block in the current frame, and the generated block mode data is included in the bitstream.

Also, the encoder 140 may generate motion information data by encoding motion information indicating a motion vector and a reference frame index determined for a block in which the block mode is inter mode in the current frame. It is included in the bitstream.

Meanwhile, the functions of the transformer and the quantizer 130 may be integrated into the encoder 140 and implemented. That is, when implemented with one encoder, the encoder generates encoded data by encoding the residual frame.

The inverse quantizer and inverse transformer 150 inverse quantizes and inverse transforms the transformed and quantized residual frame to restore the residual frame. That is, the inverse quantizer and inverse transformer 150 inverse quantizes the transformed and quantized residual frames transmitted from the transformer and quantizer 130 to restore the residual frame having the transform coefficients, and reconstructs the residual frame having the transform coefficients. Inverse conversion restores the residual frame with the residual signal. In this case, the inverse quantizer and the inverse transformer 150 may restore the remaining frames by performing the inverse transformation and quantization in the transformer and the quantizer 130.

The adder 160 reconstructs the current frame by adding the reconstructed residual frame and the filtered motion compensation frame.

The deblocking filter 170 deblocks and filters the current frame to be restored, and the memory 180 stores the deblocking filtered current frame. Here, the deblocking filtering refers to an operation of reducing block distortion generated by encoding an image in block units, and applying a deblocking filter to a block boundary and a macroblock boundary, or applying a deblocking filter only to a macroblock boundary or a deblocking filter. You can optionally use one of the methods that does not use.

Although not shown in FIG. 1, the image encoding apparatus 100 according to an embodiment of the present invention may further include an intra predictor for intra prediction in the predictor 110. In this case, the subtractor 120 may generate a residual frame by subtracting a prediction frame generated by the current frame and the intra predictor, and the transformer and quantizer 130 and the inverse quantizer and inverse transformer 150 may generate a residual frame. Calculation may be further performed for transform and quantization for and inverse transform and inverse quantization for the transformed and quantized residual frame. Also, the encoder 140 may generate image encoded data by encoding the transformed and quantized residual frames, which are included in the bitstream.

Therefore, in the exemplary embodiment of the present invention, only the remaining frame, which is a difference between the current frame and the filtered motion compensation frame, is transformed, quantized, and encoded, so that the image encoded data according to inter prediction encoding is included in the bitstream. The apparatus 100 may further generate image coded data to be intra predictively coded, and in the bitstream, not only image coded data generated by inter prediction coding, but also image coded data generated by intra predictive coding and the image coded data thereof. It will be appreciated that the information for intra prediction decoding or its encoded data may be further included.

2 is a block diagram schematically illustrating an inter prediction apparatus for image encoding according to an embodiment of the present invention.

Since the inter prediction apparatus for image encoding according to an embodiment of the present invention may be implemented as the predictor 110 in the image encoding apparatus 100 according to the embodiment of the present invention described above with reference to FIG. For convenience, the predictor 110 is called.

The predictor 110 may include a motion estimator 210, a motion compensator 220, and a motion compensation filter 230. In addition, the predictor 110 may further include an intra predictor, but is not separately illustrated.

The motion estimator 210 determines a motion vector and a reference frame index for a block in which the block mode is inter mode in the current frame. That is, the motion estimator 210 determines the motion vector and the reference frame index by estimating the motion of the blocks in which the block mode is the inter mode among the blocks in the current frame. Here, the motion estimator 210 may determine the block mode for each block in the current frame, but may be determined by a mode determiner (not shown) additionally provided in the predictor 110 or the image encoding apparatus 100. have.

To this end, the motion estimator 210 performs forward prediction on a block whose block mode is P macroblock in the current frame to estimate a forward motion vector based on the previous frame of the current frame. Bidirectional prediction may be performed on a block having a B macroblock of B macroblock within the block to estimate a forward motion vector and a backward motion vector based on the previous frame and the next frame of the current frame.

In this case, in determining the motion vector, the motion estimator 210 may determine, as the motion vector and the reference frame index for the corresponding block in the current frame, the motion vector and the reference frame index at which the encoding cost is minimum. As such, the mcost function may be used as a function for calculating the coding cost for determining the motion vector and the reference frame index. The mcost function may be expressed as in Equation 1.

[Equation 1]

here,

Is a Lagrange Multiplier for determining the motion vector and the reference frame index, and rate _{u, v, i} is predictively coded by compensating the motion of the block using the motion vector and the reference frame index. Indicates the bit rate of the bitstream. SAD _{u, v, i} represents the sum of the absolute value of the difference between the motion compensation frame and the current frame generated by compensating for the motion of the corresponding block using the motion vector and the reference frame index, and can be expressed by Equation 2 below. .

[Equation 2]

In Equation 2, M and N respectively represent the pixel size in the horizontal direction and the vertical direction of the block,

Of the current frame

Represents a pixel value for the coordinate of,

Is the i reference frame

Represents a pixel value for the coordinate of. In addition, x and y represent the coordinates in the current frame or the motion compensation frame of the pixel located at the top left of the block, m and n represents the coordinates of the pixel located inside the block. u and v represent the horizontal and vertical coordinates of the motion vector of the block.

The motion estimator 210 calculates mcost for each motion vector and reference frame index for each block whose block mode is interblock in the current frame by using a function for calculating a coding cost as shown in Equation 1, A motion vector and a reference frame index for minimizing the encoding cost are determined as the motion vector and the reference frame index of the corresponding block using a function as shown in FIG. 3.

[Equation 3]

In Equation 3, SearchRange represents a range for performing motion estimation.

Denotes the horizontal and vertical components of the motion vector determined for the block, respectively, and i ^* denotes the reference frame index determined for the block.

The motion compensator 220 generates a motion compensation block by using the determined motion vector and the reference frame index, and generates a motion compensation frame including the motion compensation block. That is, the motion compensator 220 generates a motion compensation block by compensating for the motion of the corresponding block by using the motion vector determined by the motion estimator 210 and the reference frame index, and gathers the generated motion compensation blocks in units of frames. Generate a motion compensation frame comprising motion compensation blocks. To this end, the motion compensator 220 obtains a block indicated by the motion vector determined by the motion estimator 210 in the reference frame identified by the reference frame index determined by the motion estimator 210 as a motion compensation block. And a motion compensation frame including these motion compensation blocks.

In this way, a method of generating a motion compensation frame may be represented by Equation 4 below.

[Equation 4]

In Equation 4, W and H represent the horizontal and vertical pixel size of the frame (reference frame and motion compensation frame),

Of the motion compensation frame

Means the pixel value for the coordinate. if,

If the block mode of the block containing the pixels identified by the coordinates is the intra mode, the motion vector and the reference frame index do not exist.

Has no value. Subsequently, when generation of one motion compensation frame is completed, the missing portion may be filled in a similar manner to the mirroring method.

Referring to FIG. 3 exemplarily illustrating a process of generating a motion compensation frame according to an embodiment of the present invention, four blocks among five blocks in the motion compensation frame have an interblock in four reference frames. Each block is filled with one block, but one block does not have a value because the block mode is an intra block and there is no motion vector and reference frame index. When the motion compensation frame is filtered later, since the value of the intra mode block is not filled, it is impossible to filter the neighboring pixels, and thus, the motion compensation frame may be filtered using the pixel values of the pixels around the block.

The motion compensation filter 230 filters the motion compensation frame. That is, the motion compensation filter 230 is transformed and quantized by filtering pixels of the motion compensation frame generated by the motion compensator 220 using various types of filters to make the filtered motion compensation frame more similar to the current frame. Reduce the amount of data in the residual frame to be encoded or reduce the blocking effect present in the motion compensation frame.

Here, the motion compensation filter 230 may use a predetermined filter when filtering the motion compensation frame, or may select and use an appropriate filter or a coefficient of the filter in a predetermined unit such as a block unit, a slice unit, or a frame unit. When the motion compensation filter 230 uses a preset filter, a filter pre-appointed by the video encoding apparatus 100 and a video decoding apparatus to be described later, or a filter based on a pre-promised criterion may be used. In addition, when the motion compensation filter 230 selects and uses an appropriate filter or a filter coefficient, another filter may be used according to the characteristics of the pixels in the current frame.

As a filter for filtering the motion compensation frame, various filters such as a Wiener filter or a deblocking filter may be used. When the motion compensation frame is filtered using the Wiener filter, the temporal redundancy remaining in the motion compensation frame can be further eliminated, thereby reducing the amount of data in the remaining frames and consequently reducing the amount of image coded data. As it becomes smaller, the compression efficiency can be improved. In addition, when the motion compensation frame is filtered using the deblocking filter, the blocking phenomenon in the motion compensation frame can be reduced, thereby improving the quality of the image which is decoded and then reconstructed.

When the motion compensation filter 230 selects an appropriate filter or coefficients of the filter, the filter is selected according to the characteristics of the pixels in the motion compensation frame and filtered using the selected filter, or the filter coefficients of the preset filter are calculated. Filtering may be performed using the calculated filter coefficients or the filter type may be selected and the filter coefficients of the selected filter may also be calculated and filtered.

For example, the motion compensation filter 230 determines the size of the motion compensation block in the motion compensation frame and determines the size of the motion compensation block, and the boundary between the pixels located at the boundary of the block within the corresponding motion prediction block and the block boundary. Pixels not located at may be filtered using different types of filters. In this case, the characteristic of the pixel in the motion compensation frame may be the size of the motion compensation block in the motion compensation frame. In addition, the motion compensation filter 230 determines a difference between the motion vector of the motion compensation blocks and the reference frame index, and determines whether a blocking phenomenon occurs at the boundary between the motion compensation blocks according to the difference between the motion vector and the index of the reference frame. By determining, the pixels of the portion where the blocking phenomenon occurs and the pixels of the portion where the blocking phenomenon does not occur may be filtered using different filters. In this case, the characteristic of the pixel in the motion compensation frame may be a difference between the motion vector and the reference frame index between the motion compensation blocks.

If the motion compensation filter 230 filters the motion compensation frame in an exemplary equation, it may be represented by Equation 5. In Equation 5, the case of filtering by using a two-dimensional filter is shown as an example, but may be filtered using a one-dimensional filter, depending on the implementation method or needs.

[Equation 5]

In Equation 5,

Of the motion compensation frame

Represents the pixel value of the pixel indicated by the coordinates of

Is

PP represents the second filter coefficient, and PP represents the type of filter selected according to the characteristics of the pixels in the block in the motion compensation frame.

Within the filtered motion compensation frame

Represents the pixel value of the pixel located at.

Referring to FIG. 4 exemplarily illustrating a process of filtering a motion compensation frame according to an embodiment of the present invention, the motion compensation frame having a blocking phenomenon at the boundary between the motion compensation blocks is filtered using the motion compensation filter. In the filtered motion compensation frame, a blocking phenomenon may be removed at the boundary between the motion compensation blocks. 4 exemplarily illustrates a case where a two-dimensional deblocking filter is used as a motion compensation filter. In FIG. 4, * denotes convolution.

In the conventional video compression technique, since the motion compensation is performed in units of blocks, the motion compensation frame is used as a conceptual term. However, in the exemplary embodiment of the present invention, the motion compensation frame is filtered by generating and filtering the motion compensation frame in frame units. Is applied to video compression.

As an example of calculating the filter coefficients, the motion compensation filter 230 may calculate a filter coefficient for filtering the motion compensation frame using the Wiener filter such that the square error of the current frame and the filtered motion compensation frame is minimized. Equation 6 shows an equation for calculating a squared error between the current frame and the filtered motion compensation frame, and Equation 7 calculates a filter coefficient that minimizes the square error of the current frame and the filtered motion compensation frame. It is shown. The filter coefficient may be calculated as the root of a linear equation such as Equation (7).

[Equation 6]

[Equation 7]

In Equations 6 and 7, (e ^PP ) denotes an error of the current frame and the filtered motion compensation frame.

Of the current frame

Represents a pixel value for a coordinate,

Is

The second filter coefficient,

Within the filtered motion compensation frame

Represents a pixel value for the coordinate can be calculated through Equation 5.

The filter coefficients calculated through Equations 6 and 7 are used to filter the motion compensation frame and are passed to the encoder 140. In this case, the motion compensation filter 230 may filter the motion compensation frame using the calculated filter as it is, but may filter the motion compensation frame by correcting the filter coefficient through a process such as quantization and rounding down. For example, if the filter coefficient is calculated as a coefficient having a decimal point, if it is encoded as it is, the data amount of the filter coefficient data increases, so that the coefficient is corrected to an integer coefficient through a process such as quantization, rounding, and rounding, and then the corrected filter coefficient The motion compensation frame is filtered and the corrected filter coefficients are transmitted to the encoder 140. The encoder 140 may generate filter coefficient data by encoding filter coefficient information indicating the filter coefficient, and the generated filter coefficient data may be included in the bitstream and transmitted to the image decoding apparatus. By correcting and encoding the filter coefficients as described above, the data amount of the filter coefficient data can be reduced.

The filtered motion compensation frame generated as described above is transferred to the subtractor 120, and the subtractor 120 generates a residual frame by subtracting the current frame and the filtered motion compensation frame. In this case, the filtered motion compensation block may be adaptively used for each unit such as a block or slice unit. The residual frame includes a residual signal having a difference value between the pixel value of the pixel of the current frame and the pixel value of the pixel of the motion compensation frame. The residual signal may be represented as in Equation 8.

[Equation 8]

However, the residual signal generated by subtracting the current frame and the motion compensation frame according to a typical video compression method may be represented by Equation (9).

[Equation 9]

In Equation 8 and Equation 9

Is

Represents the pixel value of the residual signal in coordinates. When the motion compensation filtering is performed using the Wiener filter, the temporal redundancy remaining in the motion compensation frame is further removed so that the residual signal of Equation 8 is smaller than the residual signal of Equation 9, and the motion is performed using a deblocking filter. When the compensation filtering is performed, the blocking phenomenon remaining in the motion compensation frame is further eliminated, so that the image quality is improved when the residual signal of Equation 8 is transformed and quantized and the encoded data is decoded.

Although Equation 8 illustrates that the residual signal is generated in units of frames, Equation 8 is not necessarily used or the residual signal is generated in units of frames, and the residual signal may be generated in various units such as blocks or slices. The residual signal may be generated using an equation other than Equation 8.

5 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

According to an embodiment of the present invention, the image encoding apparatus 100 filters the motion compensation frame generated by estimating and compensating for the motion of the current frame of the image (S510), and remaining the difference between the current frame and the filtered motion compensation frame. The frame is transformed, quantized, and encoded (S520).

In operation S510, the image encoding apparatus 100 may filter the motion compensation frame using another filter according to the characteristics of the pixels of the block in the motion compensation frame, and use the Wiener filter or the deblocking filter as the motion compensation frame. The motion compensation frame may be filtered using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame. In this case, the image encoding apparatus 100 may further encode filter coefficient information indicating the filter coefficient of the filter used to filter the motion compensation frame.

In operation S510, the image encoding apparatus 100 may determine a motion vector and a reference frame index for a block in which the block mode is an inter mode, and use the determined motion vector and the reference frame index to determine a motion compensation block. And generate a motion compensation frame including the generated motion compensation block, and filter the motion compensation frame.

In addition, the image encoding apparatus 100 may encode block mode information indicating a block mode and further encode motion information indicating a determined motion vector and a reference frame index.

In addition, the image encoding apparatus 100 may reconstruct the residual frame by inverse quantization and inverse transformation of the transformed and quantized residual frame, restore the current frame by adding the reconstructed residual frame and the filtered motion compensation frame, and restore the current frame. Frames can be deblocked filtered and saved.

6 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

The image decoding apparatus 600 according to an embodiment of the present invention includes a decoder (Decoder, 610), an inverse quantizer and an inverse transformer (620), a predictor (630), an adder (640), a deblocking filter (650), and It may be configured to include a memory 660.

The decoder 610 decodes the motion information data and the image encoded data extracted from the bitstream to restore the motion information, the transformed and quantized residual frames. In addition, the decoder 610 may recover block mode information by extracting and decoding block mode data from the bitstream, and may restore filter coefficient information by extracting and decoding filter coefficient data from the bitstream.

Here, the decoder 610 may decode various data using various encoding techniques such as entropy encoding, and may inversely decode a method encoded by the encoder 140 of the image encoding apparatus 100. .

Inverse quantizer and inverse transformer 620 inverse quantizes and inverse transforms the reconstructed transformed and quantized residual frame to reconstruct the residual frame. Here, the inverse quantizer and inverse transformer 620 may perform inverse quantization and inverse transformation by using various transformation and quantization schemes, which are transformed and quantized by the transformer and quantizer 130 of the image encoding apparatus 100. The reverse method can be used to perform inverse magnetization and inverse transformation.

The predictor 630 compensates for the motion of the current frame using the motion vector identified by the reconstructed motion information and the reference frame index to generate a motion compensation frame and filters the motion compensation frame. That is, the predictor 630 may generate a motion compensation block using the motion vector and the reference frame index identified by the reconstructed motion information, and generate a motion compensation frame including the motion compensation block.

In addition, the predictor 630 may filter by using a filter having a predetermined filter coefficient in filtering the motion compensation frame, and, if there is filter coefficient information restored by the decoder 610, the restored filter. The motion compensation frame may be filtered using a filter having a filter coefficient identified by the coefficient information.

In addition, the predictor 630 may filter using another filter according to the characteristics of the pixels of the block in the motion compensation frame, and may use a Wiener filter or a deblocking filter as a filter for filtering the motion compensation frame.

Here, the predictor 630 generates a motion compensation block by using the motion vector and the reference frame index, generates a motion compensation frame including the motion compensation block, and filters the generated motion compensation frame with reference to FIGS. 1 to 5. Through the same or similar to the above-described method, detailed description thereof will be omitted.

The adder 640 reconstructs the current frame by adding the reconstructed residual frame and the filtered motion compensation frame. That is, the adder 640 adds the pixel values of the pixels of the residual frame reconstructed by the inverse quantizer and the inverse transformer 620 and the pixel values of the pixels of the motion compensation frame filtered by the predictor 630 to add the pixels of the current frame. Restore them.

The deblocking filter 650 deblocks and filters the current frame restored by the adder 640 to output the deblocking filtered current frame as a reconstructed image, and the memory 660 uses the deblocking filtered current frame as a reference frame. The stored reference frame is used by the predictor 630 to predict the next frame.

7 is a block diagram schematically illustrating an inter prediction apparatus for image decoding according to an embodiment of the present invention.

The inter prediction apparatus for image decoding according to an embodiment of the present invention may be implemented as a predictor 630 in the image decoding apparatus 600 according to the embodiment of the present invention described above with reference to FIG. For convenience, the predictor 630 is called.

The predictor 630 may include a motion compensator 710 and a motion compensation filter 720.

The motion compensator 710 generates a motion compensation block by using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and decoded and reconstructed, and generates a motion compensation frame including the motion compensation block. That is, when the motion compensator 710 receives the motion vector and the reference frame index from the decoder 610, the block indicated by the motion vector in the reference frame indicated by the reference frame index among the reference frames stored in the memory 660. Is generated as motion compensation blocks and a motion compensation frame including the motion compensation blocks.

The motion compensation filter 720 filters the motion compensation frame. That is, the motion compensation filter 720 filters the motion compensation frame generated and transmitted by the motion compensator 710 using various filters such as a winner filter or a deblocking filter. In this case, when the motion compensation filter 720 receives the filter coefficient information from the decoder 610, the motion compensation filter 720 may generate a motion compensation frame using a filter having the filter coefficient identified by the filter coefficient information. If not, the motion compensation frame may be filtered using a preset filter or a filter selected according to pixel characteristics of a block in the motion compensation frame.

According to an image decoding method according to an embodiment of the present invention, the image decoding apparatus 600 currently uses a motion vector and a reference frame index identified by motion information reconstructed by decoding motion information data extracted from a bitstream. Compensate for the motion of the frame to generate a motion compensation frame (S810), filter the motion compensation frame (S820), and restore the current frame by adding the residual motion and the filtered motion compensation frame extracted and decoded from the bitstream (S830).

Here, in operation S810, the image decoding apparatus 600 extracts and decodes motion information data from a bitstream and decodes motion information data from a bitstream for a block having a block mode interblock in a current frame to restore motion information. The motion compensation block may be generated using the motion vector identified by the reconstructed motion information and the reference frame index, and a motion compensation frame including the generated motion compensation block may be generated.

In addition, the image decoding apparatus 600 may further reconstruct filter coefficient information by extracting and decoding filter coefficient data from a bitstream. In this case, when the motion compensation frame is filtered in step S820, the image decoding apparatus 600 may reconstruct the filter coefficient information. The motion compensation frame may be filtered using a filter having a filter coefficient identified by.

Also, in operation S820, the image decoding apparatus 600 may filter the motion compensation frame using a filter having a predetermined filter coefficient in filtering the motion compensation frame, and may include characteristics of the pixels of the block in the motion compensation frame. The motion compensation frame may be filtered using another filter, and the motion compensation frame may be filtered using a Wiener filter or a deblocking filter.

In addition, the image decoding apparatus 600 may deblock the filtered current frame and output the restored current frame. The deblocking filtered current frame may be stored and used to predictively sign another frame.

As described above, according to an embodiment of the present invention, in predictive encoding of an image, a compression efficiency may be further improved by removing temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation. In addition, it is possible to further reduce the quantization error by adaptively selecting and filtering the filter type and the filter coefficient according to the characteristics of the pixels of the block in the motion compensation frame, so that the compression efficiency can be further improved or the code is decoded after being decoded. The deterioration of the image quality of the reconstructed video to be reproduced can be further reduced.

In the above description, the image is encoded by filtering the entire motion compensation frame of the image through an embodiment of the present invention. However, in the embodiment of the present invention, although the filtering may be performed on the entire motion compensation frame, the filtering may be performed for each block of the motion compensation frame. In this case, the type of filter and the filter coefficient used to filter by block may be adaptively determined and used for each block or for each pixel in the block. In addition, in the embodiment of the present invention, when filtering is performed for each block of the motion compensation frame, the filtering is selectively performed on a block in which it is more efficient that no filtering is performed, and only the block in which filtering is efficient is performed. In some embodiments, filtering may be performed. Hereinafter, a method of encoding an image by selectively filtering the filtering blocks of the compensation frame of the image through another embodiment of the present invention will be described.

An image encoding apparatus according to another embodiment of the present invention is configured similarly or similarly to the image encoding apparatus 100 according to an embodiment of the present invention described above with reference to FIG. 1. That is, the image encoding apparatus according to another embodiment of the present invention is a predictor for generating a filtered motion compensation frame by selectively filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image for each filtering block, and a current frame. And a subtractor for generating a residual frame by subtracting the filtered motion compensation frame, a transformer for transforming and quantizing the residual frame, and an encoder for encoding the transformed and quantized residual frame.

The inter prediction apparatus for image encoding according to another embodiment of the present invention is configured similarly or similarly to the inter prediction apparatus for image encoding according to the embodiment of the present invention described above with reference to FIG. 2. That is, the inter prediction apparatus for image encoding according to another embodiment of the present invention may be implemented as a predictor of the image encoding apparatus according to another embodiment of the present invention, the motion estimator for determining motion information for a block of an image, A motion compensator for generating a motion compensation block based on the determined motion information and generating a motion compensation frame including the motion compensation block, and a motion compensation filter for selectively filtering the motion compensation frame for each filtering block.

An image encoding apparatus and an inter prediction apparatus for image encoding according to another embodiment of the present invention will be described in detail through an image encoding method according to another embodiment of the present invention in a process to be described later. Hereinafter, for convenience of description, an image encoding apparatus according to another embodiment of the present invention is called a filtered image encoding apparatus for each block.

9 is a flowchart illustrating a video encoding method according to another embodiment of the present invention.

According to an image encoding method according to another embodiment of the present invention, the block-by-block filtering image encoding apparatus generates a motion compensation frame by estimating and compensating for the motion of a current frame of an image (S910) and selectively selecting a motion compensation frame for each filtering block. In operation S920, the filtered motion compensation frame is generated by filtering, and the residual frame that is the difference between the current frame and the filtered motion compensation frame is transformed, quantized, and encoded (S930).

In addition, the block-by-block filtering image encoding apparatus may further encode block-specific filtering application information indicating whether the motion compensation frame is selectively filtered for each filtering block. That is, the block-by-block filtering image encoding apparatus determines whether to filter the entire motion compensation frame or divide the motion compensation frame into filtering blocks and selectively filter the filtering blocks according to the filtering blocks, and encodes the block filtering information indicating the same into the bitstream. You can. The filtering application information for each block may be implemented as a flag of 1 bit, but the present invention is not limited thereto, and may be implemented in various ways, and may be encoded and included in a frame header or a sequence header according to an implementation method or need. . If the image encoding apparatus and the image decoding apparatus according to another embodiment of the present invention promise to selectively filter the motion compensation frame for each filtering block in advance or to promise to filter the entire motion compensation frame, the application information for each block is applied. You will not need to encode

In operation S920, the filtering image encoding apparatus for each block filters the motion compensation frame when the encoding cost for the motion compensation frame selectively filtered for each filtering block is smaller than the encoding cost for the motion compensation frame in which the motion compensation frame is entirely filtered. You can selectively filter by block. That is, the block-by-block filtering image encoding apparatus may determine whether to filter the entire motion compensation frame or selectively filter the motion compensation frame for each filtering block based on the encoding cost. For example, the filtering image encoding apparatus for each block calculates an encoding cost when the motion compensation frame is entirely filtered and an encoding cost when the motion compensation frame is selectively filtered for each filtering block, and the motion compensation frame is filtered as a whole. If the coding cost of the case is less than or equal to the coding cost of the motion compensation frame selectively filtered for each filtering block, the entire motion compensation frame is filtered. If the motion compensation frame is entirely filtered, the coding cost of the motion compensation frame is filtered. If it is greater than the encoding cost when selectively filtering for each block, the motion compensation frame may be selectively filtered for each filtering block.

In operation S920, the filtering image encoding apparatus for each block determines whether the encoding cost when the filtering block does not filter by the filtering block of the motion compensation frame exceeds the encoding cost when the filtering block is filtered, and filters the filtering block by the filtering block. Can be determined. That is, when it is determined that the filtering image encoding apparatus for each block selectively filters the motion compensation frame for each filtering block, the filtering image encoding apparatus divides the motion compensation frame into the filtering block units to determine whether to filter each filtering block. The criterion for the determination may be an encoding cost. For example, the block-by-block filtered image encoding apparatus calculates encoding costs for each filtering block and each filtering block of the motion compensation frame, and compares the calculated encoding costs to determine the encoding cost. If the coding cost in the case of filtering is exceeded, the filtering block may be filtered. If the coding cost in the case of not filtering is less than or equal to the coding cost in the filtering, the filtering block may not be filtered. As described above, even when filtering the motion compensation frame for each block, the block-by-block filtering image encoding apparatus may further improve encoding efficiency by adaptively selecting whether to filter by each filtering block.

In operation S920, the block-by-block filtering image encoding apparatus may filter using a different filter or different filter coefficients for each filtering block or for each pixel position of the filtering block. That is, the block-by-block filtering image encoding apparatus may filter by using a different filter for each filtering block or a position of a pixel of the filtering block, or filter using different filter coefficients. As described above, in the case of filtering using different filters or different filter coefficients for each filtering block or for each pixel position of the filtering block, the filtering image encoding apparatus for each block determines what filter is used for each filtering block or for each pixel position of the filtering block. Information about the filter coefficients may be further encoded and included in the bitstream, or information about what filter coefficients are used for each filtering block or for each pixel position of the filtering block may be encoded and included in the bitstream. If the image encoding apparatus and the image decoding apparatus according to another embodiment of the present invention promise to filter by using a predetermined filter or a predetermined filter coefficient at a position of a block or a pixel, what filter is used? It will not be necessary to encode information about the C or information about which filter coefficients are used.

In addition, the filtering image encoding apparatus for each block may further encode filtering identification information indicating whether to filter each filtering block. That is, when the filtering image encoding apparatus for each block selectively filters the motion compensation frame for each filtering block, the filtering image encoding apparatus may include filtering identification information indicating whether the filtering block of the motion compensation frame is filtered and included in the bitstream. The filtering identification information may be implemented as a flag of 1 bit for each filtering block, but is not necessarily limited thereto and may be implemented in various ways. In addition, since the filtering identification information is generated for each filtering block, the filtering identification information may be encoded in an appropriate manner and included in the bitstream. The filtering identification information is used to identify which filtering block is filtered in the motion compensation frame in the image decoding apparatus according to another embodiment of the present invention.

On the other hand, the block-by-block filtering image encoding apparatus encodes the filtering identification information to prevent a large amount of overhead and further reduce the encoding efficiency. It is determined whether the motion compensation frame is selectively filtered for each filtering block based on an encoding cost when the motion compensation frame is selectively filtered for each filtering block.

In addition, the block-by-block filtering image encoding apparatus may further encode filtering block identification information indicating the size or unit of the filtering block. That is, in order to inform the image encoding apparatus according to another embodiment of the present invention, a filtering block according to another embodiment of the present invention to which a block-shaped filtering image encoding apparatus filters a motion compensation frame for each filtering block using a block having a shape and size as a filtering block. Filtering block identification information indicating a size or a unit of may be encoded and included in the bitstream. If the image encoding apparatus and the image decoding apparatus according to another embodiment of the present invention promise to use a filtering block having a predetermined size in advance, it is not necessary to encode the filtering block identification information.

The filtered motion compensation frame generated in step S920 includes an unfiltered filtering block and a filtered block of the motion compensation frame. In other words, when the filtering image encoding apparatus for each block selectively filters the motion compensation frame for each filtering block, the block filtering image encoding apparatus generates a compensation frame that moves not only the filtered filtering blocks but also the unfiltered filtering blocks. The motion compensation frame becomes a filtered motion compensation frame.

Such a filtered motion compensation frame according to another embodiment of the present invention can be represented by Equation 10. Equation 10 shows an expression of selectively applying filtering according to the characteristics of the filtering block.

[Equation 10]

In Equation 10,

Is

Indicates the position of the filtering block including the pixel position,

Is

An index used as filtering identification information indicating whether to filter on a filtering block of a position is indicated.

Is

The second filter coefficient,

Of the motion compensation frame

Represents the pixel value for the coordinate,

Within the filtered motion compensation frame

Represents a pixel value for a coordinate.

Different filters or filter coefficients may be applied to each filtering block having similar characteristics or to pixel positions within the filtering block by dividing by PP.

In addition, in another embodiment of the present invention, the filtering block may be a block as a predetermined coding unit for encoding an image, such as a macroblock or a subblock, but is not necessarily limited thereto and is set or filtered to filter a motion compensation frame. It may be a block having a shape and size determined according to a criterion. In addition, the filtering block is referred to for convenience of description, but may not necessarily be the same as the block but may be an area having various shapes such as a circle, a rhombus shape, and a triangle.

10 is a flowchart for exemplarily describing a process of determining whether to filter the entire motion compensation frame or selectively filter the motion compensation frame for each filtering block based on an encoding cost according to another embodiment of the present invention. .

The block-by-block filtering image encoding apparatus calculates filter coefficients for filtering the motion compensation frame (S1010), filters the entire motion compensation frame using the calculated filter coefficients (S1020), and filters the filtering blocks by the filtering blocks of the motion compensation frame. In operation S1030, the motion compensation frame is selectively filtered for each filtering block according to the determination of operation S1030. The filtering identification information determined for each filtering block according to operation S1040 is encoded using a hierarchical block encoding method. In operation S 1060, a filtering method is selected based on an encoding cost when the entire motion compensation frame is filtered and an encoding cost when the filtering is selectively filtered for each filtering block (S 1060). In this case, various cost functions such as rate-distortion cost and the like may be used as a cost function that may be used to calculate the encoding cost.

11 is a flowchart illustrating a process of determining whether to perform filtering for each filtering block according to another embodiment of the present invention. In this case, the filtering block may be set to a block having an arbitrary size, and there is no restriction on setting to a basic block unit of encoding.

FIG. 11 exemplarily illustrates a process of determining whether a filtering image encoding apparatus for each block determines whether to filter the filtering block for a motion compensation frame in operation S1030.

The filter coefficients for each filtering block are already calculated before determining whether to perform filtering on each filtering block of the motion compensation frame. In FIG. 11, j represents an index for identifying each filtering block in the motion compensation frame, j _end represents the total number of filtering blocks in the motion compensation frame, Denotes filtering identification information indicating whether to filter the j th filtering block.

Denotes the encoding cost when the filtering block is not filtered.

Denotes an encoding cost when filtering on a filtering block. In this case, a cost function that may be used to calculate the encoding cost may be used, such as a sum of squared difference (SSD), a sum of absolute difference (SAD), a sum of absolute transformed difference (SATD), and the like.

For example, if it is calculated on the basis of SAD, the coding cost without filtering on any filtering block

Can be calculated as shown in Equation 11, and the coding cost when filtering on any filtering block

May be calculated as shown in Equation 12.

[Equation 11]

[Equation 12]

In Equation 11, K and L represent the horizontal size and the vertical size of each filtering block,

Wow

Denotes the coordinates of the upper left of the j th filtering block, and k and l denote the coordinates of each pixel in the filtering block.

Represents the pixels of the current frame,

Represents the pixels of the motion compensation frame,

Represents the pixels of the filtered motion compensation frame.

The filtering image encoding apparatus for each block sets the 0 th filtering block of the motion compensation frame as the initial filtering block (S1110), and the encoding cost when the 0 th filtering block is not filtered.

Is calculated (S1120), and the coding cost when filtering on the 0 th filtering block

(S1130), the encoding cost

Wow

By comparing

end

It is determined whether it is larger (S1140). The block-filtering image encoding apparatus may determine the result of operation S1140.

end

Filtering identification for the 0th filtering block, if less than or equal to

Set to 0 to determine not to filter for the 0 th filtering block (S1150),

end

If greater than, the filtering identification information for the zeroth filtering block.

Set to 1 to determine to filter on the 0 th filtering block (S1160). Thereafter, the block-by-block filtering image encoding apparatus determines whether the filtering block that determines the filtering identification information is the last filtering block of the motion compensation frame (S1170), and if it is not the last filtering block, increases the j by 1 to the first filtering block. Steps S1120 to S1170 are repeatedly performed (S1180), and in the case of the last filtering block, the process of determining whether to perform filtering is finished.

12 is a diagram for exemplarily illustrating a process of encoding filtering identification information for each block. In the upper block step, split identification information on whether the current block is split and the filtering identification information is encoded or whether the current block is filtered or not is determined using one filtering identification information. When the filtering identification information is transmitted without segmentation, the filtering identification information is encoded without further encoding the partition identification information. In the case of splitting and encoding the filtering identification information, the split identification information is encoded by further splitting and encoding the split identification information to encode the filtering identification information or to encode the filtering identification information without further splitting. If the current partition layer is the maximum partition layer, the encoding process of the filtering identification information is terminated by encoding only the filtering identification information without encoding the partition identification information. In this way, the filtering identification information of all blocks of the current frame is encoded.

An image decoding apparatus according to another embodiment of the present invention is configured similarly or similarly to the image decoding apparatus 600 according to an embodiment of the present invention described above with reference to FIG. 6. That is, an image decoding apparatus according to another embodiment of the present invention is a decoder for decoding motion information data and image coded data extracted from a bitstream and restoring motion information, transformed, and quantized residual frames, reconstructed transformed and quantized An inverse quantizer and an inverse transformer to inverse quantize and inversely transform the residual frame, and generate a motion compensation frame by compensating for the motion of the current frame based on the reconstructed motion information. And a predictor for filtering to generate a filtered motion compensation frame and an adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame.

The inter prediction apparatus for image decoding according to another embodiment of the present invention is configured similarly or similarly to the inter prediction apparatus for image decoding according to an embodiment of the present invention described above with reference to FIG. 7. That is, the inter prediction apparatus for image decoding according to another embodiment of the present invention may be implemented as a predictor of the image decoding apparatus according to another embodiment of the present invention, and is based on motion information extracted, decoded, and reconstructed from a bitstream. A motion compensator for generating a motion compensation block and generating a motion compensation frame including the motion compensation block, and a motion compensation filter for selectively filtering the motion compensation frame for each filtering block.

An image decoding apparatus and an inter prediction apparatus for image decoding according to another embodiment of the present invention will be described in detail through an image decoding method according to another embodiment of the present invention in a later process. Hereinafter, for convenience of description, an image decoding apparatus according to another embodiment of the present invention is called a filtered image decoding apparatus for each block.

According to an image decoding method according to another embodiment of the present invention, the block-by-block filtering image decoding apparatus generates a motion compensation frame by compensating for the motion of the current frame based on motion information reconstructed by decoding the bitstream (S1310), The filtered motion compensation frame is selectively filtered for each filtering block to generate a filtered motion compensation frame (S1320), and the current frame is restored by adding the residual frame to be recovered by decoding the bitstream and the filtered motion compensation frame (S1330).

In operation S1320, the block-by-block filtering image decoding apparatus may selectively filter the motion compensation frame for each filtering block based on the block-by-block filtering application information reconstructed by decoding the bitstream. The filtering identification information for each filtering block is decoded according to the partition identification information of the filtering block according to the hierarchical structure, and the filtering identification information is decoded in the step of no further partitioning. That is, the block-by-block filtering image decoding apparatus decodes the bitstream to restore block-specific filtering identification information, and whether the entire motion compensation frame should be filtered based on the restored block-by-block filtering application information or filters the motion compensation frame. It is determined whether or not to selectively filter by each filter according to the determination result or the motion compensation frame is selectively filtered by the filtering block. If the image encoding apparatus and the image decoding apparatus according to another embodiment of the present invention promise to selectively filter the motion compensation frame for each filtering block in advance or to promise to filter the entire motion compensation frame, the application information for each block is applied. You will not need to restore it.

In operation S1320, the block-by-block filtered image decoding apparatus may select a filtering block to filter from among the filtering blocks of the motion compensation frame based on the filtering identification information reconstructed by decoding the bitstream. That is, when selectively filtering the motion compensation frame for each filtering block, the block-by-block filtering image decoding apparatus decodes the bitstream to restore filtering identification information, and performs filtering only on the filtering block identified by the filtering identification information.

In operation S1320, the block-by-block filtered image decoding apparatus may filter using a different filter or different filter coefficients for each filtering block or for each pixel position of the filtering block. That is, the block-by-block filtering image decoding apparatus may restore information about which filter or which filter coefficient is used from the bitstream, and different filters or different filters for each filtering block or for each pixel position of the filtering block according to the restored information. Filter using filter coefficients. If the image encoding apparatus and the image decoding apparatus according to another embodiment of the present invention promise to filter by using a predetermined filter or a predetermined filter coefficient at a position of any block or any pixel in advance, any filter or any filter There would be no need to recover information about whether to use the coefficients.

In operation S1320, the block-by-block filtered image decoding apparatus may determine, as a filtering block, a block having a size identified by the filtering block identification information reconstructed by decoding the bitstream. That is, when the motion compensation frame is selectively filtered for each filtering block, the block-by-block filtering image decoding apparatus identifies the filtering block from the bitstream in order to filter by a block unit having the same size as the filtering block filtered by the block-by-block filtering image encoding apparatus. The information is restored and the block of the size and / or shape identified by the filtering block identification information to be restored is set as the filtering block and filtered.

The filtered motion compensation frame generated as described above may include an unfiltered filtering block and a filtered filtering block of the motion compensation frame. That is, when the motion compensation frame is selectively filtered for each filtering block, a motion compensation frame including both the unfiltered filtering block and the filtered filtering block of the motion compensation frame is generated as the filtered motion compensation frame.

Meanwhile, the image encoding / decoding apparatus according to an embodiment of the present invention uses the encoded data (bitstream) output terminal of the image encoding apparatus 100 of FIG. 1 to encode data (bitstream) of the image decoding apparatus 600 of FIG. 6. This can be achieved by connecting to the input.

An image encoding / decoding apparatus according to an embodiment of the present invention generates a filtered motion compensation frame by selectively filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame for each filtering block, and filtering the current frame with the current frame. An image encoder for generating, transforming, quantizing, and encoding a residual frame by subtracting a motion compensation frame; And restoring motion information, transformed and quantized residual frames by decoding motion information data and image encoded data extracted from the bitstream, and restoring the remaining frames by inverse quantization and inverse transform. A video decoder to generate a motion compensation frame by selectively compensating the signal, and selectively filter the motion compensation frame by the filtering block to generate a filtered motion compensation frame, and add the reconstructed residual frame and the filtered motion compensation frame to restore the current frame. do.

The image encoder may be implemented by the image encoding apparatus 100 according to an embodiment of the present invention, and the image decoder may be implemented by the image decoding apparatus 600 according to an embodiment of the present invention.

An image encoding / decoding method according to an embodiment of the present invention may be realized by combining the image encoding method according to an embodiment of the present invention and the image decoding method according to an embodiment of the present invention.

In an image encoding / decoding method according to an embodiment of the present invention, in a method of encoding / decoding an image, a motion compensation frame is generated by estimating and compensating a motion of a current frame of the image and filtering the motion compensation frame. A video encoding step of selectively filtering the respective motions to generate a filtered motion compensation frame, and converting, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame; And generating a motion compensation frame by compensating for the motion of the current frame based on the motion information reconstructed by decoding the bitstream, and selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame. And a video decoding step of reconstructing the current frame by adding the residual frame to be decoded and the filtered motion compensation frame.

The image encoding step may be implemented by an image encoding method according to an embodiment of the present invention, and the image decoding step may be implemented by an image decoding method according to an embodiment of the present invention.

As described above, according to another embodiment of the present invention, in predictive encoding of an image, a compression efficiency may be further improved by removing temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation. In addition, reconstruction can be further improved by filtering only on blocks that are more efficient to filter in the motion compensation frame and not on blocks that are more efficient by not filtering, thereby further improving compression efficiency or decoding and playing after being encoded. The deterioration of the image quality of the image can be further reduced.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

As described above, according to an exemplary embodiment of the present invention, the present invention is applied to an image compression processing field for encoding and decoding a video, thereby further eliminating temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation to further improve compression efficiency. It is a very useful invention that produces an effect that can be improved.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is related to US Patent Application No. 119 for Patent Application No. 10-2009-0123538 filed in Korea on December 11, 2009 and Patent Application No. 10-2010-0126443 for Korea filed in December 10, 2010. If priority is claimed under section (a) (35 USC § 119 (a)), all of that is incorporated by reference into this patent application. In addition, if this patent application claims priority for the same reason as above for a country other than the United States, all the contents thereof are incorporated into this patent application by reference.

Claims

An apparatus for encoding / decoding a video,

Selectively filtering motion compensation frames generated by estimating and compensating for motion of a current frame to generate a filtered motion compensation frame by subtracting the current frame and the filtered motion compensation frame to generate a residual frame, converting, and An image encoder for quantizing and encoding; And

Decodes the motion information data and the image coded data extracted from the bitstream to restore the motion information, the residual and quantized residual frames, and inversely quantizes and inversely transforms the residual frames, and restores the current frame based on the restored motion information. To generate a motion compensation frame and selectively filter the motion compensation frame for each filtering block to generate a filtered motion compensation frame, and add the reconstructed residual frame and the filtered motion compensation frame to restore a current frame. Decoder

Image encoding / decoding apparatus comprising a.
In the apparatus for encoding a video,

A predictor configured to selectively filter a motion compensation frame generated by estimating and compensating for a motion of the current frame of the image for each filtering block to generate a filtered motion compensation frame;

A subtractor for generating a residual frame by subtracting the current frame and the filtered motion compensation frame;

A converter and quantizer for transforming and quantizing the residual frame; And

An encoder for encoding the transformed and quantized residual frames;

An image encoding apparatus comprising a.
The method of claim 2, wherein the predictor is

The motion compensation frame is selectively filtered for each filtering block when the coding cost for the motion compensation frame selectively filtered for each filtering block is smaller than the coding cost for the motion compensation frame in which the motion compensation frame is entirely filtered. An image encoding device.
The method of claim 2, wherein the predictor is

Image encoding, characterized in that it is determined whether the filtering cost for filtering the filtering block for each filtering block of the motion compensation frame exceeds the coding cost for not filtering the filtering block and for each filtering block. Device.
The method of claim 2, wherein the filtered motion compensation frame,

And an unfiltered filtering block and a filtered block of the motion compensation frame.
The method of claim 2, wherein the predictor is

And filtering by using different filters or different filter coefficients for each of the filtering blocks or for each pixel position of the filtering block.
The method of claim 2, wherein the encoder,

And encoding filtering information indicating whether the filtering block is filtered by the filtering block or by a bundle or a combination of the filtering blocks.
The method of claim 2, wherein the encoder,

And the filtering block is configured to encode filtering identification information using a basic coding block.
The method of claim 2, wherein the encoder,

And encoding the filtering block by using the segmentation identification information and the filtering identification information by using a hierarchical filtering block structure.
The method of claim 2, wherein the encoder,

And encoding block identification information indicating a size or a unit of the filtering block.
In the apparatus for decoding an image,

A decoder configured to decode the motion information data and the image encoded data extracted from the bitstream and to restore the motion information and the residual and quantized frames;

An inverse quantizer and an inverse converter that inverse quantizes and inverse transforms the reconstructed transformed and quantized residual frame to reconstruct the residual frame;

A predictor for generating a motion compensation frame by compensating for the motion of the current frame based on the reconstructed motion information and selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame; And

An adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame;

Video decoding apparatus comprising a.
The method of claim 11, wherein the predictor,

And selectively filtering the motion compensation frame for each filtering block based on the filtering application information for each block, which is reconstructed by decoding the bitstream.
The method of claim 11, wherein the predictor,

And a filtering block to be filtered among the filtering blocks of the motion compensation frame based on the filtering identification information reconstructed by decoding the bitstream.
The method of claim 11, wherein the filtered motion compensation frame,

And an unfiltered filtering block and a filtered filtering block of the motion compensation frame.
The method of claim 11, wherein the predictor,

And filtering using different filters or different filter coefficients for each of the filtering blocks or for each pixel position of the filtering block.
The method of claim 11, wherein the predictor,

And determining, as the filtering block, a block having a size identified by the filtering block identification information reconstructed by decoding the bitstream.
In the inter prediction apparatus,

A motion estimator for determining motion information for a block of an image;

A motion compensator for generating a motion compensation block based on the determined motion information and generating a motion compensation frame including the motion compensation block; And

A motion compensation filter for selectively filtering the motion compensation frame for each filtering block;

Inter prediction apparatus comprising a.
In the inter prediction apparatus,

A motion compensator for generating a motion compensation block on the basis of motion information extracted from the bitstream, decoded and reconstructed, and generating a motion compensation frame including the motion compensation block; And

A motion compensation filter for selectively filtering the motion compensation frame for each filtering block;

Inter prediction apparatus comprising a.
In the method of encoding / decoding an image,

A motion compensation frame is generated by estimating and compensating the motion of the current frame of the image, and selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame, the difference between the current frame and the filtered motion compensation frame. An image encoding step of transforming, quantizing, and encoding the remaining frame; And

Compensate the motion of the current frame based on the motion information reconstructed by decoding the bitstream to generate a motion compensation frame, and selectively filter the motion compensation frame for each filtering block to generate a filtered motion compensation frame and decode the bitstream. Image decoding step of reconstructing the current frame by adding the residual frame to be restored and the filtered motion compensation frame

Image encoding / decoding method comprising a.
In the method of encoding an image,

Generating a motion compensation frame by estimating and compensating for the motion of the current frame of the image;

Selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame; And

Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame;

Image encoding method comprising a.
The method of claim 20, wherein the video encoding method comprises:

And encoding the filtering application information for each block indicating whether the motion compensation frame is selectively filtered for each filtering block.
The method of claim 20, wherein generating the filtered motion compensation frame comprises:

The motion compensation frame is selectively filtered for each filtering block when the coding cost for the motion compensation frame selectively filtered for each filtering block is smaller than the coding cost for the filtered motion compensation frame as a whole. A video encoding method.
The method of claim 20, wherein generating the filtered motion compensation frame comprises:

Image encoding, characterized in that it is determined whether the filtering cost for filtering the filtering block for each filtering block of the motion compensation frame exceeds the coding cost for not filtering the filtering block and for each filtering block. Way.
The method of claim 20, wherein the filtered motion compensation frame,

And an unfiltered filtering block and a filtered block of the motion compensation frame.
The method of claim 20, wherein generating the filtered motion compensation frame comprises:

And filtering using different filters or different filter coefficients for each of the filtering blocks or for each pixel position of the filtering block.
The method of claim 20, wherein the video encoding method comprises:

And encoding the filtering identification information indicating whether the filtering block is filtered for each of the filtering blocks.
The method of claim 20, wherein the video encoding method comprises:

And encoding filtering identification information indicating whether to filter by the filtering block or by a bundle or a combination unit of the filtering blocks.
The method of claim 20, wherein the video encoding method comprises:

And the filtering block further comprises encoding filtering identification information using a basic coding block.
The method of claim 20, wherein the video encoding method comprises:

And encoding whether or not the filtering block is encoded using the segmentation identification information and the filtering identification information by using the hierarchical filtering block structure.
The method of claim 20, wherein the video encoding method comprises:

And encoding the filtering block identification information indicating the size or unit of the filtering block.
In the method of decoding an image,

Generating a motion compensation frame by compensating for the motion of the current frame based on the motion information reconstructed by decoding the bitstream;

Selectively filtering the motion compensation frame for each filtering block to generate a filtered motion compensation frame; And

Restoring the current frame by adding the filtered motion compensation frame and the residual frame to be recovered by decoding the bitstream;

Image decoding method comprising a.
32. The method of claim 31, wherein generating the filtered motion compensation frame comprises:

And selectively filtering the motion compensation frame for each filtering block based on the block-by-block filtering application information reconstructed by decoding the bitstream.
32. The method of claim 31, wherein generating the filtered motion compensation frame comprises:

And a filtering block to be filtered from among the filtering blocks of the motion compensation frame based on the filtering identification information reconstructed by decoding the bitstream.
The method of claim 31, wherein the filtered motion compensation frame,

And an unfiltered filtering block and a filtered filtering block of the motion compensation frame.
32. The method of claim 31, wherein generating the filtered motion compensation frame comprises:

And filtering using different filters or different filter coefficients for each of the filtering blocks or for each pixel position of the filtering block.
32. The method of claim 31, wherein generating the filtered motion compensation frame comprises:

And determining a block having a size identified by the filtering block identification information reconstructed by decoding the bitstream as the filtering block.
37. The method of claim 36, wherein restoring the filtering identification information comprises:

Filtering in units of a filtering block that is recovered by decoding the bitstream And a filtering unit to be used for decoding among the frame unit and the filtering block unit based on the identification information.
38. The method of claim 37, wherein the restoring the filtering identification information in units of the filtering block includes:

And reconstructing size information of the filtering block by the filtering block size information reconstructed by decoding the bitstream.
38. The method of claim 37, wherein the restoring the filtering identification information in units of the filtering block includes:

And reconstructing identification information of the filtering block on a basic coding block basis to determine whether to filter.
38. The method of claim 37, wherein the restoring the filtering identification information in units of the filtering block includes:

And determining whether to filter filtering blocks having a hierarchical structure in consideration of various filtering block sizes based on the filtering block identification information reconstructed by decoding the bitstream.
In the inter prediction method,

Determining motion information for a block of an image;

Generating a motion compensation block based on the determined motion information and generating a motion compensation frame including the motion compensation block; And

Selectively filtering the motion compensation frame for each filtering block;

Inter prediction method comprising a.
In the inter prediction method,

Generating a motion compensation block based on the motion information extracted from the bitstream, decoded and reconstructed, and generating a motion compensation frame including the motion compensation block; And

Selectively filtering the motion compensation frame for each filtering block;

Inter prediction method comprising a.