KR101379189B1 - Video Coding Method and Apparatus by Using Filtering Motion Compensation Frame - Google Patents

Abstract

The present invention relates to a method and apparatus for image encoding / decoding using filtering of a motion compensation frame.

The present invention provides a video encoding method comprising filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image, and converting, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame. to provide.

According to the present invention, since the temporal redundancy remaining in the remaining frame can be removed by filtering the motion compensation frame, the compression efficiency of the video can be improved.

Image, Encoding, Decoding, Compression, Motion Compensation, Frame, Filtering

Description

Video Coding Method and Apparatus by Using Filtering Motion Compensation Frame

The present invention relates to a method and apparatus for image encoding / decoding using filtering of a motion compensation frame. More particularly, the present invention relates to a method and apparatus for improving motion picture compression efficiency by filtering motion compensation frames generated through motion estimation and motion compensation in a moving picture encoding and decoding process.

In order to efficiently store or transmit moving image data, it is necessary to encode and compress the data. Techniques for compressing video data include H.261, H.263, H.264, MPEG-2, and MPEG-4. In the compression technique of moving picture data, luminance components and chrominance components of each frame image of the moving picture are temporally predicted through motion estimation and motion compensation, and prediction residuals are transformed, quantized, and entropy-encoded to be transmitted in bit stream form.

H.264 / AVC, the latest video compression technology, was developed jointly by the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG). Macroblocks are 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, Motion estimation and motion compensation are performed by dividing into smaller blocks having a size of 4x4. In this case, up to 16 frames may also be used for the reference frame for motion estimation and motion compensation. In addition, by adopting the 1/4 pixel interpolation method in H.264 / AVC, it is possible to improve the compression ratio while maintaining the image quality when compared with MPEG-4, which is a conventional moving picture compression technique.

As described above, in the conventional moving picture compression technique, a high compression ratio can be achieved by eliminating temporal redundancy in an image by using motion estimation and motion compensation. However, in a motion compensation frame generated through motion estimation and motion compensation, There is a problem in that redundancy is much left, and prediction residuals that are transformed, quantized, and entropy-encoded become large, and thus a large amount of residual data must be coded.

In order to solve the above problems, the present invention is to improve the compression efficiency of a video by filtering a motion compensation frame generated through motion estimation and motion compensation and encoding the remaining frames during video encoding and decoding. have.

In order to achieve the above object, the present invention provides a method of encoding an image, the method comprising: filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image; And transforming, quantizing, and encoding the remaining frame that is a difference between the current frame and the filtered motion compensation frame.

According to another object of the present invention, there is provided an apparatus for encoding an image, comprising: a predictor for filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image; A subtractor for generating a residual frame by subtracting the current frame and the filtered motion compensation frame; A converter and quantizer for transforming and quantizing residual frames; And an encoder for encoding the transformed and quantized residual frames.

According to still another object of the present invention, in a method of decoding an image, the current frame is obtained by using a motion vector and a reference frame index identified by the motion information reconstructed by decoding motion information data extracted from the bitstream. Compensating for motion to generate a motion compensation frame and filtering the motion compensation frame; And reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted and decoded from the bitstream to recover the current frame.

According to still another aspect of the present invention, there is provided an apparatus for decoding an image, comprising: a decoder for decoding motion information data and image encoded data extracted from a bitstream and restoring motion information, transformed and quantized residual frames; An inverse quantizer and an inverse converter that inverse quantizes and inverse transforms the reconstructed transform and quantized residual frames to recover the residual frames; A predictor for generating a motion compensation frame by compensating for the motion of the current frame by using the motion vector identified by the reconstructed motion information and the reference frame index, and filtering the motion compensation frame; And an adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame.

According to still another object of the present invention, there is provided an inter prediction method comprising: determining a motion vector and a reference frame index for a block in which a block mode is an inter mode within a current frame of an image; Generating a motion compensation block using the determined motion vector and the reference frame index and generating a motion compensation frame including the motion compensation block; And filtering the motion compensation frame.

According to still another object of the present invention, there is provided an inter prediction apparatus comprising: a motion estimator for determining a motion vector and a reference frame index for a block in which a block mode is an inter mode within a current frame of an image; A motion compensator for generating a motion compensation block using the determined motion vector and the reference frame index and generating a motion compensation frame including the motion compensation block; And a motion compensation filter for filtering the motion compensation frame.

In addition, according to another object of the present invention, in the inter prediction method, a motion compensation block is generated by using a motion vector and a reference frame index identified by motion information extracted from the bitstream, and decoded from the bitstream. Generating a motion compensation frame comprising a; And filtering the motion compensation frame.

According to still another object of the present invention, in an inter prediction apparatus, a motion compensation block is generated by using a motion vector and a reference frame index identified by motion information extracted from a bitstream, decoded, and recovered. A motion compensator for generating a motion compensation frame comprising a; And a motion compensation filter for filtering the motion compensation frame.

As described above, according to the present invention, since the temporal redundancy remaining in the remaining frames can be further removed by filtering the motion compensation frame, the compression efficiency of the video can be improved.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. It should be noted that, in adding reference numerals to the constituent elements of the drawings, the same constituent elements are denoted by the same reference symbols as possible even if they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In describing the components of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are intended to distinguish the constituent elements from other constituent elements, and the terms do not limit the nature, order or order of the constituent elements. When a component is described as being "connected", "coupled", or "connected" to another component, the component may be directly connected or connected to the other component, Quot; may be "connected," "coupled," or "connected. &Quot;

A video encoding apparatus (Video Encoding Apparatus), a video decoding apparatus (Video Decoding Apparatus) to be described below is a personal computer (PC), notebook computer, personal digital assistant (PDA), portable multimedia player (PMP) : User terminal such as Portable Multimedia Player (PSP), PlayStation Portable (PSP: PlayStation Portable), Mobile Communication Terminal (Mobile Communication Terminal), or a server terminal such as an application server or a service server, and communicates with various devices or wired / wireless communication networks. The present invention refers to various devices including a communication device such as a communication modem, a memory for storing various programs and data for encoding or decoding an image, a microprocessor for executing and operating a program, and the like.

In addition, the image encoded in the bitstream by the video encoding apparatus is real-time or non-real-time through the wired or wireless communication network, such as the Internet, local area wireless communication network, wireless LAN network, WiBro network, mobile communication network, or the like, or a cable, universal serial bus (USB: Universal) It may be transmitted to an image decoding apparatus through various communication interfaces such as a serial bus, and may be decoded by the image decoding apparatus to restore and reproduce the image.

Usually, a moving picture is composed of a series of pictures, and each picture is divided into a predetermined area such as a block. In the case where an image area is divided into blocks, the divided blocks are classified into an intra block and an inter block according to a coding method. The intra-block refers to a block that is coded using Intra Prediction Coding (P-Coding) scheme. The intra-prediction coding is performed by using the pixels of previously decoded and decoded blocks in the current picture, A prediction block is generated by predicting the pixels of the block and a difference value between the pixel of the current block and the pixel of the current block is encoded. Inter-block refers to a block that is coded using Inter Prediction Coding. Inter-prediction coding refers to one or more past pictures or a future picture to generate a prediction block by predicting a current block in the current picture, And the difference value is encoded. Here, a frame referred to for encoding or decoding the current picture is referred to as a reference frame.

1 is a block diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus 100 according to an embodiment of the present invention is an apparatus for encoding an image, and includes a predictor 110, a subtractor 120, a transformer and quantizer 130, and an encoder. The encoder may include an encoder 140, an inverse quantizer and an inverse transformer 150, an adder 160, a deblocking filter 170, and a memory 180.

The predictor 110 filters motion compensation frames generated by estimating and compensating a motion of a frame to be encoded (hereinafter, referred to as 'current frame') in the input image. That is, the predictor 110 estimates the motion of each block in the current frame according to an inter prediction mode in a reference frame stored in the memory 180 after being encoded, decoded, And generates a motion compensation frame compensated for the motion of the current frame by compensating the motion of each block using the motion vector, and filters the generated motion compensation frame.

For this purpose, the predictor 110 determines a motion vector and a reference frame index for a block whose block mode is the inter mode in the current frame, and uses the determined motion vector and the reference frame index A motion compensation block is generated, and a motion compensation frame including the generated motion compensation block is generated to filter the motion compensation frame.

Here, the predictor 110 may filter a motion compensation frame using another filter according to the characteristics of a block in a motion compensation frame, and may use a Winner Filter or a Deblocking Filter as a motion compensation frame, And the like may be used for filtering.

Also, the predictor 110 may filter the motion compensation frame using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame.

The subtractor 120 subtracts the current frame and the filtered motion compensation frame to generate a residual frame. Herein, the residual frame refers to a frame including a residue that is generated by subtracting a current frame from a filtered motion compensation frame, and includes a residual signal that is a difference between a pixel of the current frame and a pixel of the filtered motion compensation frame.

The transformer and quantizer 130 transforms and quantizes the residual frames. That is, the transformer and the quantizer 130 convert the residual signal of the residual frame generated by the subtractor 120 into the frequency domain to generate a transformed residual frame having a transform coefficient, And quantizes the transform coefficients of the remaining frames to generate transformed and quantized residual frames. In this case, as a conversion method used, a technique of converting an image signal of a spatial domain into a frequency domain such as a Hadamard Transform and a Discrete Cosine Transform Based Integer Transform is used, Various quantization techniques such as Dead Zone Uniform Threshold Quantization (DZUTQ) or a Quantization Weighted Matrix may be used.

The encoder 140 encodes the transformed and quantized residual frames. That is, the quantized transform coefficient of the residual frame converted and quantized by the transformer and the quantizer 130 is encoded to generate image encoded data. The generated image coded data is included in the bit stream. As such an encoding technique, entropy encoding technology may be used, but various other encoding techniques may be used without being limited thereto.

Also, the encoder 140 may generate filter coefficient data by encoding the filter coefficient information indicating the filter coefficient of the filter used for filtering the motion compensation frame. The generated filter coefficient data is included in the bitstream.

Also, the encoder 140 may generate block mode data by encoding block mode information indicating a block mode for each block in the current frame, and the generated block mode data is included in the bitstream.

In addition, the encoder 140 may generate motion information data by encoding a motion vector determined for a block in which a block mode is an inter mode in a current frame and motion information indicating a reference frame index, and the generated motion information data is And is included in the bitstream.

Meanwhile, the functions of the transformer and the quantizer 130 may be integrated into the encoder 140. FIG. That is, when the encoder is implemented by one encoder, the encoder encodes the residual frame to generate encoded data.

Inverse quantizer and inverse transformer 150 inverse quantizes and inverse transforms the transformed and quantized residual frame to recover the residual frame. That is, the inverse quantizer and the inverse transformer 150 inverse quantizes the transformed and quantized residual frames transmitted from the transformer and the quantizer 130 to restore the residual frame having the transform coefficients, and the residual frame having the transform coefficients. Inverse conversion is performed again to restore the residual frame with the residual signal. In this case, the inverse quantizer and the inverse transformer 150 may restore the remaining frame by performing the transformed and quantized method in the inverse of the transformer and the quantizer 130.

The adder 160 adds the residual frame to be restored and the filtered motion compensation frame to reconstruct the current frame.

The deblocking filter 170 deblock filters the current frame to be restored, and the memory 180 stores the deblocking filtered current frame. Here, deblocking filtering refers to a task of reducing block distortion caused by coding an image block by block. The deblocking filter may be applied to a block boundary or a macro block boundary, a deblocking filter may be applied to a macro block boundary, Can be selectively used.

Although not shown in FIG. 1, the image encoding apparatus 100 according to an embodiment of the present invention may further include an intra predictor for Intra prediction in the predictor 110. FIG. In this case, the subtractor 120 may generate a residual frame by subtracting the current frame and the prediction frame generated by the intra predictor, and the transformer and quantizer 130 and the inverse quantizer and the inverse transformer 150 are residual frames. Calculation may be further performed for transform and quantization for and inverse transform and inverse quantization for the transformed and quantized residual frame. In addition, the encoder 130 may generate the image encoded data by encoding the transformed and quantized residual frame, which is included in the bitstream.

Therefore, in the exemplary embodiment of the present invention, only the remaining frame, which is a difference between the current frame and the filtered motion compensation frame, is transformed, quantized, and encoded, so that the image encoded data according to inter prediction encoding is included in the bitstream. The apparatus 100 may further generate image coded data to be intra predictively coded, and in the bitstream, image coded data generated by intra predictive coding as well as image coded data generated by inter predictive coding and the image coded data. It will be appreciated that information for intra prediction decoding the data or its encoded data may be further included.

2 is a block diagram schematically illustrating an inter prediction apparatus for image encoding according to an embodiment of the present invention.

The inter prediction apparatus for image coding according to an embodiment of the present invention can be implemented by the predictor 110 in the image coding apparatus 100 according to an embodiment of the present invention described above with reference to FIG. It is referred to as a predictor 110 for convenience.

The predictor 110 may include a motion estimator 210, a motion compensator 220, and a motion compensation filter 230. In addition, the predictor 110 may further include an intra predictor, but is not separately shown.

The motion estimator 210 determines a motion vector and a reference frame index for a block in which the block mode is inter mode in the current frame. That is, the motion estimator 210 estimates the motion of the blocks in which the block mode is the inter mode among the blocks in the current frame to determine the motion vector and the reference frame index. Here, the block mode for each block in the current frame may be determined by the motion estimator 210, but may be determined by a mode determiner (not shown) additionally provided in the predictor 110 or the image encoding apparatus 100 have.

For this, the motion estimator 210 estimates a forward motion vector based on a previous frame of the current frame by performing forward prediction on a block in which the block mode is a P-macroblock in the current frame, A bidirectional prediction is performed on a block in which a block mode is a B macroblock (Bidirectional Macroblock) to estimate a forward motion vector and a backward motion vector based on a previous frame and a subsequent frame of the current frame, respectively.

In this case, the motion estimator 210 may determine a motion vector and a reference frame index that minimize the coding cost as a motion vector and a reference frame index for the current block in the current frame. As described above, the mcost function can be used as a function for calculating a coding cost for determining a motion vector and a reference frame index. The mcost function can be expressed as Equation (1).

Here, λ is a Lagrange Multiplier for determining the motion vector and the reference frame index, and rate _{u, v, i} are predictive coding by compensating the motion of the block using the motion vector and the reference frame index. To indicate the bit rate of the generated bit stream. SAD _{u, v, i} represents the sum of squares of the difference between the current frame and the motion compensation frame generated by compensating for the motion of the corresponding block using the motion vector and the reference frame index, and can be expressed by Equation 2.

는 현재 프레임의 (x+m,y+n) 좌표에 대한 화소값을 의미하고,

는 i번째 참조 프레임의 (x+m+u,y+n+v) 좌표에 대한 화소값을 나타낸다. 또한, x와 y는 해당 블록의 제일 왼쪽의 제일 위쪽에 위치한 화소의 현재 프레임 또는 움직임 보상 프레임 내에서의 좌표를 나타내며, m와 n은 해당 블록의 내부에 위치한 화소의 좌표를 나타낸다. u와 v는 해당 블록의 움직임 벡터를 나타낸다.In Equation 2, M and N respectively represent the pixel size in the horizontal direction and the vertical direction of the block,

Is the pixel value for the ( x + m, y + n ) coordinate of the current frame,

Denotes a pixel value for ( x + m + u, y + n + v ) coordinates of the i th reference frame. In addition, x and y represent the coordinates in the current frame or the motion compensation frame of the pixel located at the top left of the block, m and n represents the coordinates of the pixel located inside the block. u and v represent the motion vectors of the block.

The motion estimator 210 calculates mcost for each motion vector and reference frame index for each block whose block mode is interblock in the current frame by using a function for calculating a coding cost as shown in Equation 1, A motion vector and a reference frame index for minimizing the encoding cost are determined as a motion vector and a reference frame index of the corresponding block by using a function as shown in FIG. 3.

는 각각 해당 블록에 대해 결정된 움직임 벡터의 가로 성분과 세로 성분을 내타내며,

는 해당 블록에 대해 결정된 참조 프레임 색인을 나타낸다.In Equation (3), SearchRange represents a range for performing motion estimation,

Represents the horizontal and vertical components of the motion vector determined for that block,

Denotes a reference frame index determined for the block.

The motion compensator 220 generates a motion compensation block using the determined motion vector and the reference frame index, and generates a motion compensation frame including the motion compensation block. That is, the motion compensator 220 generates a motion compensation block by compensating the motion of the corresponding block using the motion vector and the reference frame index determined by the motion estimator 210, and collects the generated motion compensation blocks on a frame basis And generates a motion compensation frame including motion compensation blocks. For this purpose, the motion compensator 220 takes a block indicated by the motion vector determined by the motion estimator 210 in the reference frame identified by the reference frame index determined by the motion estimator 210, And generates a motion compensation frame including these motion compensation blocks.

A method of generating a motion compensated frame in this manner can be expressed by Equation (4).

는 움직임 보상 프레임의 (x,y) 좌표에 대한 화소값을 의미한다. 만약, (x,y) 좌표에 의해 식별되는 화소를 포함하는 블록의 블록 모드가 인트라 모드인 경우에는 움직임 벡터와 참조 프레임 색인이 존재하지 않기 때문에 일단

는 값을 가지지 않는다. 이후, 하나의 움직임 보상 프레임 생성이 완료되면 미러링(Mirroring) 방식과 유사한 방식으로 앞에 내용을 채워 넣을 수도 있다.In Equation 4, W and H represent the horizontal and vertical pixel size of the frame (reference frame and motion compensation frame),

Denotes a pixel value for the ( x , y ) coordinate of the motion compensation frame. If the block mode of the block including the pixel identified by the ( x , y ) coordinate is the intra mode, since the motion vector and the reference frame index do not exist,

Does not have a value. Subsequently, when generation of one motion compensation frame is completed, the content may be filled in a manner similar to a mirroring method.

Referring to FIG. 3 illustrating a process of generating a motion compensation frame according to an exemplary embodiment of the present invention, four blocks among five blocks in a motion compensation frame are interblocked so that four reference frames Each block is filled with one block, but one block has no motion vector and reference frame index because the block mode is an intra block. When the motion compensation frame is later filtered, it is not possible to filter the neighboring pixels because the intra-mode block is not filled with values. Therefore, the motion compensation frame may be filtered using the pixel values of the neighboring pixels.

The motion compensation filter 230 filters the motion compensation frame. That is, the motion compensation filter 230 filters the pixels of the motion compensation frame generated by the motion compensator 220 using various types of filters so that the filtered motion compensation frame is more similar to the current frame, thereby transforming and quantizing. Thereby reducing the amount of data in the remaining frame to be encoded or reducing the blocking effect present in the motion compensation frame.

Here, the motion compensation filter 230 may use a predetermined filter when filtering a motion compensation frame, and may select a suitable filter or a coefficient of a filter in a predetermined unit such as a block unit, a slice unit, or a frame unit. When the motion compensation filter 230 uses a predetermined filter, the image encoding apparatus 100 and the image decoding apparatus to be described later can use a filter promised in advance or a filter according to a preset reference. Further, when the motion compensation filter 230 selects and uses an appropriate filter or a coefficient of the filter, another filter may be used depending on the characteristics of the pixel in the current frame.

As the filter for filtering the motion compensation frame, various filters such as a Wiener filter or a deblocking filter may be used. In the case of filtering the motion compensated frame using the Wiener filter, temporal redundancy remaining in the motion compensated frame can be further eliminated, so that the amount of data of the residual frame is reduced and accordingly the amount of the image encoded data is reduced. As a result, The compression efficiency can be improved. In addition, when the motion compensation frame is filtered using the deblocking filter, the blocking phenomenon in the motion compensation frame can be reduced, thereby improving the quality of the image which is decoded and then reconstructed.

When the motion compensation filter 230 selects and uses the coefficients of the appropriate filter or filter, the type of the filter is selected according to the characteristics of the pixels in the motion compensation frame, and the selected filter is used to filter the selected filter or the filter coefficient of the predetermined filter It is possible to filter by using the calculated filter coefficient or to select the type of the filter and to calculate and filter the filter coefficient of the selected filter.

For example, the motion compensation filter 230 determines the size of the motion compensation block in the motion compensation frame, and determines the size of the motion compensation block according to the size of the motion compensation block, May be filtered using different kinds of filters. In this case, the characteristics of the pixels in the motion compensation frame may be the size of the motion compensation block in the motion compensation frame. In addition, the motion compensation filter 230 determines the difference between the motion vector of the motion compensation blocks and the index of the reference frame, and determines whether a blocking phenomenon occurs in the boundary between the motion compensation blocks according to the difference between the indexes of the motion vector and the reference frame Pixels of a block where a blocking phenomenon occurs at a boundary between motion compensation blocks and pixels of a block where no blocking phenomenon occurs may be filtered using different filters. In this case, the characteristics of the pixels in the motion compensation frame may be the difference between the motion vector between the motion compensation blocks and the reference frame index.

The method of filtering the motion compensation frame by the motion compensation filter 230 may be expressed as Equation (5). In Equation 5, the case of filtering by using a two-dimensional filter is shown as an example, but may be filtered using a one-dimensional filter, depending on the implementation method or needs.

는 움직임 보상 프레임의 (x+m,y+n) 좌표에 의해 지시되는 화소의 화소값을 나타내고, β ^PP (m,n)은 (m,n) 번째 필터 계수를 나타내며, PP는 움직임 보상 프레임 내의 블록 내의 화소의 특성에 따라 선택되는 필터의 종류를 나타낸다.

은 필터링된 움직임 보상 프레임 내에서 (x,y) 좌표에 위치하는 화소의 화소값을 나타낸다.In Equation (5)

Denotes the pixel value of the pixel indicated by the ( x + m, y + n ) coordinate of the motion compensation frame, β ^PP ( m , n ) denotes the ( m, n ) th filter coefficients, and PP is the motion compensation frame The type of filter selected according to the characteristics of the pixels in the blocks within the blocks is shown.

Denotes a pixel value of a pixel located at ( x , y ) coordinates in the filtered motion compensation frame.

Referring to FIG. 4 illustrating a process of filtering a motion compensation frame according to an exemplary embodiment of the present invention, a motion compensation frame in which a blocking phenomenon occurs at a boundary between motion compensation blocks is filtered using a motion compensation filter In the filtered motion compensation frame, the blocking phenomenon can be eliminated at the boundary between the motion compensation blocks. 4 illustrates an example in which a two-dimensional deblocking filter is used as a motion compensation filter. In Fig. 4, * indicates a convolution.

In a typical moving picture compression technique, a motion compensation frame is used as a conceptual term because motion compensation is performed on a block basis. However, in an embodiment of the present invention, a motion compensation frame To video compression.

As an example of calculating a filter coefficient, the motion compensation filter 230 may calculate a filter coefficient for filtering a motion compensation frame using a Wiener filter so that the squared error between the current frame and the filtered motion compensation frame is minimized. Equation (6) is a mathematical expression for calculating a squared error between a current frame and a filtered motion compensation frame, and Equation (7) is a mathematical expression for calculating a filter coefficient that minimizes a squared error between a current frame and a filtered motion- Lt; / RTI >

는 현재 프레임의 (x,y) 좌표에 대한 화소 값을 나타내고,

는 (m,n) 번째 필터 계수를 나타내며,

는 필터링된 움직임 보상 프레임의 (x,y) 좌표에 대한 화소값을 나타내며 수학식 5를 통해 계산될 수 있다.In equations (6) and (7), ( e ^PP ) represents the squared error of the current frame and the filtered motion compensation frame,

Denotes the pixel value for the ( x, y ) coordinate of the current frame,

Represents the ( m , n ) th filter coefficient,

Denotes a pixel value for the ( x, y ) coordinate of the filtered motion compensation frame and may be calculated through Equation 5.

 The filter coefficients calculated through Equations (6) and (7) are used to filter the motion compensation frames and are transmitted to the encoder (140). At this time, the motion compensation filter 230 may filter the motion compensation frame using the calculated filter as it is, but may also filter the motion compensation frame by correcting the filter coefficient through a process such as quantization, up and down. For example, if the filter coefficient is calculated as a coefficient having a decimal point, since the data amount of the filter coefficient data becomes larger when the data is encoded as it is, the data is corrected by an integer coefficient through quantization, up and down, And transmits the corrected filter coefficient to the encoder 140. The encoder 140 decodes the motion compensation frame. The encoder 140 encodes the filter coefficient information indicating the filter coefficient to generate filter coefficient data, and the generated filter coefficient data may be included in the bitstream and transmitted to the image decoding apparatus. In this manner, the amount of data of the filter coefficient data can be reduced by correcting and encoding the filter coefficients.

The filtered motion compensation frame generated as described above is transmitted to the subtractor 120, and the subtractor 120 subtracts the current frame and the filtered motion compensation frame to generate a residual frame. At this time, the filtered motion compensation block can be used adaptively for various units such as a block or a slice unit. The residual frame includes a residual signal having a difference value between the pixel value of the pixel of the current frame and the pixel value of the pixel of the motion compensation frame. The residual signal can be expressed by Equation (8).

However, the residual signal generated by subtracting the current frame and the motion compensation frame according to a conventional moving picture compression scheme can be expressed by Equation (9).

In Equations 8 and 9, resi ( x , y ) represents the pixel value of the residual signal at the (x, y) coordinate. When the motion compensated filtering is performed using the Wiener filter, temporal redundancy remaining in the motion compensation frame is further removed, so that the residual signal of Equation (8) is smaller than the residual signal of Equation (9) When the compensation filtering is performed, the blocking phenomenon remaining in the motion compensation frame is further removed, and the residual signal of Equation (8) is transformed and quantized to improve the image quality when the encoded data is decoded.

In Equation (8), the residual signal is generated on a frame-by-frame basis. However, it is not necessary to use Equation (8) or generate a residual signal on a frame-by-frame basis. A residual signal may be generated in various units such as a block or a slice A residual signal may be generated using an equation other than equation (8).

5 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

According to an embodiment of the present invention, the image encoding apparatus 100 filters a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image (S510) The frame is transformed, quantized, and encoded (S520).

In step S510, the image encoding apparatus 100 may filter the motion compensation frame using another filter according to the characteristics of the block in the motion compensation frame, and may use the Wiener filter or deblocking filter And filter a motion compensation frame using a filter having a filter coefficient that minimizes a square error between a current frame and a filtered motion compensation frame. At this time, the image encoding apparatus 100 may further encode the filter coefficient information indicating the filter coefficient of the filter used for filtering the motion compensation frame.

In step S510, the image encoding apparatus 100 determines a motion vector and a reference frame index for a block in which the block mode is an inter mode in the current frame, and calculates a motion vector and a reference frame index using a motion vector and a reference frame index, Generates a motion compensation frame including the generated motion compensation block, and filters the motion compensation frame.

In addition, the image encoding apparatus 100 may encode block mode information indicating a block mode, and additionally encode motion information indicating a determined motion vector and a reference frame index.

In addition, the image encoding apparatus 100 restores the residual frame by inverse-quantizing and inverse-transforming the transformed and quantized residual frames, restores the current frame by adding the residual frame to be reconstructed and the filtered motion compensation frame, The frame can be de-blocking filtered and stored.

FIG. 6 is a block diagram of a video decoding apparatus according to an embodiment of the present invention. Referring to FIG.

The image decoding apparatus 600 according to an embodiment of the present invention includes a decoder (Decoder, 610), an inverse quantizer and an inverse converter (620), a predictor (630), an adder (640), a deblocking filter (650), and It may be configured to include a memory 660.

The decoder 610 decodes the motion information data and the image coded data extracted from the bit stream to recover the motion information and the residual frame converted and quantized. In addition, the decoder 610 may extract the block mode data from the bitstream and decode the block mode data to recover the block mode information. The decoder 610 may extract the filter coefficient data from the bitstream and decode the data to recover the filter coefficient information.

Here, the decoder 610 can decode various data by using various encoding techniques such as entropy encoding, and can decode the encoded data of the encoder 140 of the image encoding apparatus 100 in reverse order .

Inverse quantizer and inverse transformer 620 inverse quantizes and inverse transforms the reconstructed transformed and quantized residual frame to reconstruct the residual frame. Here, the inverse quantizer and the inverse transformer 620 may perform inverse quantization and inverse transformation by using various transformations and quantization schemes, and the transformer and quantizer 130 of the image encoding apparatus 100 may perform transformation and quantization. One way can be done inversely to perform inverse magnetization and inverse transformation.

The predictor 630 generates a motion compensation frame by compensating the motion of the current frame using the motion vector and the reference frame index identified by the restored motion information, and filters the motion compensation frame. That is, the predictor 630 may generate a motion compensation block using the motion vector and the reference frame index identified by the restored motion information, and generate a motion compensation frame including the motion compensation block.

In addition, the predictor 630 may filter the motion compensation frame using a filter having a predetermined filter coefficient. If the filter coefficient information is restored by the decoder 610, A motion compensation frame may be filtered using a filter having filter coefficients identified by the coefficient information.

In addition, the predictor 630 may perform filtering using other filters according to the characteristics of the pixels of the block in the motion compensation frame, or may employ a Wiener filter or deblocking filter as a filter for filtering the motion compensation frame.

Here, the predictor 630 generates a motion compensation block by using the motion vector and the reference frame index, generates a motion compensation frame including the motion compensation block, and filters the generated motion compensation frame with reference to FIGS. 1 to 5. Through the same or similar to the above-described method, detailed description thereof will be omitted.

The adder 640 restores the current frame by adding the residual frame to be restored and the motion compensation frame to be filtered. That is, the adder 640 adds the pixel values of the pixels of the residual frame reconstructed by the inverse quantizer and the inverse transformer 620 and the pixel values of the pixels of the motion compensation frame filtered by the predictor 630 to thereby add the pixels of the current frame. Restore them.

The deblocking filter 650 deblock filters the current frame reconstructed by the adder 640 and outputs the deblock-filtered current frame as a reconstructed image. The memory 660 reconstructs the deblocking filtered current frame as a reference frame And the stored reference frame is used to predict the next frame in the predictor 630. [

FIG. 7 is a block diagram schematically illustrating an inter prediction apparatus for video decoding according to an embodiment of the present invention. Referring to FIG.

The inter prediction apparatus for video decoding according to an embodiment of the present invention can be implemented as a predictor 630 in the video decoding apparatus 600 according to an embodiment of the present invention described above with reference to FIG. It is referred to as a predictor 630. [

The predictor 630 may comprise a motion compensator 710 and a motion compensation filter 720.

The motion compensator 710 generates a motion compensation block by using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and decoded and reconstructed, and generates a motion compensation frame including the motion compensation block. In other words, when the motion compensator 710 receives the motion vector and the reference frame index from the decoder 610, the motion compensator 710 decodes the motion vector of the block indicated by the motion vector in the reference frame indicated by the reference frame index among the reference frames stored in the memory 660 As motion compensation blocks and generates motion compensation frames including motion compensation blocks.

The motion compensation filter 720 filters motion compensation frames. That is, the motion compensation filter 720 filters the motion compensation frame generated by the motion compensator 710 using various filters such as a Wiener filter or a deblocking filter. In this case, when the motion compensation filter 720 receives the filter coefficient information from the decoder 610, the motion compensation filter 720 can generate a motion compensation frame using a filter having the filter coefficient identified by the filter coefficient information. The motion compensation frame may be filtered using a predetermined filter or a filter selected according to the pixel characteristics of the block in the motion compensation frame.

8 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

According to an image decoding method according to an embodiment of the present invention, the image decoding apparatus 600 currently uses a motion vector and a reference frame index identified by motion information reconstructed by decoding motion information data extracted from a bitstream. Compensate for the motion of the frame to generate a motion compensation frame (S810), filter the motion compensation frame (S820), and restore the current frame by adding the residual motion and the filtered motion compensation frame extracted and decoded from the bitstream (S830).

Here, in step S810, the image decoding apparatus 600 extracts motion information data from the bit stream and decodes the motion information data for a block in which the block mode is an inter-block in the current frame in step S810 to restore motion information And generates a motion compensation block using a motion vector and a reference frame index identified by the motion information to be reconstructed, and generates a motion compensation frame including the generated motion compensation block.

In addition, the image decoding apparatus 600 may further reconstruct filter coefficient information by extracting and decoding filter coefficient data from the bitstream. In this case, when the motion compensation frame is filtered in step S820, the image decoding apparatus 600 may reconstruct the filter coefficient information. The motion compensation frame may be filtered using a filter having a filter coefficient identified by.

In step S820, the image decoding apparatus 600 may filter a motion compensation frame using a filter having a predetermined filter coefficient in filtering a motion compensation frame, A motion compensation frame may be filtered using another filter according to the motion compensation frame, or the motion compensation frame may be filtered using a Wiener filter or a deblocking filter.

In addition, the image decoding apparatus 600 may deblock and filter the current frame to be restored. The deblocking filtered current frame can be stored and used to predict other frames.

As described above, according to an embodiment of the present invention, in predicting an image, temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation is removed to further improve compression efficiency The quantization error can be further reduced by adaptively selecting and filtering the type of the filter and the filter coefficients according to the characteristics of the pixels of the block in the motion compensation frame. Therefore, the compression efficiency can be further improved, The deterioration of the image quality of the reconstructed image to be reproduced can be further reduced.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. The codes and code segments constituting the computer program may be easily deduced by those skilled in the art. Such a computer program can be stored in a computer-readable storage medium, readable and executed by a computer, thereby realizing an embodiment of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

Furthermore, the terms "comprises", "comprising", or "having" described above mean that a component can be implanted unless otherwise specifically stated, But should be construed as including other elements. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be construed according to the following claims, and all technical ideas falling within the scope of the same shall be construed as falling within the scope of the present invention.

As described above, the present invention can be applied to an image compression processing field for encoding and decoding a video, thereby further improving compression efficiency by removing temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation. In addition, since the quantization error can be further reduced by adaptively selecting and filtering the filter type and the filter coefficient according to the characteristics of the pixels of the block in the motion compensation frame, the compression efficiency can be further improved, or the code is decoded and reproduced. It is a very useful invention to produce an effect that can further reduce the deterioration of the image quality of the restored image.

1 is a block diagram schematically illustrating an image encoding apparatus according to an embodiment of the present invention.

2 is a block diagram schematically illustrating an inter prediction apparatus for image encoding according to an embodiment of the present invention;

3 is an exemplary view illustrating a process of generating a motion compensation frame according to an embodiment of the present invention;

4 is an exemplary view illustrating a process of filtering a motion compensation frame according to an embodiment of the present invention;

5 is a flowchart illustrating a video encoding method according to an embodiment of the present invention;

6 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention;

7 is a block diagram schematically illustrating an inter prediction apparatus for image decoding according to an embodiment of the present invention;

8 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

Claims (21)

In the method of encoding an image, Filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; And Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame Including, Filtering the motion compensation frame, And the motion compensation frame is filtered using a different filter according to a characteristic of a pixel of a block in the motion compensation frame. delete In the method of encoding an image, Filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; And Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame The filtering of the motion compensation frame may include: And the motion compensation frame is filtered using a Wiener filter or a deblocking filter. The method of claim 1, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame. In the method of encoding an image, Filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; And Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame In addition, the video encoding method, And encoding filter coefficient information indicating filter coefficients of a filter used to filter the motion compensation frame. In the method of encoding an image, Filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; And Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame The filtering of the motion compensation frame may include: Determining a motion vector and a reference frame index for a block in which the block mode is inter mode within the current frame; Generating a motion compensation block by using the determined motion vector and the reference frame index and generating the motion compensation frame including the generated motion compensation block; And Filtering the motion compensation frame Image encoding method comprising a. The method of claim 6, wherein the video encoding method comprises: Encoding block mode information indicating the block mode; And Encoding motion information identifying the determined motion vector and a reference frame index. The video encoding method further comprises. In the method of encoding an image, Filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; And Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame In addition, the video encoding method, Inverse quantization and inverse transformation of the transformed and quantized residual frame to restore the residual frame; Reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame; And Deblocking filtering and storing the restored current frame The video encoding method further comprises. In the apparatus for encoding a video, A predictor for filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; A subtractor for generating a residual frame by subtracting the current frame and the filtered motion compensation frame; A converter and quantizer for transforming and quantizing the residual frame; And An encoder for encoding the transformed and quantized residual frames Including, The predictor, A motion estimator for determining a motion vector and a reference frame index for a block in which the block mode is inter mode within the current frame; A motion compensator for generating a motion compensation block using the determined motion vector and the reference frame index, and generating the motion compensation frame including the motion compensation block; And A motion compensation filter for filtering the motion compensation frame An image encoding apparatus comprising a. delete delete In the method of decoding an image, Generating a motion compensation frame by compensating for the motion of the current frame by using a motion vector identified by the motion information reconstructed by decoding the motion information data extracted from the bitstream and a reference frame index, and filtering the motion compensation frame; And Reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted from the bitstream and decoded and reconstructed The filtering of the motion compensation frame may include: Restoring motion information by extracting and decoding the motion information data from the bitstream for a block having a block mode of inter block in the current frame; Generating a motion compensation block using the motion vector identified by the reconstructed motion information and a reference frame index, and generating a motion compensation frame including the generated motion compensation block; And Filtering the motion compensation frame Image decoding method comprising a. In the method of decoding an image, Generating a motion compensation frame by compensating for the motion of the current frame by using a motion vector identified by the motion information reconstructed by decoding the motion information data extracted from the bitstream and a reference frame index, and filtering the motion compensation frame; And Reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted from the bitstream and decoded and reconstructed Including, the image decoding method, Extracting and decoding filter coefficient data from the bitstream to restore filter coefficient information, wherein filtering the motion compensation frame comprises filtering a filter having a filter coefficient identified by the reconstructed filter coefficient information. And the motion compensation frame is filtered using the image decoding method. In the method of decoding an image, Generating a motion compensation frame by compensating for the motion of the current frame by using a motion vector identified by the motion information reconstructed by decoding the motion information data extracted from the bitstream and a reference frame index, and filtering the motion compensation frame; And Reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted from the bitstream and decoded and reconstructed The filtering of the motion compensation frame may include: And the motion compensation frame is filtered using a filter having a predetermined filter coefficient. In the method of decoding an image, Generating a motion compensation frame by compensating for the motion of the current frame by using a motion vector identified by the motion information reconstructed by decoding the motion information data extracted from the bitstream and a reference frame index, and filtering the motion compensation frame; And Reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted from the bitstream and decoded and reconstructed The filtering of the motion compensation frame may include: And the motion compensation frame is filtered using a different filter according to the characteristics of the pixels of the block in the motion compensation frame. In the method of decoding an image, Generating a motion compensation frame by compensating for the motion of the current frame by using a motion vector identified by the motion information reconstructed by decoding the motion information data extracted from the bitstream and a reference frame index, and filtering the motion compensation frame; And Reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted from the bitstream and decoded and reconstructed The filtering of the motion compensation frame may include: And the motion compensation frame is filtered using a Wiener filter or a deblocking filter. In the apparatus for decoding an image, A decoder configured to decode the motion information data and the image encoded data extracted from the bitstream and to restore the motion information and the residual and quantized frames; An inverse quantizer and an inverse converter that inverse quantizes and inverse transforms the reconstructed transform and quantized residual frames to recover the residual frames;  A predictor for generating a motion compensation frame by compensating for the motion of the current frame using the motion vector identified by the reconstructed motion information and a reference frame index, and filtering the motion compensation frame; And An adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame; Including, And the predictor filters the motion compensation frame using a filter having a predetermined filter coefficient. In the inter prediction method, Determining a motion vector and a reference frame index for a block in which a block mode is inter mode within a current frame of an image; Generating a motion compensation block by using the determined motion vector and the reference frame index, and generating a motion compensation frame including the motion compensation block; And Filtering the motion compensation frame The inter prediction method comprising the steps of: In the inter prediction apparatus, A motion estimator for determining a motion vector and a reference frame index for a block in which a block mode is inter mode within a current frame of an image; A motion compensator for generating a motion compensation block using the determined motion vector and the reference frame index, and generating a motion compensation frame including the motion compensation block; And A motion compensation filter for filtering the motion compensation frame Inter prediction apparatus comprising a. In the inter prediction method, Generating a motion compensation block using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and decoded and reconstructed, and generating a motion compensation frame including the motion compensation block; And Filtering the motion compensation frame The inter prediction method comprising the steps of: In the inter prediction apparatus, A motion compensator for generating a motion compensation block using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and reconstructed from the bitstream, and generating a motion compensation frame including the motion compensation block; And A motion compensation filter for filtering the motion compensation frame Inter prediction apparatus comprising a.

Images