KR20110042602A - Video coding method and apparatus by using filtering motion compensation frame - Google Patents

Abstract

PURPOSE: A video encoding/decoding method and device thereof using filtering of a movement compensation frame are provided to improve a compression efficiency of a video by filtering a movement compensation frame. CONSTITUTION: A movement compensation frame which is generated by estimating and compensating movement of a current frame of an image is filter(S510). A residual frame which is a difference of the current frame and the filtered movement compensation frame is quantized and encoded(S520).

Description

Video Coding Method and Apparatus by Using Filtering Motion Compensation Frame

The present invention relates to a method and apparatus for image encoding / decoding using filtering of a motion compensation frame. More specifically, the present invention relates to a method and apparatus for improving the compression efficiency of a video by filtering a motion compensation frame generated through motion estimation and motion compensation during video encoding and decoding.

In order to efficiently store or transmit video data, it is necessary to compress and compress the data. Techniques for compressing video data include H.261, H.263, H.264, MPEG-2, and MPEG-4. In the video compression technique, the luminance component and the chrominance component of each frame image of the video are predicted temporally through motion estimation and motion compensation, and the prediction residuals are transformed, quantized, and entropy encoded to be transmitted in the form of a bitstream.

H.264 / AVC, the latest video compression technology, was developed jointly by the Moving Picture Experts Group (MPEG) and the Video Coding Experts Group (VCEG). Macroblocks are 16x16, 16x8, 8x16, 8x8, 8x4, 4x8, Motion estimation and motion compensation are performed by dividing into smaller blocks having a size of 4x4. In this case, up to 16 frames may also be used for the reference frame for motion estimation and motion compensation. In addition, the H.264 / AVC adopts a 1/4 pixel interpolation method, and the compression rate can be improved while maintaining the image quality compared to MPEG-4, which is a conventional video compression technology.

As such, in the conventional video compression technology, high compression ratio can be achieved by removing temporal redundancy in the image using motion estimation and motion compensation, but the motion compensation frame generated through motion estimation and motion compensation is still temporal. Since a lot of redundancy remains, the prediction residual to be transformed, quantized, and entropy encoded becomes large, and thus a large amount of residual data needs to be encoded.

In order to solve the above problems, the present invention is to improve the compression efficiency of a video by filtering a motion compensation frame generated through motion estimation and motion compensation and encoding the remaining frames during video encoding and decoding. have.

In order to achieve the above object, the present invention provides a method of encoding an image, the method comprising: filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image; And transforming, quantizing, and encoding the remaining frame that is a difference between the current frame and the filtered motion compensation frame.

According to another object of the present invention, there is provided an apparatus for encoding an image, comprising: a predictor for filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of an image; A subtractor for generating a residual frame by subtracting the current frame and the filtered motion compensation frame; A converter and quantizer for transforming and quantizing residual frames; And an encoder for encoding the transformed and quantized residual frames.

According to still another object of the present invention, in a method of decoding an image, the current frame is obtained by using a motion vector and a reference frame index identified by the motion information reconstructed by decoding motion information data extracted from the bitstream. Compensating for motion to generate a motion compensation frame and filtering the motion compensation frame; And reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted and decoded from the bitstream to recover the current frame.

According to still another aspect of the present invention, there is provided an apparatus for decoding an image, comprising: a decoder for decoding motion information data and image encoded data extracted from a bitstream and restoring motion information, transformed and quantized residual frames; An inverse quantizer and an inverse converter that inverse quantizes and inverse transforms the reconstructed transform and quantized residual frames to recover the residual frames; A predictor for generating a motion compensation frame by compensating for the motion of the current frame by using the motion vector identified by the reconstructed motion information and the reference frame index, and filtering the motion compensation frame; And an adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame.

According to still another object of the present invention, there is provided an inter prediction method comprising: determining a motion vector and a reference frame index for a block in which a block mode is an inter mode within a current frame of an image; Generating a motion compensation block using the determined motion vector and the reference frame index and generating a motion compensation frame including the motion compensation block; And filtering the motion compensation frame.

According to still another object of the present invention, there is provided an inter prediction apparatus comprising: a motion estimator for determining a motion vector and a reference frame index for a block in which a block mode is an inter mode within a current frame of an image; A motion compensator for generating a motion compensation block using the determined motion vector and the reference frame index and generating a motion compensation frame including the motion compensation block; And a motion compensation filter for filtering the motion compensation frame.

In addition, according to another object of the present invention, in the inter prediction method, a motion compensation block is generated by using a motion vector and a reference frame index identified by motion information extracted from the bitstream, and decoded from the bitstream. Generating a motion compensation frame comprising a; And filtering the motion compensation frame.

According to still another object of the present invention, in an inter prediction apparatus, a motion compensation block is generated by using a motion vector and a reference frame index identified by motion information extracted from a bitstream, decoded, and recovered. A motion compensator for generating a motion compensation frame comprising a; And a motion compensation filter for filtering the motion compensation frame.

As described above, according to the present invention, since the temporal redundancy remaining in the remaining frames can be further removed by filtering the motion compensation frame, the compression efficiency of the video can be improved.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are assigned to the same components as much as possible even though they are shown in different drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

In addition, in describing the component of this invention, terms, such as 1st, 2nd, A, B, (a), (b), can be used. These terms are only for distinguishing the components from other components, and the nature, order or order of the components are not limited by the terms. If a component is described as being "connected", "coupled" or "connected" to another component, that component may be directly connected or connected to that other component, but between components It will be understood that may be "connected", "coupled" or "connected".

A video encoding apparatus (Video Encoding Apparatus), a video decoding apparatus (Video Decoding Apparatus) to be described below is a personal computer (PC), notebook computer, personal digital assistant (PDA), portable multimedia player (PMP) : User terminal such as Portable Multimedia Player (PSP), PlayStation Portable (PSP: PlayStation Portable), Mobile Communication Terminal (Mobile Communication Terminal), or a server terminal such as an application server or a service server, and communicates with various devices or wired / wireless communication networks. The present invention refers to various devices including a communication device such as a communication modem, a memory for storing various programs and data for encoding or decoding an image, a microprocessor for executing and operating a program, and the like.

In addition, the image encoded in the bitstream by the video encoding apparatus is real-time or non-real-time through the wired or wireless communication network, such as the Internet, local area wireless communication network, wireless LAN network, WiBro network, mobile communication network, or the like, or a cable, universal serial bus (USB: Universal) It may be transmitted to an image decoding apparatus through various communication interfaces such as a serial bus, and may be decoded by the image decoding apparatus to restore and reproduce the image.

In general, a moving picture is composed of a series of pictures, and each picture is divided into a predetermined area such as a block. When a region of an image is divided into blocks, the divided blocks are largely classified into intra blocks and inter blocks according to encoding methods. An intra block refers to a block that is encoded by using an intra prediction coding method. An intra prediction coding is performed by using pixels of blocks previously encoded, decoded, and reconstructed in a current picture that performs current encoding. A prediction block is generated by predicting pixels of a block and a difference value with pixels of the current block is encoded. An inter block refers to a block that is encoded using inter prediction coding. Inter prediction coding generates a prediction block by predicting a current block within a current picture by referring to one or more past or future pictures, and then generates a current block. This is a method of encoding the difference value with. Here, a frame referred to for encoding or decoding the current picture is referred to as a reference frame.

1 is a block diagram schematically illustrating a video encoding apparatus according to an embodiment of the present invention.

The image encoding apparatus 100 according to an embodiment of the present invention is an apparatus for encoding an image, and includes a predictor 110, a subtractor 120, a transformer and quantizer 130, and an encoder. The encoder may include an encoder 140, an inverse quantizer and an inverse transformer 150, an adder 160, a deblocking filter 170, and a memory 180.

The predictor 110 filters a motion compensation frame generated by estimating and compensating for a motion of a frame to be encoded in the input image (hereinafter referred to as a current frame). That is, the predictor 110 estimates the motion of each block in the current frame according to the inter prediction mode in the reference frame stored in the memory 180 after it is already encoded, decoded, and reconstructed, and then is a motion vector. ) And generate a motion compensation frame in which the motion of the current frame is compensated by compensating the motion of each block using the motion vector, and filtering the generated motion compensation frame.

To this end, the predictor 110 determines a motion vector and a reference frame index for a block in which the block mode is inter mode in the current frame, and uses the determined motion vector and reference frame index. The motion compensation block may be generated, and the motion compensation frame may be filtered by generating a motion compensation frame including the generated motion compensation block.

Here, the predictor 110 may filter the motion compensation frame using another filter according to the characteristics of the pixels of the block in the motion compensation frame, and the motion compensation frame may be a Winner filter or a deblocking filter. Filtering may also be performed using various filters such as the above.

Also, the predictor 110 may filter the motion compensation frame using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame.

The subtractor 120 subtracts the current frame and the filtered motion compensation frame to generate a residual frame. Here, the residual frame is a frame including a residual generated by subtracting the current frame and the filtered motion compensation frame, and refers to a frame including a residual signal that is a difference between pixels of the current frame and pixels of the filtered motion compensation frame.

Converter and quantizer 130 transforms and quantizes the remaining frames. That is, the transformer and quantizer 130 converts the residual signal of the residual frame generated by the subtractor 120 into a frequency domain to generate a transformed residual frame having a transform coefficient and transforms the transformed signal. The transform coefficients of the residual frame are quantized to produce transformed and quantized residual frames. In this case, as a transform method used, a method of transforming an image signal in a spatial domain into a frequency domain such as a Hadamard transform or a discrete cosine transform based integer transform is used. As the scheme, various quantization techniques such as Dead Zone Uniform Threshold Quantization (DZUTQ) or quantization weighted matrix (DZUTQ) may be used.

The encoder 140 encodes the transformed and quantized residual frames. That is, the quantized transform coefficients of the residual frame transformed and quantized by the transformer and the quantizer 130 are encoded to generate image encoded data. The generated image coded data is included in the bit stream. As the encoding technique, an entropy encoding technique may be used, but various encoding techniques may be used without being limited thereto.

In addition, the encoder 140 may generate filter coefficient data by encoding filter coefficient information indicating filter coefficients of a filter used to filter the motion compensation frame. The generated filter coefficient data is included in the bitstream.

In addition, the encoder 140 may generate block mode data by encoding block mode information indicating a block mode of each block in the current frame, and the generated block mode data is included in the bitstream.

Also, the encoder 140 may generate motion information data by encoding motion information indicating a motion vector and a reference frame index determined for a block in which the block mode is inter mode in the current frame. It is included in the bitstream.

Meanwhile, the functions of the transformer and the quantizer 130 may be integrated into the encoder 140 and implemented. That is, when implemented with one encoder, the encoder generates encoded data by encoding the residual frame.

Inverse quantizer and inverse transformer 150 inverse quantizes and inverse transforms the transformed and quantized residual frame to recover the residual frame. That is, the inverse quantizer and the inverse transformer 150 inverse quantizes the transformed and quantized residual frames transmitted from the transformer and the quantizer 130 to restore the residual frame having the transform coefficients, and the residual frame having the transform coefficients. Inverse conversion is performed again to restore the residual frame with the residual signal. In this case, the inverse quantizer and the inverse transformer 150 may restore the remaining frame by performing the transformed and quantized method in the inverse of the transformer and the quantizer 130.

The adder 160 reconstructs the current frame by adding the reconstructed residual frame and the filtered motion compensation frame.

The deblocking filter 170 deblocks and filters the current frame to be restored, and the memory 180 stores the deblocking filtered current frame. Here, the deblocking filtering refers to an operation of reducing block distortion generated by encoding an image in block units, and applying a deblocking filter to a block boundary and a macroblock boundary, or applying a deblocking filter only to a macroblock boundary or a deblocking filter. You can optionally use one of the methods that does not use.

Although not shown in FIG. 1, the image encoding apparatus 100 according to an embodiment of the present invention may further include an intra predictor for intra prediction in the predictor 110. In this case, the subtractor 120 may generate a residual frame by subtracting the current frame and the prediction frame generated by the intra predictor, and the transformer and quantizer 130 and the inverse quantizer and the inverse transformer 150 are residual frames. Calculation may be further performed for transform and quantization for and inverse transform and inverse quantization for the transformed and quantized residual frame. In addition, the encoder 130 may generate the image encoded data by encoding the transformed and quantized residual frame, which is included in the bitstream.

Therefore, in the exemplary embodiment of the present invention, only the remaining frame, which is a difference between the current frame and the filtered motion compensation frame, is transformed, quantized, and encoded, so that the image encoded data according to inter prediction encoding is included in the bitstream. The apparatus 100 may further generate image coded data to be intra predictively coded, and in the bitstream, image coded data generated by intra predictive coding as well as image coded data generated by inter predictive coding and the image coded data. It will be appreciated that information for intra prediction decoding the data or its encoded data may be further included.

2 is a block diagram schematically illustrating an inter prediction apparatus for image encoding according to an embodiment of the present invention.

Since the inter prediction apparatus for image encoding according to an embodiment of the present invention may be implemented as the predictor 110 in the image encoding apparatus 100 according to the embodiment of the present invention described above with reference to FIG. For convenience, the predictor 110 is called.

The predictor 110 may include a motion estimator 210, a motion compensator 220, and a motion compensation filter 230. In addition, the predictor 110 may further include an intra predictor, but is not separately illustrated.

The motion estimator 210 determines a motion vector and a reference frame index for a block in which the block mode is inter mode in the current frame. That is, the motion estimator 210 determines the motion vector and the reference frame index by estimating the motion of the blocks in which the block mode is the inter mode among the blocks in the current frame. Here, the motion estimator 210 may determine the block mode for each block in the current frame, but may be determined by a mode determiner (not shown) additionally provided in the predictor 110 or the image encoding apparatus 100. have.

To this end, the motion estimator 210 performs forward prediction on a block whose block mode is P macroblock in the current frame to estimate a forward motion vector based on the previous frame of the current frame. Bidirectional prediction may be performed on a block having a B macroblock of B macroblock within the block to estimate a forward motion vector and a backward motion vector based on the previous frame and the next frame of the current frame.

In this case, in determining the motion vector, the motion estimator 210 may determine, as the motion vector and the reference frame index for the corresponding block in the current frame, the motion vector and the reference frame index at which the encoding cost is minimum. As such, the mcost function may be used as a function for calculating the coding cost for determining the motion vector and the reference frame index. The mcost function may be expressed as in Equation 1.

Here, λ is a Lagrange Multiplier for determining the motion vector and the reference frame index, and rate _{u, v, i} are predictive coding by compensating the motion of the block using the motion vector and the reference frame index. To indicate the bit rate of the generated bit stream. SAD _{u, v, i} represents the sum of squares of the difference between the current frame and the motion compensation frame generated by compensating for the motion of the corresponding block using the motion vector and the reference frame index, and can be expressed by Equation 2.

는 현재 프레임의 (x+m,y+n) 좌표에 대한 화소값을 의미하고,

는 i번째 참조 프레임의 (x+m+u,y+n+v) 좌표에 대한 화소값을 나타낸다. 또한, x와 y는 해당 블록의 제일 왼쪽의 제일 위쪽에 위치한 화소의 현재 프레임 또는 움직임 보상 프레임 내에서의 좌표를 나타내며, m와 n은 해당 블록의 내부에 위치한 화소의 좌표를 나타낸다. u와 v는 해당 블록의 움직임 벡터를 나타낸다.In Equation 2, M and N respectively represent the pixel size in the horizontal direction and the vertical direction of the block,

Is the pixel value for the ( x + m, y + n ) coordinate of the current frame,

Denotes a pixel value for ( x + m + u, y + n + v ) coordinates of the i th reference frame. In addition, x and y represent the coordinates in the current frame or the motion compensation frame of the pixel located at the top left of the block, m and n represents the coordinates of the pixel located inside the block. u and v represent the motion vectors of the block.

The motion estimator 210 calculates mcost for each motion vector and reference frame index for each block whose block mode is interblock in the current frame by using a function for calculating a coding cost as shown in Equation 1, A motion vector and a reference frame index for minimizing the encoding cost are determined as a motion vector and a reference frame index of the corresponding block by using a function as shown in FIG. 3.

는 각각 해당 블록에 대해 결정된 움직임 벡터의 가로 성분과 세로 성분을 내타내며,

는 해당 블록에 대해 결정된 참조 프레임 색인을 나타낸다.In Equation 3, SearchRange represents a range for performing motion estimation.

Represents the horizontal and vertical components of the motion vector determined for that block,

Denotes a reference frame index determined for the block.

The motion compensator 220 generates a motion compensation block by using the determined motion vector and the reference frame index, and generates a motion compensation frame including the motion compensation block. That is, the motion compensator 220 generates a motion compensation block by compensating for the motion of the corresponding block by using the motion vector determined by the motion estimator 210 and the reference frame index, and gathers the generated motion compensation blocks in units of frames. Generate a motion compensation frame comprising motion compensation blocks. To this end, the motion compensator 220 obtains a block indicated by the motion vector determined by the motion estimator 210 in the reference frame identified by the reference frame index determined by the motion estimator 210 as a motion compensation block. And a motion compensation frame including these motion compensation blocks.

In this way, a method of generating a motion compensation frame may be represented by Equation 4 below.

는 움직임 보상 프레임의 (x,y) 좌표에 대한 화소값을 의미한다. 만약, (x,y) 좌표에 의해 식별되는 화소를 포함하는 블록의 블록 모드가 인트라 모드인 경우에는 움직임 벡터와 참조 프레임 색인이 존재하지 않기 때문에 일단

는 값을 가지지 않는다. 이후, 하나의 움직임 보상 프레임 생성이 완료되면 미러링(Mirroring) 방식과 유사한 방식으로 앞에 내용을 채워 넣을 수도 있다.In Equation 4, W and H represent the horizontal and vertical pixel size of the frame (reference frame and motion compensation frame),

Denotes a pixel value for the ( x , y ) coordinate of the motion compensation frame. If the block mode of the block including the pixel identified by the ( x , y ) coordinate is the intra mode, since the motion vector and the reference frame index do not exist,

Has no value. Subsequently, when generation of one motion compensation frame is completed, the content may be filled in a manner similar to a mirroring method.

Referring to FIG. 3 exemplarily illustrating a process of generating a motion compensation frame according to an embodiment of the present invention, four blocks among five blocks in the motion compensation frame have an interblock in four reference frames. Each block is filled with one block, but one block does not have a value because the block mode is an intra block and there is no motion vector and reference frame index. When the motion compensation frame is filtered later, since the value of the intra mode block is not filled, it is impossible to filter the neighboring pixels, and thus, the motion compensation frame may be filtered using the pixel values of the pixels around the block.

The motion compensation filter 230 filters the motion compensation frame. That is, the motion compensation filter 230 filters the pixels of the motion compensation frame generated by the motion compensator 220 using various types of filters so that the filtered motion compensation frame is more similar to the current frame, thereby transforming and quantizing. Thereby reducing the amount of data in the remaining frame to be encoded or reducing the blocking effect present in the motion compensation frame.

Here, the motion compensation filter 230 may use a predetermined filter when filtering the motion compensation frame, or may select and use an appropriate filter or a coefficient of the filter in a predetermined unit such as a block unit, a slice unit, or a frame unit. When the motion compensation filter 230 uses a preset filter, a filter pre-appointed by the video encoding apparatus 100 and a video decoding apparatus to be described later, or a filter based on a pre-promised criterion may be used. In addition, when the motion compensation filter 230 selects and uses an appropriate filter or a filter coefficient, another filter may be used according to the characteristics of the pixels in the current frame.

As a filter for filtering the motion compensation frame, various filters such as a Wiener filter or a deblocking filter may be used. When the motion compensation frame is filtered using the Wiener filter, the temporal redundancy remaining in the motion compensation frame can be further eliminated, thereby reducing the amount of data in the remaining frames and consequently reducing the amount of image coded data. As it becomes smaller, the compression efficiency can be improved. In addition, when the motion compensation frame is filtered using the deblocking filter, the blocking phenomenon in the motion compensation frame can be reduced, thereby improving the quality of the image which is decoded and then reconstructed.

When the motion compensation filter 230 selects an appropriate filter or coefficients of the filter, the filter is selected according to the characteristics of the pixels in the motion compensation frame and filtered using the selected filter, or the filter coefficients of the preset filter are calculated. Filtering may be performed using the calculated filter coefficients or the filter type may be selected and the filter coefficients of the selected filter may also be calculated and filtered.

For example, the motion compensation filter 230 determines the size of the motion compensation block in the motion compensation frame and determines the size of the motion compensation block, and the boundary between the pixels located at the boundary of the block within the corresponding motion prediction block and the block boundary. Pixels not located at may be filtered using different types of filters. In this case, the characteristic of the pixel in the motion compensation frame may be the size of the motion compensation block in the motion compensation frame. In addition, the motion compensation filter 230 determines a difference between the motion vector of the motion compensation blocks and the reference frame index, and determines whether a blocking phenomenon occurs at the boundary between the motion compensation blocks according to the difference between the motion vector and the index of the reference frame. By determining, the pixels of the portion where the blocking phenomenon occurs and the pixels of the portion where the blocking phenomenon does not occur may be filtered using different filters. In this case, the characteristic of the pixel in the motion compensation frame may be a difference between the motion vector and the reference frame index between the motion compensation blocks.

If the motion compensation filter 230 filters the motion compensation frame in an exemplary equation, it may be represented by Equation 5. In Equation 5, the case of filtering by using a two-dimensional filter is shown as an example, but may be filtered using a one-dimensional filter, depending on the implementation method or needs.

는 움직임 보상 프레임의 (x+m,y+n) 좌표에 의해 지시되는 화소의 화소값을 나타내고, β ^PP (m,n)은 (m,n) 번째 필터 계수를 나타내며, PP는 움직임 보상 프레임 내의 블록 내의 화소의 특성에 따라 선택되는 필터의 종류를 나타낸다.

은 필터링된 움직임 보상 프레임 내에서 (x,y) 좌표에 위치하는 화소의 화소값을 나타낸다.In Equation 5,

Denotes the pixel value of the pixel indicated by the ( x + m, y + n ) coordinate of the motion compensation frame, β ^PP ( m , n ) denotes the ( m, n ) th filter coefficients, and PP is the motion compensation frame The type of filter selected according to the characteristics of the pixels in the blocks within the blocks is shown.

Denotes a pixel value of a pixel located at ( x , y ) coordinates in the filtered motion compensation frame.

Referring to FIG. 4 exemplarily illustrating a process of filtering a motion compensation frame according to an embodiment of the present invention, the motion compensation frame having a blocking phenomenon at the boundary between the motion compensation blocks is filtered using the motion compensation filter. In the filtered motion compensation frame, a blocking phenomenon may be removed at the boundary between the motion compensation blocks. 4 exemplarily illustrates a case where a two-dimensional deblocking filter is used as a motion compensation filter. In FIG. 4, * denotes convolution.

In the conventional video compression technique, since the motion compensation is performed in units of blocks, the motion compensation frame is used as a conceptual term. However, in the exemplary embodiment of the present invention, the motion compensation frame is filtered by generating and filtering the motion compensation frame in frame units. Is applied to video compression.

As an example of calculating the filter coefficients, the motion compensation filter 230 may calculate a filter coefficient for filtering the motion compensation frame using the Wiener filter such that the square error of the current frame and the filtered motion compensation frame is minimized. Equation 6 shows an equation for calculating a squared error between the current frame and the filtered motion compensation frame, and Equation 7 calculates a filter coefficient that minimizes the square error of the current frame and the filtered motion compensation frame. It is shown.

는 현재 프레임의 (x,y) 좌표에 대한 화소 값을 나타내고,

는 (m,n) 번째 필터 계수를 나타내며,

는 필터링된 움직임 보상 프레임의 (x,y) 좌표에 대한 화소값을 나타내며 수학식 5를 통해 계산될 수 있다.In equations (6) and (7), ( e ^PP ) represents the squared error of the current frame and the filtered motion compensation frame,

Denotes the pixel value for the ( x, y ) coordinate of the current frame,

Represents the ( m , n ) th filter coefficient,

Denotes a pixel value for the ( x, y ) coordinate of the filtered motion compensation frame and may be calculated through Equation 5.

 The filter coefficients calculated through Equations 6 and 7 are used to filter the motion compensation frame and are passed to the encoder 140. In this case, the motion compensation filter 230 may filter the motion compensation frame using the calculated filter as it is, but may filter the motion compensation frame by correcting the filter coefficient through a process such as quantization and rounding down. For example, if the filter coefficient is calculated as a coefficient having a decimal point, if it is encoded as it is, the data amount of the filter coefficient data increases, so that the coefficient is corrected to an integer coefficient through a process such as quantization, rounding, and rounding, and then the corrected filter coefficient The motion compensation frame is filtered and the corrected filter coefficients are transmitted to the encoder 140. The encoder 140 may generate filter coefficient data by encoding filter coefficient information indicating the filter coefficient, and the generated filter coefficient data may be included in the bitstream and transmitted to the image decoding apparatus. By correcting and encoding the filter coefficients as described above, the data amount of the filter coefficient data can be reduced.

The filtered motion compensation frame generated as described above is transferred to the subtractor 120, and the subtractor 120 generates a residual frame by subtracting the current frame and the filtered motion compensation frame. In this case, the filtered motion compensation block may be adaptively used for each unit such as a block or slice unit. The residual frame includes a residual signal having a difference value between the pixel value of the pixel of the current frame and the pixel value of the pixel of the motion compensation frame. The residual signal may be represented as in Equation 8.

However, the residual signal generated by subtracting the current frame and the motion compensation frame according to a typical video compression method may be represented by Equation (9).

In Equations 8 and 9, resi ( x , y ) represents the pixel value of the residual signal at the (x, y) coordinate. When the motion compensation filtering is performed using the Wiener filter, the temporal redundancy remaining in the motion compensation frame is further removed so that the residual signal of Equation 8 is smaller than the residual signal of Equation 9, and the motion is performed using a deblocking filter. When the compensation filtering is performed, the blocking phenomenon remaining in the motion compensation frame is further removed, so that the image quality when the residual data of Equation 8 is transformed and quantized and the encoded data is decoded is further improved.

Although Equation 8 illustrates that the residual signal is generated in units of frames, Equation 8 is not necessarily used or the residual signal is generated in units of frames, and the residual signal may be generated in various units such as blocks or slices. The residual signal may be generated using an equation other than Equation 8.

5 is a flowchart illustrating an image encoding method according to an embodiment of the present invention.

According to an embodiment of the present invention, the image encoding apparatus 100 filters the motion compensation frame generated by estimating and compensating for the motion of the current frame of the image (S510), and remaining the difference between the current frame and the filtered motion compensation frame. The frame is transformed, quantized, and encoded (S520).

In operation S510, the image encoding apparatus 100 may filter the motion compensation frame using another filter according to the characteristics of the pixels of the block in the motion compensation frame, and use the Wiener filter or the deblocking filter as the motion compensation frame. The motion compensation frame may be filtered using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame. In this case, the image encoding apparatus 100 may further encode filter coefficient information indicating the filter coefficient of the filter used to filter the motion compensation frame.

In operation S510, the image encoding apparatus 100 may determine a motion vector and a reference frame index for a block in which the block mode is an inter mode, and use the determined motion vector and the reference frame index to determine a motion compensation block. And generate a motion compensation frame including the generated motion compensation block, and filter the motion compensation frame.

In addition, the image encoding apparatus 100 may encode block mode information indicating a block mode, and additionally encode motion information indicating a determined motion vector and a reference frame index.

In addition, the image encoding apparatus 100 may reconstruct the residual frame by inverse quantization and inverse transformation of the transformed and quantized residual frame, restore the current frame by adding the reconstructed residual frame and the filtered motion compensation frame, and restore the current frame. Frames can be deblocked filtered and saved.

6 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention.

The image decoding apparatus 600 according to an embodiment of the present invention includes a decoder (Decoder, 610), an inverse quantizer and an inverse converter (620), a predictor (630), an adder (640), a deblocking filter (650), and It may be configured to include a memory 660.

The decoder 610 decodes the motion information data and the image encoded data extracted from the bitstream to restore the motion information, the transformed and quantized residual frames. In addition, the decoder 610 may recover block mode information by extracting and decoding block mode data from the bitstream, and may restore filter coefficient information by extracting and decoding filter coefficient data from the bitstream.

Here, the decoder 610 may decode various data using various encoding techniques such as entropy encoding, and may inversely decode a method encoded by the encoder 140 of the image encoding apparatus 100. .

Inverse quantizer and inverse transformer 620 inverse quantizes and inverse transforms the reconstructed transformed and quantized residual frame to reconstruct the residual frame. Here, the inverse quantizer and the inverse transformer 620 may perform inverse quantization and inverse transformation by using various transformations and quantization schemes, and the transformer and quantizer 130 of the image encoding apparatus 100 may perform transformation and quantization. One way can be done inversely to perform inverse magnetization and inverse transformation.

The predictor 630 compensates for the motion of the current frame using the motion vector identified by the reconstructed motion information and the reference frame index to generate a motion compensation frame and filters the motion compensation frame. That is, the predictor 630 may generate a motion compensation block using the motion vector and the reference frame index identified by the reconstructed motion information, and generate a motion compensation frame including the motion compensation block.

In addition, the predictor 630 may filter by using a filter having a predetermined filter coefficient in filtering the motion compensation frame, and, if there is filter coefficient information restored by the decoder 610, the restored filter. The motion compensation frame may be filtered using a filter having a filter coefficient identified by the coefficient information.

In addition, the predictor 630 may filter using another filter according to the characteristics of the pixels of the block in the motion compensation frame, and may use a Wiener filter or a deblocking filter as a filter for filtering the motion compensation frame.

Here, the predictor 630 generates a motion compensation block by using the motion vector and the reference frame index, generates a motion compensation frame including the motion compensation block, and filters the generated motion compensation frame with reference to FIGS. 1 to 5. Through the same or similar to the above-described method, detailed description thereof will be omitted.

The adder 640 reconstructs the current frame by adding the reconstructed residual frame and the filtered motion compensation frame. That is, the adder 640 adds the pixel values of the pixels of the residual frame reconstructed by the inverse quantizer and the inverse transformer 620 and the pixel values of the pixels of the motion compensation frame filtered by the predictor 630 to thereby add the pixels of the current frame. Restore them.

The deblocking filter 650 deblocks and filters the current frame restored by the adder 640 to output the deblocking filtered current frame as a reconstructed image, and the memory 660 uses the deblocking filtered current frame as a reference frame. The stored reference frame is used by the predictor 630 to predict the next frame.

7 is a block diagram schematically illustrating an inter prediction apparatus for image decoding according to an embodiment of the present invention.

The inter prediction apparatus for image decoding according to an embodiment of the present invention may be implemented as a predictor 630 in the image decoding apparatus 600 according to the embodiment of the present invention described above with reference to FIG. For convenience, the predictor 630 is called.

The predictor 630 may include a motion compensator 710 and a motion compensation filter 720.

The motion compensator 710 generates a motion compensation block by using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and decoded and reconstructed, and generates a motion compensation frame including the motion compensation block. That is, when the motion compensator 710 receives the motion vector and the reference frame index from the decoder 610, the block indicated by the motion vector in the reference frame indicated by the reference frame index among the reference frames stored in the memory 660. Is generated as motion compensation blocks and a motion compensation frame including the motion compensation blocks.

The motion compensation filter 720 filters the motion compensation frame. That is, the motion compensation filter 720 filters the motion compensation frame generated and transmitted by the motion compensator 710 using various filters such as a winner filter or a deblocking filter. In this case, when the motion compensation filter 720 receives the filter coefficient information from the decoder 610, the motion compensation filter 720 may generate a motion compensation frame using a filter having the filter coefficient identified by the filter coefficient information. If not, the motion compensation frame may be filtered using a preset filter or a filter selected according to pixel characteristics of a block in the motion compensation frame.

8 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

According to an image decoding method according to an embodiment of the present invention, the image decoding apparatus 600 currently uses a motion vector and a reference frame index identified by motion information reconstructed by decoding motion information data extracted from a bitstream. Compensate for the motion of the frame to generate a motion compensation frame (S810), filter the motion compensation frame (S820), and restore the current frame by adding the residual motion and the filtered motion compensation frame extracted and decoded from the bitstream (S830).

Here, in operation S810, the image decoding apparatus 600 extracts and decodes motion information data from a bitstream and decodes motion information data from a bitstream for a block having a block mode interblock in a current frame to restore motion information. The motion compensation block may be generated using the motion vector identified by the reconstructed motion information and the reference frame index, and a motion compensation frame including the generated motion compensation block may be generated.

In addition, the image decoding apparatus 600 may further reconstruct filter coefficient information by extracting and decoding filter coefficient data from the bitstream. In this case, when the motion compensation frame is filtered in step S820, the image decoding apparatus 600 may reconstruct the filter coefficient information. The motion compensation frame may be filtered using a filter having a filter coefficient identified by.

Also, in operation S820, the image decoding apparatus 600 may filter the motion compensation frame using a filter having a predetermined filter coefficient in filtering the motion compensation frame, and may include characteristics of the pixels of the block in the motion compensation frame. The motion compensation frame may be filtered using another filter, and the motion compensation frame may be filtered using a Wiener filter or a deblocking filter.

In addition, the image decoding apparatus 600 may deblock the filtered current frame and output the restored current frame. The deblocking filtered current frame can be stored and used to predict other frames.

As described above, according to an embodiment of the present invention, in predictive encoding of an image, a compression efficiency may be further improved by removing temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation. In addition, it is possible to further reduce the quantization error by adaptively selecting and filtering the filter type and the filter coefficient according to the characteristics of the pixels of the block in the motion compensation frame, so that the compression efficiency can be further improved or the code is decoded after being decoded. The deterioration of the image quality of the reconstructed video to be reproduced can be further reduced.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. In other words, within the scope of the present invention, all of the components may be selectively operated in combination with one or more. In addition, although all of the components may be implemented in one independent hardware, each or all of the components may be selectively combined to perform some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having a. Codes and code segments constituting the computer program may be easily inferred by those skilled in the art. Such a computer program may be stored in a computer readable storage medium and read and executed by a computer, thereby implementing embodiments of the present invention. The storage medium of the computer program may include a magnetic recording medium, an optical recording medium, a carrier wave medium, and the like.

In addition, the terms "comprise", "comprise" or "having" described above mean that the corresponding component may be inherent unless specifically stated otherwise, and thus excludes other components. It should be construed that it may further include other components instead. All terms, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. Terms used generally, such as terms defined in a dictionary, should be interpreted to coincide with the contextual meaning of the related art, and shall not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present invention.

The foregoing description is merely illustrative of the technical idea of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

As described above, the present invention can be applied to an image compression processing field for encoding and decoding a video, thereby further improving compression efficiency by removing temporal redundancy remaining in a motion compensation frame generated through motion estimation and motion compensation. In addition, since the quantization error can be further reduced by adaptively selecting and filtering the filter type and the filter coefficient according to the characteristics of the pixels of the block in the motion compensation frame, the compression efficiency can be further improved, or the code is decoded and reproduced. It is a very useful invention to produce an effect that can further reduce the deterioration of the image quality of the restored image.

1 is a block diagram schematically illustrating a video encoding apparatus according to an embodiment of the present invention;

2 is a block diagram schematically illustrating an inter prediction apparatus for image encoding according to an embodiment of the present invention;

3 is an exemplary view illustrating a process of generating a motion compensation frame according to an embodiment of the present invention;

4 is an exemplary view illustrating a process of filtering a motion compensation frame according to an embodiment of the present invention;

5 is a flowchart illustrating a video encoding method according to an embodiment of the present invention;

6 is a block diagram schematically illustrating an image decoding apparatus according to an embodiment of the present invention;

7 is a block diagram schematically illustrating an inter prediction apparatus for image decoding according to an embodiment of the present invention;

8 is a flowchart illustrating an image decoding method according to an embodiment of the present invention.

Claims (21)

In the method of encoding an image, Filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; And Transforming, quantizing, and encoding a residual frame that is a difference between the current frame and the filtered motion compensation frame Image encoding method comprising a. The method of claim 1, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a different filter according to a characteristic of a pixel of a block in the motion compensation frame. The method of claim 1, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a Wiener filter or a deblocking filter. The method of claim 1, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a filter having a filter coefficient that minimizes a square error between the current frame and the filtered motion compensation frame. The method of claim 1, wherein the video encoding method comprises: And encoding filter coefficient information indicating filter coefficients of a filter used to filter the motion compensation frame. The method of claim 1, wherein the filtering of the motion compensation frame comprises: Determining a motion vector and a reference frame index for a block in which the block mode is inter mode within the current frame; Generating a motion compensation block by using the determined motion vector and a reference frame index and generating the motion compensation frame including the generated motion compensation block; And Filtering the motion compensation frame Image encoding method comprising a. The method of claim 6, wherein the video encoding method comprises: Encoding block mode information indicating the block mode; And Encoding motion information identifying the determined motion vector and a reference frame index. The video encoding method further comprises. The method of claim 1, wherein the video encoding method comprises: Inverse quantization and inverse transformation of the transformed and quantized residual frame to restore the residual frame; Reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame; And Deblocking filtering and storing the restored current frame The video encoding method further comprises. In the apparatus for encoding a video, A predictor for filtering a motion compensation frame generated by estimating and compensating for a motion of a current frame of the image; A subtractor for generating a residual frame by subtracting the current frame and the filtered motion compensation frame; A converter and quantizer for transforming and quantizing the residual frame; And An encoder for encoding the transformed and quantized residual frames An image encoding apparatus comprising a. The method of claim 9, wherein the predictor, A motion estimator for determining a motion vector and a reference frame index for a block in which the block mode is inter mode within the current frame; A motion compensator for generating a motion compensation block by using the determined motion vector and the reference frame index and generating the motion compensation frame including the motion compensation block; And A motion compensation filter for filtering the motion compensation frame An image encoding apparatus comprising a. In the method of decoding an image, Generating a motion compensation frame by compensating for the motion of the current frame by using a motion vector identified by the motion information reconstructed by decoding the motion information data extracted from the bitstream and a reference frame index, and filtering the motion compensation frame; And Reconstructing the current frame by adding the filtered motion compensation frame and the residual frame extracted from the bitstream and decoded and reconstructed Image decoding method comprising a. The method of claim 11, wherein the filtering of the motion compensation frame comprises: Restoring motion information by extracting and decoding the motion information data from the bitstream with respect to a block in which the block mode is interblock in the current frame; Generating a motion compensation block using a motion vector identified by the reconstructed motion information and a reference frame index, and generating a motion compensation frame including the generated motion compensation block; And Filtering the motion compensation frame Image decoding method comprising a. The method of claim 11, wherein the image decoding method comprises: Extracting and decoding filter coefficient data from the bitstream to restore filter coefficient information, wherein filtering the motion compensation frame comprises filtering a filter having a filter coefficient identified by the reconstructed filter coefficient information. And the motion compensation frame is filtered using the image decoding method. The method of claim 11, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a filter having a predetermined filter coefficient. The method of claim 11, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a different filter according to the characteristics of the pixels of the block in the motion compensation frame. The method of claim 11, wherein the filtering of the motion compensation frame comprises: And the motion compensation frame is filtered using a Wiener filter or a deblocking filter. In the apparatus for decoding an image, A decoder configured to decode the motion information data and the image encoded data extracted from the bitstream and to restore the motion information and the residual and quantized frames; An inverse quantizer and an inverse converter that inverse quantizes and inverse transforms the reconstructed transformed and quantized residual frame to reconstruct the residual frame;  A predictor for generating a motion compensation frame by compensating for the motion of the current frame by using the motion vector identified by the reconstructed motion information and a reference frame index, and filtering the motion compensation frame; And An adder for reconstructing the current frame by adding the reconstructed residual frame and the filtered motion compensation frame; Video decoding apparatus comprising a. In the inter prediction method, Determining a motion vector and a reference frame index for a block in which a block mode is inter mode within a current frame of an image; Generating a motion compensation block using the determined motion vector and the reference frame index and generating a motion compensation frame including the motion compensation block; And Filtering the motion compensation frame Inter prediction method comprising a. In the inter prediction apparatus, A motion estimator for determining a motion vector and a reference frame index for a block in which a block mode is inter mode within a current frame of an image; A motion compensator for generating a motion compensation block using the determined motion vector and the reference frame index and generating a motion compensation frame including the motion compensation block; And A motion compensation filter for filtering the motion compensation frame Inter prediction apparatus comprising a. In the inter prediction method, Generating a motion compensation block using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and decoded and reconstructed, and generating a motion compensation frame including the motion compensation block; And Filtering the motion compensation frame Inter prediction method comprising a. In the inter prediction apparatus, A motion compensator for generating a motion compensation block using a motion vector and a reference frame index identified by the motion information extracted from the bitstream and reconstructed from the bitstream, and generating a motion compensation frame including the motion compensation block; And A motion compensation filter for filtering the motion compensation frame Inter prediction apparatus comprising a.

Images