WO2009028780A1

WO2009028780A1 - Method and apparatus for estimating and compensating spatiotemporal motion of image

Info

Publication number: WO2009028780A1
Application number: PCT/KR2008/002274
Authority: WO
Inventors: Il-Koo Kim; Woo-Jin Han
Original assignee: Samsung Electronics Co., Ltd.
Priority date: 2007-08-28
Filing date: 2008-04-23
Publication date: 2009-03-05
Also published as: EP2186341A1; KR20090021758A; CN101796844A; EP2186341A4; CN101796844B; US20090060359A1; US20120263239A1; US8229233B2; KR101396365B1

Abstract

A method of estimating and compensating for motion in image decoding are provided. The method involves determining a reference block of a reference frame indicated by a motion vector of a current block of a current frame being decoded, and generating a spatiotemporal estimation block of the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame.

Description

METHOD AND APPARATUS FOR ESTIMATING AND COMPENSATING SPATIOTEMPORAL MOTION OF IMAGE

Technical Field

[1] The present invention relates to a method of estimating and compensating for motion in image encoding and decoding, and more particularly, to a method of estimating and compensating for motion when a current block refers to a region that is outside a reference frame by using a motion vector in image encoding and decoding. Background Art

[2] Video codecs such as H.263, Moving Picture Experts Group 2 (MPEG 2), MPEG 4,

H.264/ Advanced Video Coding (AVC), etc. use correlation between frames, perform a method of estimating and compensating for motion, and thereby increase compression efficiency, even when motion of an object appears in an image.

[3] The method of estimating and compensating for motion refers to a reference block of a reference frame to reconstruct a block of a current frame. However, a part of the reference block determined by a motion vector may be outside the reference frame. The conventional video codecs such as H.263, MPEG 2, MPEG 4, H.264/AVC, etc. extend an outside region of the reference frame by using boundary pixels of the reference frame, and thereby estimate the part of the reference block which is outside the reference frame.

[4] FIG. 1 is a diagram of a conventional method of estimating and compensating for motion when a reference block is outside a reference frame.

[5] A current block 120 of a current frame 110 refers to a reference block 160 of a reference frame 150 by a motion vector. A region 170 included in the reference frame 150 is inserted into an equivalent region 130 of the current block 120. However, when a part 180 of the reference block 160 is outside the reference frame 150, an equivalent region 140 of the current block 120 referring to the part 180 that is outside the reference frame 150 is padded with pixel values of boundary pixels 190 of the reference frame 150. This conventional technique is known as a padding method.

[6] When motion occurs in an outside region of a frame, compression efficiency of a block which refers to the outside of the frame by using the conventional simple padding method is decreased. In particular, compression efficiency of an active image sequence having panning, tilting, zooming in/out, fast camera motion, and fast object motion is greatly decreased.

[7] The conventional padding method extends an outside region of a frame prior to performing a method of compensating for motion, and as such only limited in- formation is used in the padding method. Thus, a region that is related to motion and is outside the frame is not accurately estimated. Disclosure of Invention Technical Solution

[8] The present invention provides a method and apparatus for accurately estimating and compensating for motion by using information of a region included in a neighboring block of a current block and a reference frame, when a reference block selected by a motion vector of the current block has a region that is outside the reference frame.

[9] The present invention also provides a method and apparatus for estimating and compensating for motion, wherein the method and apparatus use correlation with a neighboring block and a reference frame, and can thereby greatly enhance compression efficiency of an active image sequence having panning, tilting, zooming in/out, fast camera motion, and fast object motion. Advantageous Effects

[10] While the conventional technology uses a padding method which depends on limited information corresponding to boundary pixels of a reference frame, the method and apparatus for estimating and compensating for motion in image decoding according to the present invention use the information of the region included in the neighboring block and the reference frame. By doing so, the method and apparatus has reflected therein correlation with the reference frame, thereby has high compression efficiency. In particular, the compression efficiency of an active image sequence having panning, tilting, zooming in/out, fast camera motion, and fast object motion is greatly increased.

[11] Among exemplary embodiments of the present invention, an effect of the method of estimating and compensating for motion in image decoding is higher than an effect of the method of estimating and compensating for motion in image encoding, in terms of calculation time.

[12] The present invention can be applied to video codecs based on temporal motion estimation, or to all methods and apparatuses such as mobile phones, camcorders, digital cameras, Portable Multimedia Players (PMPs), next- generation Digital Video Discs (DVDs), software video codes, and the like which are capable of using the video codecs. Description of Drawings

[13] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:

[14] FIG. 1 is a diagram of a conventional method of estimating and compensating for motion when a reference block is outside a reference frame; [15] FIG. 2 is a block diagram illustrating a motion estimation and compensation apparatus for image decoding according to an exemplary embodiment of the present invention;

[16] FIG. 3 is a block diagram illustrating a motion estimation and compensation apparatus for image encoding according to another exemplary embodiment of the present invention;

[17] FIG. 4 is a diagram illustrating a method of generating an estimation block when a reference block is outside a reference frame, according to an exemplary embodiment of the present invention;

[18] FIG. 5 is a diagram illustrating an example of a type of a search pattern for estimating a region that is outside a reference frame;

[19] FIG. 6 A provides diagrams illustrating one of various methods of estimating a region that is outside a reference frame;

[20] FIG. 6B provides diagrams illustrating another method from among various methods of estimating a region that is outside a reference frame;

[21] FIG. 6C provides a diagram illustrating another method from among various methods of estimating a region that is outside a reference frame;

[22] FIG. 7 is a flowchart of a method of estimating and compensating for motion in image decoding, according to another exemplary embodiment of the present invention; and

[23] FIG. 8 is a flowchart of a method of estimating and compensating for motion in image encoding, according to another exemplary embodiment of the present invention. Best Mode

[24] According to an aspect of the present invention, there is provided a method of estimating and compensating for motion in image decoding, the method including the operations of determining a reference block of a reference frame indicated by a motion vector of a current block of a current frame being decoded; and generating a spati- otemporal estimation block of the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame.

[25] When a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the operation of generating the spatiotemporal estimation block may include the operations of determining the first block region of the reference block to be an estimation region of the first block region of the spatiotemporal estimation block; and generating an estimation region of the second block region of the spatiotemporal estimation block by using at least one of pixel values of neighboring blocks of the current block, and pixel values of an estimation region of the first block region.

[26] When a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the operation of generating the spatiotemporal estimation block may include the operations of determining pixels of the neighboring blocks of the current block and pixels surrounding the second block region to be a search pattern region, wherein the neighboring blocks are from among regions reconstructed prior to the current block, and the pixels surrounding the second block region are part of the first block region of the spatiotemporal estimation block, when the spatiotemporal estimation block is allocated to the current block; searching for an outside estimation region surrounded by a similar pattern region having a minimum difference compared with the search pattern region in the regions reconstructed prior to the current block; and determining the outside estimation region to be an estimation region of the second block region of the spatiotemporal estimation block.

[27] The operation of searching for the outside estimation region may include the operation of searching for the similar pattern region having a same form as the search pattern region in the regions reconstructed prior to the current block, and having a minimum Sum of Absolute Differences (SAD) between pixel values of the search pattern region and pixel values of the similar pattern region.

[28] The operation of determining the estimation region of the second block region of the spatiotemporal estimation block may include the operations of selecting at least one of a pixel line of the neighboring blocks of the current block and a pixel line surrounding the second block region, wherein the pixel line surrounding the second block region is a part of the first block region of the spatiotemporal estimation block, when the spatiotemporal estimation block is allocated to the current block; and determining pixels of the selected pixel line to be estimation pixels related to pixels of the second block region located either vertically or diagonally to the selected pixel line.

[29] The operation of determining the estimation region of the second block region of the spatiotemporal estimation block may include the operation of determining an average value between pixel values of the neighboring blocks of the current block and pixel values surrounding the second block region to be an estimation pixel value of the second block region, wherein the pixel values surrounding the second block region are a part of the first block region of the spatiotemporal estimation block, when the spatiotemporal estimation block is allocated to the current block.

[30] According to another aspect of the present invention, there is provided a method of estimating and compensating for motion in image encoding, the method including the operations of searching for a reference block having a minimum difference compared with a current block in a reference frame, calculating a motion vector, and thereby performing motion estimation; generating an estimation block for the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame; and encoding an image by using the estimation block and the motion vector.

[31] When a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the operation of generating the estimation block may include the operations of determining the first block region of the reference block to be an estimation region of the first block region of the estimation block; and generating an estimation region of the second block region of the estimation block by using at least one of pixels of neighboring blocks of the current block and pixels of an estimation region of the first block region.

[32] The operation of generating the estimation region of the second block region of the estimation block may include the operations of determining pixels of the neighboring blocks of the current block and pixels surrounding the second block region to be a search pattern region, wherein the neighboring blocks are from among regions reconstructed prior to the current block, and the pixels surrounding the second block region are of a part of the first block region of the estimation block, when the estimation block is allocated to the current block; searching for an outside estimation region surrounded by a similar pattern region having a minimum difference compared with the search pattern region in the regions reconstructed prior to the current block; and determining the outside estimation region to be an estimation region of the second block region of the estimation block.

[33] The operation of searching for the outside estimation region may include the operation of searching for the similar pattern region having a same form as the search pattern region in the regions reconstructed prior to the current block, and having a minimum SAD between pixel values of the search pattern region and pixel values of the similar pattern region.

[34] The operation of generating the estimation region of the second block region of the estimation block may include the operations of selecting at least one of a pixel line of the neighboring blocks of the current block and a pixel line surrounding the second block region, wherein the pixel line surrounding the second block region is a part of the first block region of the estimation block, when the estimation block is allocated to the current block; and determining pixels of the selected pixel line to be estimation pixels related to pixels of the second block region located either vertically or diagonally to the selected pixel line.

[35] The operation of generating the estimation region of the second block region of the estimation block may include the operation of determining an average value between pixel values of the neighboring blocks of the current block and pixel values surrounding the second block region to be an estimation pixel value of the second block region, wherein the pixel values surrounding the second block region are a part of the first block region of the estimation block, when the estimation block is allocated to the current block.

[36] According to another aspect of the present invention, there is provided a motion estimation and compensation apparatus in image decoding, the motion estimation and compensation apparatus including a reference block determining unit determining a reference block of a reference frame for a current block of a current frame being decoded; and a spatiotemporal estimation block generation unit generating a spati- otemporal estimation block of the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame.

[37] According to another aspect of the present invention, there is provided a motion estimation and compensation apparatus in image encoding, the motion estimation and compensation apparatus including a motion estimation performing unit searching for a reference block having a minimum difference compared with a current block in a reference frame, calculating a motion vector, and thereby performing motion estimation; an estimation block generation unit generating an estimation block for the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame; and an encoding unit encoding an image by using the estimation block and the motion vector.

[38] According to another aspect of the present invention, there is provided a computer readable recording medium having recorded thereon a program for executing the method of estimating and compensating for motion in image decoding.

[39] According to another aspect of the present invention, there is provided a computer readable recording medium having recorded thereon a program for executing the method of estimating and compensating for motion in image encoding. Mode for Invention

[40] This application claims priority from Korean Patent Application No.

10-2007-0086549, filed on August 28, 2007, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.

[41] Hereinafter, a method and apparatus for estimating and compensating for motion in image decoding, and a method and apparatus for encoding an image, according to exemplary embodiments of the present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown.

[42] FIG. 2 is a block diagram illustrating a motion estimation and compensation apparatus 200 for image decoding according to an embodiment of the present invention.

[43] The motion estimation and compensation apparatus 200 for image decoding according to the current embodiment of the present invention includes a reference block determination unit 210, and a spatiotemporal estimation block generation unit 220. The spatiotemporal estimation block generation unit 220 includes a first block region determination unit 230, and a second block region determination unit 240.

[44] The reference block determination unit 210 determines a reference block of a reference frame for a current block of a current frame being decoded.

[45] When some pixels of the reference block are outside the reference frame, the spatiotemporal estimation block generation unit 220 generates a spatiotemporal estimation block of the current block by using the current frame and the reference frame.

[46] For convenience of description, it is assumed that a block region in the same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in the same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block.

[47] The first block region determination unit 230 determines the first block region of the reference block to be an estimation region of a first block region of the spatiotemporal estimation block.

[48] The second block region determination unit 240 generates an estimation region of a second block region of the spatiotemporal estimation block by using at least one of pixel values of neighboring blocks of the current block and pixel values of the estimation region of the first block region.

[49] Embodiments related to the second block region determination unit 240 will be described later in detail with reference to FIGS. 4 through 6C.

[50] An aspect of the present invention determines blocks estimated by the reference block, and in particular, the spatiotemporal estimation blocks generated by the spatiotemporal estimation block generation unit 220 when the reference block is outside the reference frame, to be a current block, and as such an image is reconstructed during image decoding.

[51] FIG. 3 is a block diagram illustrating a motion estimation and compensation apparatus 300 for image encoding according to another embodiment of the present invention.

[52] The motion estimation and compensation apparatus 300 for image encoding according to the current embodiment of the present invention includes a motion estimation performing unit 310, an estimation block generation unit 320, and an encoding unit 330.

[53] The motion estimation performing unit 310 searches for a reference block in a reference frame, wherein the reference block has a minimum difference compared with a current block, calculates a motion vector, and thereby performs motion estimation.

[54] The motion estimation performing unit 310 may determine a block to be the reference block, wherein the block has a minimum Sum of Absolute Differences (SAD) between each pixel value of a block in the reference frame and each pixel value of the current block. The motion vector may indicate a positional distance between the current block and the reference block.

[55] When some pixels of the reference block are outside the reference frame, the estimation block generation unit 320 generates an estimation block for the current block by using the current frame and the reference frame.

[56] An embodiment for the estimation block generation unit 320 is not illustrated.

However, similar to the motion estimation and compensation apparatus 200 for image decoding, the estimation block generation unit 320 includes a first block region determination unit and a second block region determination unit which are respectively for a first block region and a second block region of the estimation block. The first and second block region determination units of the estimation block generation unit 320 respectively have the same operating principle as the first and second block region determination units of the spatiotemporal estimation block generation unit 220. Thus, the operating principle of the first and second block region determination units of the estimation block generation unit 320 will be described later with reference to FIGS. 4 through 6C.

[57] The encoding unit 330 encodes an image by using the estimation block and the motion vector.

[58] The encoding unit 330 may determine the estimation block to be the current block, and encode the image by using a difference value between the current block and the reference block.

[59] FIG. 4 is a diagram illustrating a method of generating an estimation block when a reference block is outside a reference frame, according to an embodiment of the present invention.

[60] Referring to FIG. 4, a method of operating the spatiotemporal estimation block generation unit 220 of the motion estimation and compensation apparatus 200 for image decoding, and the estimation block generation unit 320 of the motion estimation and compensation apparatus 300 for image encoding will now be described in detail. In particular, a method of operating the first and second block region determination units of the spatiotemporal estimation block generation unit 220 and the estimation block generation unit 320 will now be described in detail.

[61] A frame 110, a frame 150, and a block 160 respectively indicate a current frame, a reference frame, and a reference block.

[62] A region 170 of the reference block 160 indicates a region included in the reference frame 150. A region 180 of the reference block 160 indicates a region excluded from the reference frame 150.

[63] A block 420 indicates a spatiotemporal estimation block. Since the spatiotemporal estimation block generated by the present invention is determined to be a current block, the block 420 eventually becomes a current block 120.

[64] A region 430 indicates a first block region of the spatiotemporal estimation block

420.

[65] A region 440 indicates a second block region of the spatiotemporal estimation block

420.

[66] A region 450 indicates pixels surrounding the current block 120, wherein the pixels are from among neighboring blocks of the current block 120.

[67] When the reference block 160 related to the current block 120 is inside the reference frame 150, an estimation block related to the current block 120 can be determined by a motion estimation/compensation method that is well-known to one of ordinary skill in the art. Hereinafter, a case in which the reference block 160 is outside the reference frame 150 will now be described in detail.

[68] As described above, it is assumed that a block region in the same position as the region 170 included in the reference frame 150 is defined to be a first block region, wherein the block region is a part of the reference block 160, and a block region in the same position as the region 180 excluded from the reference frame 150 is defined to be a second block region, wherein the block region is a part of the reference block 160. Thus, the region 430 and the region 440 respectively become the first block region and the second block region, which are of the spatiotemporal estimation block 420 that is to be inserted into the current block 120.

[69] The first block region determination unit 230 may determine the first block region

170 of the reference block 160 to be an estimation region of the first block region 430 of the spatiotemporal estimation block 420.

[70] The second block region determination unit 240 may generate an estimation region of the second block region 440 of the spatiotemporal estimation block 420 by using at least one of pixel values of neighboring blocks of the current block 120, and pixel values of the estimation region of the first block region 430 of the spatiotemporal es- timation block 420.

[71] The second block region determination unit 240 may use a method of using a search pattern including the pixels 450, and a method of using values of some of the pixels 450 without a change, wherein the pixels 450 surrounding the current block 120 are from among pixels of the neighboring blocks and first block region 430, so as to use the neighboring blocks of the current block 120 and the estimation region of the first block region 430 of the spatiotemporal estimation block 420. The method of using the search pattern will be described later with reference to FIG. 5, and the method of using the pixels 450 will be described later with reference to FIGS. 6 A through 6C.

[72] The first and second block region determination units of the estimation block generation unit 320 for image encoding respectively have the same operating principle as the first and second block region determination units 230 and 240 of the spatiotemporal estimation block generation unit 220 for image decoding.

[73] FIG. 5 is a diagram illustrating an example of a type of a search pattern for estimating a region that is outside a reference frame.

[74] A method of operating an example of the second block region determination unit 240 of the spatiotemporal estimation block generation unit 220 for image decoding, and an example of the second block region determination unit of the estimation block generation unit 320 for image encoding will now be described in detail with reference to FIG. 5.

[75] A region 510 indicates a search pattern region for determining the second block region 440.

[76] A region 520 indicates a similar pattern region searched for by the search pattern region 510.

[77] A region 530 indicates an outside estimation region.

[78] The second block region determination unit 240 may determine pixels of the neighboring blocks of the current block 120 and pixels surrounding the second block region 440 to be the search pattern region 510, wherein the neighboring blocks are from among regions reconstructed prior to the current block 120, and the pixels surrounding the second block region 440 are part of the first block region 430 of the spatiotemporal estimation block (estimation block) 420.

[79] Also, the second block region determination unit 240 may search for the similar pattern region 520 having a minimum difference compared with the search pattern region 510 in the regions reconstructed prior to the current block 120. The similar pattern region 520 has the same form as the search pattern region 510, and a region having a minimum SAD (that is the sum of the absolute values of the difference between pixel values of the search pattern region 510 and pixel values of the similar pattern region 520) in the regions reconstructed prior to the current block 120 is de- termined to be the similar pattern region 520.

[80] The outside estimation region 530 corresponds to a region surrounded by the similar pattern region 520 determined by the search pattern region 510. The second block region determination unit 240 determines the outside estimation region 530 to be an estimation region of the second block region 440, and thereby completes the spati- otemporal estimation block (estimation block) 420.

[81] FIGS. 6A through 6C are diagrams illustrating various methods of estimating a region that is outside a reference frame, according to another embodiment of the present invention.

[82] A method of operating an example for the second block region determination unit

240 of the spatiotemporal estimation block generation unit 220 for image decoding, and an example of the second block region determination unit of the estimation block generation unit 320 for image encoding will now be described in detail with reference to FIGS. 6A through 6C. In particular, the method is related to a method of determining the second block region 440 by using the aforementioned pixels 450.

[83] FIG. 6A provides diagrams illustrating one of the various methods of estimating the region that is outside the reference frame.

[84] A block 610 indicates the spatiotemporal estimation block (estimation block) 420 in which each pixel value of the first block region 430 is inserted vertically to a boundary line and to each pixel of the second block region 440.

[85] A block 620 indicates the spatiotemporal estimation block (estimation block) 420 in which each pixel value of a previously reconstructed neighboring block is inserted vertically to a boundary line and to each pixel of the second block region 440.

[86] A block 630 indicates the spatiotemporal estimation block (estimation block) 420 in which each pixel value of the previously reconstructed neighboring block is inserted horizontally to a boundary line and to each pixel of the second block region 440.

[87] The pixel values of the previously reconstructed neighboring block that is the nearest block to the second block region 440 of the current block 120 are inserted vertically and horizontally to the boundary line and to each pixel of the second block region 440 from the boundary lines between the current block 120 and the previously reconstructed neighboring block, such that the estimation blocks 620 and 630 are generated.

[88] The pixel values of the first block region 430 that is the nearest block region to the second block region 440 are inserted vertically to each pixel of the second block region 440 from the boundary line between the first block region 430 and the second block region 440, and such that the estimation block 610 is generated.

[89] FIG. 6B provides diagrams illustrating another method from among the various methods of estimating the region that is outside the reference frame.

[90] A block 640 indicates the spatiotemporal estimation block (estimation block) 420 in which the pixel values of the previously reconstructed neighboring block are inserted diagonally left to each pixel of the second block region 440.

[91] A block 650 indicates the spatiotemporal estimation block (estimation block) 420 in which the pixel values of the previously reconstructed neighboring block are inserted diagonally right to each pixel of the second block region 440.

[92] A block 660 indicates the spatiotemporal estimation block (estimation block) 420 in which each pixel value of the first block region 430 is inserted diagonally right to each pixel of the second block region 440.

[93] A block 670 indicates the spatiotemporal estimation block (estimation block) 420 in which each pixel value of the first block region 430 is inserted diagonally left to each pixel of the second block region 440.

[94] The pixel values of the previously reconstructed neighboring block that is the nearest block to the second block region 440 of the current block 120 are inserted diagonally to the boundary line and to each pixel of the second block region 440 from the boundary line between the current block 120 and the previously reconstructed neighboring block, such that the estimation blocks 640 and 650 are generated.

[95] The pixel values of the first block region 430 that is the nearest block region to the second block region 440 are inserted diagonally to each pixel of the second block region 440 from the boundary line between the first block region 430 and the second block region 440, such that the estimation blocks 660 and 670 are generated.

[96] FIG. 6C provides a diagram illustrating another method from among the various methods of estimating the region that is outside the reference frame.

[97] A block 680 indicates the spatiotemporal estimation block (estimation block) 420 in which an average value of pixel values included in the region 510 is determined by estimation pixel values of the second block region 440.

[98] The estimation block 680 determines an average value of pixel values of the previously reconstructed neighboring blocks from among neighboring blocks of the current block 120, and a region that is the same as the search pattern region 510 that comprises pixels of the first block region 430, wherein the pixels are the nearest pixels to the second block region 440, to be an estimation region.

[99] FIG. 7 is a flowchart of a method of estimating and compensating for motion in image decoding, according to another embodiment of the present invention.

[100] In operation 710, a reference block of a reference frame for a current block of a current frame is determined by using a received frame and a motion vector.

[101] In operation 720, when some pixels of the reference block are outside the reference frame, a spatiotemporal estimation block for the current block is generated by using the current frame and the reference frame.

[102] In the current embodiment, the spatiotemporal estimation block is divided into a first block region corresponding to a region included in the reference frame, wherein the region is a part of the reference block, and a second block region corresponding to a region excluded from the reference frame, wherein the region is a part of the reference block, and as such the spatiotemporal estimation block corresponding to the respective regions is generated. That is, temporal estimation using pixel values of the reference block is performed on the first block region of the spatiotemporal estimation block, and spatial estimation using pixel values of a neighboring block of the current block is performed on the second block region of the spatiotemporal estimation block.

[103] FIG. 8 is a flowchart of a method of estimating and compensating for motion in image encoding, according to another embodiment of the present invention.

[104] In operation 810, a reference block having a minimum difference compared with a current block is searched for in a reference frame, a motion vector is calculated, and thereby motion estimation is performed.

[105] In operation 820, when some pixels of the reference block are outside the reference frame, an estimation block for the current block is generated by using a current frame and the reference frame.

[106] In the current embodiment, the spatiotemporal estimation block is divided into a first block region corresponding to a region included in the reference frame, wherein the region is a part of the reference block, and a second block region corresponding to a region excluded from the reference frame, wherein the region is a part of the reference block, and as such the spatiotemporal estimation block corresponding to the respective regions is generated. That is, temporal estimation using pixel values of the reference block is performed on the first block region of the spatiotemporal estimation block, and spatial estimation using pixel values of a neighboring block of the current block is performed on the second block region of the spatiotemporal estimation block.

[107] In operation 830, an image is encoded by using the estimation block and the motion vector. In this manner, motion compensation is performed during image encoding and then the image to be encoded is reconstructed. Thus, it is possible to use the image in which information regarding motion is more accurately reflected.

[108] When the reference block, which is selected by the motion vector, for the current block has the region that is outside the reference frame, the method and apparatus for estimating and compensating for the motion in image decoding according to the present invention use information of the region included in the neighboring block of the current block and the reference frame. Accordingly, the method and apparatus can accurately estimate and compensate for the motion in comparison with the conventional technology.

[109] While the conventional technology uses a padding method which depends on limited information corresponding to boundary pixels of a reference frame, the method and apparatus for estimating and compensating for motion in image decoding according to the present invention use the information of the region included in the neighboring block and the reference frame. By doing so, the method and apparatus has reflected therein correlation with the reference frame, thereby has high compression efficiency. In particular, the compression efficiency of an active image sequence having panning, tilting, zooming in/out, fast camera motion, and fast object motion is greatly increased.

[110] Among exemplary embodiments of the present invention, an effect of the method of estimating and compensating for motion in image decoding is higher than an effect of the method of estimating and compensating for motion in image encoding, in terms of calculation time.

[I l l] The present invention can be applied to video codecs based on temporal motion estimation, or to all methods and apparatuses such as mobile phones, camcorders, digital cameras, Portable Multimedia Players (PMPs), next- generation Digital Video Discs (DVDs), software video codes, and the like which are capable of using the video codecs.

[112] Exemplary embodiments of the present invention can be written as computer programs and can be implemented in general-use digital computers that execute the programs using a computer readable recording medium and other media. In addition, a data structure used in the embodiments of the present invention can be written in a computer readable recording medium through various means. Examples of the computer readable recording medium include magnetic storage media (e.g., ROM, floppy disks, hard disks, etc.) and optical recording media (e.g., CD-ROMs, or DVDs)An example of other media is carrier waves (e.g., transmission through the Internet).

[113] While this invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The exemplary embodiments should be considered in a descriptive sense only and not for purposes of limitation. Therefore, the scope of the invention is defined not by the detailed description of the invention but by the appended claims, and all differences within the scope will be construed as being included in the present invention.

Claims

[1] L A method of estimating and compensating for motion in image decoding, the method comprising: determining a reference block of a reference frame indicated by a motion vector of a current block of a current frame being decoded; and generating a spatiotemporal estimation block of the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame.

[2] 2. The method of claim 1, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the generating of the spatiotemporal estimation block comprises: determining the first block region of the reference block to be an estimation region of the first block region of the spatiotemporal estimation block; and generating an estimation region of the second block region of the spatiotemporal estimation block by using at least one of pixel values of neighboring blocks of the current block, and pixel values of an estimation region of the first block region.

[3] 3. The method of claim 1, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the generating of the spatiotemporal estimation block comprises: determining pixels of the neighboring blocks of the current block and pixels surrounding the second block region to be a search pattern region, wherein the neighboring blocks are from among regions reconstructed prior to the current block, and the pixels surrounding the second block region are part of the first block region of the spatiotemporal estimation block, when the spatiotemporal estimation block is allocated to the current block; searching for an outside estimation region surrounded by a similar pattern region having a minimum difference compared with the search pattern region in the regions reconstructed prior to the current block; and determining the outside estimation region to be an estimation region of the second block region of the spatiotemporal estimation block.

[4] 4. The method of claim 3, wherein the searching for the outside estimation region comprises searching for the similar pattern region having a same form as the search pattern region in the regions reconstructed prior to the current block, and having a minimum SAD (Sum of Absolute Differences) between pixel values of the search pattern region and pixel values of the similar pattern region.

[5] 5. The method of claim 3, wherein the determining of the estimation region of the second block region of the spatiotemporal estimation block comprises: selecting at least one of a pixel line of the neighboring blocks of the current block and a pixel line surrounding the second block region, wherein the pixel line surrounding the second block region is a part of the first block region of the spatiotemporal estimation block, when the spatiotemporal estimation block is allocated to the current block; and determining pixels of the selected pixel line to be estimation pixels related to pixels of the second block region located either vertically or diagonally to the selected pixel line.

[6] 6. The method of claim 3, wherein the determining of the estimation region of the second block region of the spatiotemporal estimation block comprises determining an average value between pixel values of the neighboring blocks of the current block and pixel values surrounding the second block region to be an estimation pixel value of the second block region, wherein the pixel values surrounding the second block region are a part of the first block region of the spatiotemporal estimation block, when the spatiotemporal estimation block is allocated to the current block.

[7] 7. A method of estimating and compensating for motion in image encoding, the method comprising: searching for a reference block having a minimum difference compared with a current block in a reference frame, calculating a motion vector, and thereby performing motion estimation; generating an estimation block for the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame; and encoding an image by using the estimation block and the motion vector.

[8] 8. The method of claim 7, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the generating of the estimation block comprises: determining the first block region of the reference block to be an estimation region of the first block region of the estimation block; and generating an estimation region of the second block region of the estimation block by using at least one of pixels of neighboring blocks of the current block and pixels of an estimation region of the first block region.

[9] 9. The method of claim 8, wherein the generating of the estimation region of the second block region of the estimation block comprises: determining pixels of the neighboring blocks of the current block and pixels surrounding the second block region to be a search pattern region, wherein the neighboring blocks are from among regions reconstructed prior to the current block, and the pixels surrounding the second block region are part of the first block region of the estimation block, when the estimation block is allocated to the current block; searching for an outside estimation region surrounded by a similar pattern region having a minimum difference compared with the search pattern region in the regions reconstructed prior to the current block; and determining the outside estimation region to be an estimation region of the second block region of the estimation block.

[10] 10. The method of claim 9, wherein the searching for the outside estimation region comprises searching for the similar pattern region having a same form as the search pattern region in the regions reconstructed prior to the current block, and having a minimum SAD between pixel values of the search pattern region and pixel values of the similar pattern region.

[11] 11. The method of claim 8, wherein the generating of the estimation region of the second block region of the estimation block comprises: selecting at least one of a pixel line of the neighboring blocks of the current block and a pixel line surrounding the second block region, wherein the pixel line surrounding the second block region is a part of the first block region of the estimation block, when the estimation block is allocated to the current block; and determining pixels of the selected pixel line to be estimation pixels related to pixels of the second block region located either vertically or diagonally to the selected pixel line.

[12] 12. The method of claim 8, wherein the generating of the estimation region of the second block region of the estimation block comprises determining an average value between pixel values of the neighboring blocks of the current block and pixel values surrounding the second block region to be an estimation pixel value of the second block region, wherein the pixel values surrounding the second block region are a part of the first block region of the estimation block, when the estimation block is allocated to the current block.

[13] 13. A motion estimation and compensation apparatus in image decoding, the motion estimation and compensation apparatus comprising: a reference block determining unit which determines a reference block of a reference frame for a current block of a current frame being decoded; and a spatiotemporal estimation block generation unit which generates a spati- otemporal estimation block of the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame.

[14] 14. The motion estimation and compensation apparatus of claim 13, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the spatiotemporal estimation block generation unit comprises: a first block region determination unit which determines the first block region of the reference block to be an estimation region of the first block region of the spatiotemporal estimation block; and a second block region determination unit which determines pixels of neighboring blocks of the current block and pixels surrounding the second block region to be a search pattern region, wherein the pixels surrounding the second block region are part of the first block region of the spatiotemporal estimation block, and determines a region corresponding to the second block region of the spatiotemporal estimation block in the current frame by using the determined search pattern region, when the spatiotemporal estimation block is allocated to the current block.

[15] 15. The motion estimation and compensation apparatus of claim 13, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the spatiotemporal estimation block generation unit comprises: a first block region determination unit which determines the first block region of the reference block to be an estimation region of the first block region of the spatiotemporal estimation block; and a second block region determination unit which selects at least one of a pixel line of the neighboring blocks of the current block and a pixel line surrounding the second block region, wherein the pixel line surrounding the second block region is a part of the first block region of the spatiotemporal estimation block, and determines an estimation region of the second block region of the spatiotemporal estimation block in the current frame by using the selected pixel line, when the spatiotemporal estimation block is allocated to the current block.

[16] 16. A motion estimation and compensation apparatus in image encoding, the motion estimation and compensation apparatus comprising: a motion estimation performing unit which searches for a reference block having a minimum difference compared with a current block in a reference frame, calculating a motion vector, and thereby performs motion estimation; an estimation block generation unit which generates an estimation block for the current block by using the current frame and the reference frame, when some pixels of the reference block are outside the reference frame; and an encoding unit which encodes an image by using the estimation block and the motion vector.

[17] 17. The motion estimation and compensation apparatus of claim 16, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the estimation block generation unit comprises: a first block region determination unit which determines the first block region of the reference block to be an estimation region of the first block region of the estimation block; and a second block region determination unit which determines pixels of neighboring blocks of the current block and pixels surrounding the second block region to be a search pattern region, wherein the pixels surrounding the second block region are part of the first block region of the estimation block, and determines a region corresponding to the second block region of the estimation block in the current frame by using the determined search pattern region, when the estimation block is allocated to the current block.

[18] 18. The motion estimation and compensation apparatus of claim 16, wherein, when a block region in a same position as a region included in the reference frame is defined to be a first block region, wherein the block region is a part of the reference block, and a block region in a same position as a region excluded from the reference frame is defined to be a second block region, wherein the block region is a part of the reference block, the estimation block generation unit comprises: a first block region determination unit which determines the first block region of the reference block to be an estimation region of the first block region of the estimation block; and a second block region determination unit which selects at least one of a pixel line of the neighboring blocks of the current block and a pixel line surrounding the second block region, wherein the pixel line surrounding the second block region is a part of the first block region of the estimation block, and determining an estimation region of the second block region of the estimation block in the current frame by using the selected pixel line, when the estimation block is allocated to the current block.

[19] 19. A computer readable recording medium having recorded thereon a program for executing the method of claim 1.

[20] 20. A computer readable recording medium having recorded thereon a program for executing the method of claim 7.