KR100688383B1 - Motion estimation and compensation for panorama image - Google Patents

Abstract

The present invention relates to motion estimation and compensation of a panoramic image. The motion estimation and compensation of an image according to the present invention has a high spatial correlation between a right boundary portion and a left boundary portion of a panoramic image including a 360 degree omnidirectional image. By using the property, the motion prediction and compensation of the panoramic image is performed. According to the present invention, it is possible to improve the image quality by improving the efficiency and accuracy of motion estimation and compensation of the panoramic image. In particular, it is effective to improve the image quality of the right boundary and the left boundary of the panoramic image.

Description

Motion estimation and compensation for panorama image}

1 is a diagram illustrating a plurality of previous macroblocks used for prediction of a motion vector of a current macroblock according to the prior art;

2A to 2C are diagrams illustrating a case in which there are no previous macroblocks to use for prediction of a motion vector of a current macroblock;

3 is a view for explaining a padding method of a reference image according to the prior art;

4A illustrates a cylindrical image including a 360 degree omnidirectional image;

4B is a diagram illustrating a 2D panoramic image corresponding to the cylindrical image illustrated in FIG. 4A;

5 is a block diagram of a panoramic image motion vector encoder according to an embodiment of the present invention;

6 is a flowchart of a panoramic image motion estimation method according to an embodiment of the present invention;

7A is a diagram illustrating an example of previous macroblock selection to be used for prediction of a motion vector of a current macroblock;

7B is a diagram illustrating another example of previous macroblock selection to be used for prediction of a motion vector of a current macroblock;

8A illustrates a case in which a reference macroblock exists across a boundary of a reference picture;

8B is a diagram illustrating a case in which a reference macroblock exists in an outer region of a reference picture;

9 is a diagram illustrating a padding method of a reference image according to an embodiment of the present invention;

10 is a diagram illustrating a motion vector of a current macroblock X;

11 is a flowchart illustrating an embodiment of a method for searching for motion of a half pixel according to the present invention;

12 is an example of a basic input frame for explaining a process of searching for a motion shown in FIG. 11;

FIG. 13A is an example of a reference frame in which a padding area is added to a basic input frame shown in FIG. 12 and is enlarged twice to explain an embodiment of FIG. 11;

FIG. 13B is a reference diagram for describing processing of pixel values belonging to a padding area in a reference frame enlarged 2 times according to one embodiment of FIG. 11;

14 is a flowchart illustrating another embodiment of a method for searching a half-pixel motion according to the present invention;

FIG. 15A illustrates an example of a reference frame in which a padding area is added to a basic input frame shown in FIG. 12 and is enlarged twice to explain another embodiment of FIG. 14;

FIG. 15B is a reference diagram for describing processing of pixel values belonging to a padding area and pixel values located outside the reference frame in a reference frame enlarged twice according to another embodiment of FIG. 14;

16 is a block diagram of a panoramic image motion vector decoder according to an embodiment of the present invention;

17 is a flowchart illustrating a panoramic image motion compensation method according to an embodiment of the present invention.

The present invention relates to motion estimation and compensation of a panoramic image, and more particularly, to a motion estimation method, a motion estimation device, a motion compensation method, and a motion compensation device of a panoramic image including 360 degree omnidirectional image information.

An omni-directional video camera system refers to a camera system capable of capturing 360-degree omnidirectional images from a fixed viewpoint. Omni-directional video camera systems are equipped with a special type of mirror, such as a hyperbolic mirror, a special lens, such as a fisheye lens, or use a number of cameras to shoot omni-directional.

One example of applying the omnidirectional video coding method is 3D realistic broadcasting. For example, in a three-dimensional realistic broadcast, video information about a scene at various points of time in a baseball game or the like is all provided to the viewer terminal. That is, a variety of video information is provided to the viewer from the perspective of the pitcher, the perspective of the catcher, the perspective of the batter, and the perspective of the first-base spectator. The viewer can determine a point in time to watch and watch the image at that point in time.

The image captured by the omni-directional camera system has a feature corresponding to a three-dimensional spherical environment. Therefore, the 3D image photographed by the omnidirectional camera system is converted into a 2D planar image. In this case, the 2D planar image may be a panoramic image including a 360 degree omnidirectional image. Omni-directional video coding is performed on the 2D panoramic image.

Meanwhile, motion estimation, which is one of image encoding techniques, searches for a data unit most similar to a data unit in a current frame using a predetermined evaluation function in a previous frame to obtain a motion vector indicating a difference between the positions of both data units. It is the process of obtaining. Here, the data unit is generally mainly used macroblock of size 16X16, but blocks of various sizes, such as 16X8, 8X16 and 8X8 block may be used.

A motion estimation process according to an example of the prior art performed in units of 16 × 16 macroblocks will be described in more detail. First, a motion vector of the current macroblock is predicted using the motion vectors of a plurality of previous macroblocks adjacent to the current macroblock. 1 is a diagram illustrating a plurality of previous macroblocks used for prediction of a motion vector of a current macroblock. X represents the current macroblock, and A, B, C and D are the previous macroblocks already encoded before the current macroblock X.

However, depending on the position of the current macroblock X in the current frame, previous macroblocks are not available. 2A illustrates a case in which no previous macroblocks B, C, and D to be used for prediction of the motion vector of the current macroblock X exist. In this case, the motion vector of the current macroblock X is set to zero.

2B illustrates a case in which no previous macroblocks A and D to be used for prediction of the motion vector of the current macroblock X exist. In this case, the motion vectors of the previous macroblocks A and D are each set to zero. The motion vector of the current macroblock X is set to the median value of the previous macroblocks A, B, C and D.

2C illustrates a case in which no previous macroblock C to be used for prediction of the motion vector of the current macroblock X exists. In this case, the motion vectors of the previous macroblocks C are each set to zero. The motion vector of the current macroblock X is set to the median value of the previous macroblocks A, B, C and D.

When the predicted motion vector of the current macroblock X is obtained, the degree of similarity between the reference macroblock and the current macroblock in the reference frame indicated by the predicted motion vector is calculated using a predetermined evaluation function. The reference macroblock in the reference frame most similar to the current macroblock is then searched within a predetermined search range. As the predetermined evaluation function, a sum of absolute difference (SAD) function, a sum of absolute transformed difference (SATD) function, or a sum of squared difference (SSD) function is generally used.

When searching for a reference macroblock most similar to the current macroblock within a predetermined search range, some or all pixels of the reference macroblock may exist outside the reference picture. In this case, as shown in FIG. 3, the pixel value located at the left boundary of the reference image is padded to an area outside the left boundary of the reference image, and the pixel value located at the right boundary of the reference image is After padding the area outside the right boundary, motion prediction and compensation are performed. Motion prediction and compensation according to this method are called motion prediction and compensation according to the Unrestricted Motion Vector (UMV) mode.

4A illustrates a cylindrical image including a 360 degree omnidirectional image, and FIG. 4B illustrates a panoramic image including a 360 degree omnidirectional image generated by cutting the cylindrical image illustrated in FIG. 4A according to a reference line X. FIG. . Referring to FIG. 4B, the left portion A of the human-shaped object is located at the right boundary of the panoramic image, and the right portion B of the human-shaped object is located at the left boundary of the panoramic image. Can be. That is, the spatial correlation between the right boundary part and the left boundary part of the panoramic image including the 360 degree omnidirectional image is very high.

Without considering these characteristics of the panoramic image including the omnidirectional image information, it is not efficient to apply the above-described motion prediction and compensation method according to the prior art to the panoramic image including the omnidirectional image information as it is. Accordingly, a motion prediction and compensation method suitable for a panoramic image including omnidirectional image information is required.

SUMMARY OF THE INVENTION The present invention has been made in an effort to provide a more efficient and accurate motion estimation method and apparatus for the characteristics of a panoramic image including omnidirectional image information.

Another object of the present invention is to provide a more efficient and accurate motion compensation method and apparatus for the characteristics of a panoramic image including omnidirectional image information.

One feature of the present invention for achieving the above object is a motion estimation method of a panoramic image including 360-degree omnidirectional image information, the right border region in the left padding area of the basic reference image for motion estimation of the current panoramic image Padding using the padding, padding the left padding region on the right side of the basic reference image using a left boundary region, and then extending the padded image to set the reference image, and presenting the current data unit included in the current panoramic image. Predicting a motion vector of the current data unit using motion vectors of a plurality of adjacent previous data units; and if the subpixel included in the reference data unit represented by the predicted motion vector belongs to the reference image, The pixel value of the pixel is obtained as it is, and the subpixel is true. When located outside of an image, the distance on the x-axis between one boundary of the base reference image near the subpixel and the subpixel is added to or subtracted from the x coordinate of the other boundary of the base reference image. Obtaining all pixel values of the reference data unit in the reference image by setting the x coordinate of the subpixel to obtain pixel values of the subpixel, and using the evaluation function, the current data unit and the reference data unit Determining the similarity of.

According to another aspect of the present invention, in the motion estimation method of a panoramic image including 360-degree omnidirectional image information, the left padding area of the basic reference image for motion estimation of the current panoramic image is padded using a left boundary pixel value. And padding the right padding region of the basic reference image using a right boundary pixel value, and then expanding the padded image to set the reference image, and a plurality of previous ones adjacent to the current data unit included in the current panoramic image. Predicting a motion vector of the current data unit using motion vectors of data units; and if the subpixel included in a reference data unit represented by the predicted motion vector belongs to the basic reference picture of the reference picture Obtain a pixel value of a pixel, and the subpixels refer to the basic If the image falls outside of the image, the subpixel value is obtained by adding or subtracting a distance on the x-axis between one boundary of the base reference image and the subpixel closer to the subpixel from the x coordinate of the other boundary of the base reference image. Obtaining all pixel values of the reference data unit in the reference image by setting the x coordinate value of the sub-pixel to obtain a pixel value of the subpixel, and using the predetermined evaluation function, the current data unit and the reference data unit. Determining the similarity of.

According to still another aspect of the present invention, in a motion estimation apparatus for a panoramic image including 360-degree omnidirectional image information, the left padding region of the basic reference image for motion estimation of the current panoramic image is padded using a right boundary region. And a motion vector of a plurality of previous data units adjacent to the current data unit included in the reference image and the current panorama image generated by padding the right padding region by using a left boundary region and then expanding the padded image. And a subpixel included in the reference data unit represented by the predicted motion vector, and predicts the motion vector of the current data unit using the memory for storing the data and the motion vectors of the plurality of previous data units. If belonging, the pixel value of the sub-pixel is obtained as it is. If the subpixel is located outside the reference image, the distance on the x-axis between one boundary of the base reference image near the subpixel and the subpixel is x of the other boundary of the base reference image. After all the pixel values of the reference data unit are obtained from the reference image by setting a value added or subtracted from the coordinates to the x-coordinate of the subpixel to obtain the pixel value of the subpixel, the current value is obtained using a predetermined evaluation function. And a motion estimator for determining similarity between the data unit and the reference data unit.

According to still another aspect of the present invention, in a motion estimation apparatus for a panoramic image including 360-degree omnidirectional image information, the left padding region of the basic reference image for motion estimation of the current panoramic image is padded using a left boundary pixel value. In addition, padding is performed using a right boundary pixel value in a right padding area of the base reference image, and then a plurality of previous data units adjacent to a current data unit included in a reference image and a current panoramic image generated by expanding the padded image. The memory storing motion vectors and the motion vectors of the plurality of previous data units predict the motion vector of the current data unit, and the sub-pixel included in the reference data unit represented by the predicted motion vector indicates the reference. The incubation if it belongs to the base reference image of the image When a small pixel value is obtained and the subpixel belongs to the outside of the base reference image, the distance of one boundary of the base reference image near the subpixel and the subpixel on the x-axis is determined by the base reference image. After obtaining all the pixel values of the reference data unit in the reference image by setting a value added or subtracted from the x coordinate of another boundary of the subpixel to obtain the pixel value of the subpixel, thereby obtaining all the pixel values of the reference data unit in the reference image. And a motion estimator for determining similarity between the current data unit and the reference data unit using an evaluation function.

According to another aspect of the present invention, in the motion compensation method of a panoramic image including 360-degree omnidirectional image information, the left padding region of the basic reference image for motion compensation of the current panoramic image is padded using a right boundary region. And padding the left padding region of the basic reference image using a left boundary region, expanding the padded image to generate a reference image, and receiving a motion vector of the current data unit included in the current panoramic image. And the subpixel included in the reference data unit indicated by the motion vector of the current data unit belong to the reference image, the pixel value of the subpixel is acquired as it is, and the subpixel is located outside the reference image. And one boundary of the base reference image close to the subpixel and the subpixel. The reference data in the reference image is obtained by setting a value of the sub-pixel to the x-coordinate of the sub-pixel by adding or subtracting the distance on the x-axis from the x-axis of another boundary of the basic reference image. Obtaining all pixel values of the unit, and reproducing the current data unit using the pixel values of the reference data unit.

According to another aspect of the present invention, in a motion compensation method for a panoramic image including 360-degree omnidirectional image information, padding is performed using a left boundary pixel value in a left padding area of a basic reference image for motion compensation of a current panoramic image. And padding the right padding region of the basic reference image using a right boundary pixel value, generating a reference image by expanding the padded image, and inputting a motion vector of the current data unit included in the current panorama image. And a pixel value of the sub-pixel when the sub-pixel included in the reference data unit indicated by the motion vector of the current data unit belongs to the base reference image of the reference image, and the sub-pixel receives the base reference. If it is outside of the image, the lower of the base reference image near the subpixel The pixel value of the subpixel is obtained by setting a value of the subpixel by adding or subtracting the distance on the x-axis between the boundary of the subpixel and the subpixel to the x coordinate value of the subpixel. Obtaining all pixel values of the reference data unit from a reference image, and reproducing the current data unit using the pixel values of the reference data unit.

According to another aspect of the present invention, in a motion compensation apparatus for a panoramic image including 360-degree omnidirectional image information, the left padding region of the basic reference image for motion compensation of the current panoramic image is padded using a right boundary region. And a memory for storing a reference image generated by padding a left boundary region and then expanding the padded image in the right padding region of the basic reference image, and a motion vector of a current data unit included in the current panoramic image. When the subpixel included in the reference data unit indicated by the input motion vector belongs to the reference image, the pixel value of the subpixel is acquired as it is, and the subpixel is located outside the reference image. , Between one boundary of the base reference image near the subpixel and the subpixel The value of the sub-pixel is obtained by adding or subtracting the distance on the x-axis from the x-coordinate of another boundary of the basic reference image to the x-coordinate of the sub-pixel to obtain all pixel values of the reference data unit in the reference image. And obtaining a pixel value, and including a motion compensator for reproducing the current data unit using the pixel values of the reference data unit.

According to another aspect of the present invention, in a motion compensation apparatus for a panoramic image including 360-degree omnidirectional image information, the left padding region of the basic reference image for motion compensation of the current panoramic image is padded using a left boundary pixel value. The padding region of the basic reference image is padded using a right boundary pixel value, and thereafter, a memory for storing a reference image generated by expanding the padded image, and movement of a current data unit included in the current panoramic image. When the subpixel included in the reference data unit indicated by the motion vector of the current data unit belongs to the basic reference image, the pixel value of the subpixel is obtained, and the subpixel is the basic pixel. The base reference zero, which is close to the subpixel, if it falls outside the reference image; A pixel value of the subpixel is obtained by setting a value of the subpixel by adding or subtracting the distance on the x-axis between one boundary of the image and the subpixel to the x coordinate value of the subpixel. And a motion compensator for reproducing the current data unit using the pixel values of the reference data unit after obtaining all pixel values of the reference data unit from the reference image.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

5 is a block diagram of a panoramic image motion vector encoder according to an embodiment of the present invention. Referring to FIG. 5, a panoramic image motion vector encoder according to an embodiment of the present invention may include a transform unit 110, a quantizing unit 115, and an inverse quantizing unit 120. ), An inverse transforming unit 125, an adding unit 130, a clipping unit 140, a frame memory 150, a panorama image motion estimation unit 160, a panorama image motion compensation unit 170, a subtractor 180, and a variable length coder 190 (VLC).

The converting unit 110 receives the panorama image, converts the input panorama image according to a conversion method of a predetermined method, and outputs conversion coefficient values. The input panoramic image is a panoramic image including a 360 degree omnidirectional image generated by cutting the cylindrical image shown in FIG. 4A according to the baseline X. For example, the predetermined transform method performed by the transform unit 110 may include a DCT (Discrete Cosine Transform) in units of 8 × 8 blocks.

The quantization unit 115 quantizes transform coefficient values received from the transform unit 110. The inverse quantization unit 120 and the inverse transform unit 125 reproduce the input panorama image after inverse quantization and inverse transformation, respectively. The reproduced panoramic image is normalized by the clipping unit 140 and then stored in the frame memory 150. The panoramic image stored in the frame memory 150 is used as a reference image for motion estimation and compensation of the newly input panoramic image.

The panoramic image motion estimator 160 performs motion estimation according to the present invention using the reference panoramic image stored in the frame memory 150. That is, the panoramic image motion estimator 160 receives current panoramic image information and performs motion estimation of the current panoramic image using the reference panoramic image stored in the frame memory 150 to obtain a motion vector of the current panoramic image. After generation, it is output to the VLC 190. Motion estimation and compensation are performed in units of blocks of a predetermined size. The unit block of motion estimation and compensation is called a data unit. In this embodiment, the data unit is assumed to be a 16 × 16 macroblock.

The panoramic image motion compensator 170 performs motion compensation according to the present invention. That is, the motion vector of the current macroblock generated by the panoramic image motion estimator 160 is input to output the reference macroblock corresponding to the current macroblock to the subtractor 180. The subtractor 180 outputs the residual signal between the current macroblock and the reference macroblock to the converter 110. The residual signal is transformed and quantized by the transformer 110 and the quantizer 115, and then variably coded by the variable length encoder 190. Meanwhile, the motion vector of the current macroblock generated by the panoramic image motion estimator 160 is directly input to the variable length encoder 190 and is variable length encoded.

Hereinafter, the operation of the panoramic image motion estimator 160 will be described in more detail with reference to the accompanying drawings.

6 is a flowchart illustrating a panoramic image motion estimation method for integer pixel search according to an embodiment of the present invention. The panoramic image motion estimator 160 predicts a motion vector of the current data unit by using motion vectors of a plurality of previous data units adjacent to the current data unit (310). As shown in FIG. 1, when data unit X is a current data unit, data units A, B, C, and D are previous data units necessary for motion vector prediction of current data unit X. In this embodiment, the data unit is a 16 × 16 macroblock.

The panoramic image motion estimator 160 searches for motion vectors of previous macroblocks A, B, C, and D stored in the internal memory. If all of the motion vectors of the previous macroblocks A, B, C and D are present, these motion vectors are used to predict the motion vector of the current macroblock X according to a predetermined scheme or the prior art.

However, at least one of the motion vectors of the previous macroblocks A, B, C, and D may not exist. FIG. 7A illustrates a case in which the motion vector of the previous macroblocks A and D cannot be used because the previous macroblocks A and D do not exist, and FIG. 7B illustrates the motion vector of the previous macroblock C because the previous macroblock C does not exist. Indicates no case.

As described above, the spatial correlation between the right boundary portion and the left boundary portion of the panoramic image including the 360 degree omnidirectional image is very high. That is, the distance between the right boundary and the left boundary of the panoramic image is 0 in real space. Accordingly, in the present invention, if there is a previous macroblock that does not exist among the previous macroblocks A, C, and D required for the motion vector prediction of the current macroblock X, the motion vector of the previous macroblock required by using the above-described panorama image is determined. Decide That is, in the example illustrated in FIG. 7A, the previous macroblock D ′ existing at the right boundary on the same Y axis as the previous macroblock D may be regarded as the same as the previous macroblock D. FIG. Therefore, the motion vector of the previous macroblock D 'is regarded as the motion vector of the previous macroblock D and used for the motion vector prediction of the current macroblock X. However, since the previous macroblock existing at the right boundary on the same Y-axis as the previous macroblock A is motion estimated after the motion estimation of the current macroblock X, there is no motion vector to use. Therefore, the motion vector of the previous macroblock A to be used for motion vector prediction of the current macroblock X is set to zero.

In the example shown in FIG. 7B, the previous macroblock C ′ present at the left boundary portion on the same Y axis as the previous macroblock C may be regarded as the same as the previous macroblock C. Therefore, the motion vector of the previous macroblock C 'is regarded as the motion vector of the previous macroblock C and used for the motion vector prediction of the current macroblock X.

The panoramic image motion estimator 160 predicts a motion vector of the current macroblock X according to step 310 and then determines whether a reference macroblock indicated by the predicted motion vector exists in the reference image (315). The reference image is stored in the frame memory 150.

If all the pixels of the reference macroblock indicated by the motion vector of the current macroblock X exist in the reference image, after all pixel values of the reference macroblocks are obtained from the frame memory 150 (330), the current macroblock X and the reference macroblock The similarity between the two is evaluated using a predetermined evaluation function (335).

However, if some or all pixels of the reference macroblock indicated by the motion vector of the current macroblock X exist outside the boundary of one of the left boundary or the right boundary of the reference image, from the other boundary of the reference image to a predetermined range The image of the padding is padded outside the one boundary (320).

FIG. 8A illustrates a case in which a reference macroblock exists over a boundary of a reference picture, and FIG. 8B illustrates a case in which a reference macroblock exists in an outer region of the reference picture. According to the prior art illustrated in FIG. 3, a pixel value positioned at a left boundary of a reference image is padded to an area outside the left boundary of the reference image, and a pixel value positioned at a right boundary of the reference image is outside the right boundary of the reference image. After padding in the region, motion prediction and compensation were performed.

However, in the present invention, the spatial correlation between the right boundary portion and the left boundary portion of the panoramic image including the 360 degree omnidirectional image is padded to the outer region of the reference image as shown in FIG. 9 using a very high property. . Referring to FIG. 9, the outer region 480 of the left boundary of the reference image 400 is padded with pixel values located in the right boundary region 470 of the reference image 400. The outer region 460 of the right boundary of the reference image 400 is padded with pixel values located in the left boundary region 450 of the reference image 400.

The panoramic image motion estimator 160 pads the reference image according to operation 320 and then imports all pixel values of the reference macroblock from the frame memory 150 in the padded reference image (325). Thereafter, the similarity between the current macroblock X and the reference macroblock is evaluated using a predetermined evaluation function (335). As the predetermined evaluation function, a sum of absolute difference (SAD) function, a sum of absolute transformed difference (SATD) function, or a sum of squared difference (SSD) function is generally used.

On the other hand, without performing the above-mentioned padding process, it is assumed that the reference image is a cylindrical image by connecting the left boundary and the right boundary of the reference image, thereby obtaining all pixel values of the reference data unit in the cylindrical image. It may be. In detail, the reference image is a two-dimensional planar image as illustrated in FIG. 4B. When the left boundary and the right boundary of the 2D plane image are connected, a cylindrical image as shown in FIG. 4A is obtained. If it is assumed that the reference image is a cylindrical image, all pixel values of the reference data unit may be obtained from the cylindrical image.

The panoramic image motion estimator 160 changes the position of the reference macroblock within a predetermined search range and evaluates the similarity between the changed reference macroblock and the current macroblock X (340 and 345). When the similarity evaluation between the plurality of reference macroblocks and the current macroblock X is completed within the predetermined search range, the panoramic image motion estimator 160 selects the reference macroblock most similar to the current macroblock X among the plurality of reference macroblocks. Determine and generate a motion vector representing the reference macroblock (350).

10 is a diagram illustrating a motion vector of the current macroblock X. Referring to FIG. 10, an unpadded reference picture 500 corresponding to the reference macroblock 530 is a reference macroblock 530 on a reference picture in which a reference macroblock most similar to the current macroblock X 510 is padded. The macroblock on) is a macroblock 540. When the reference macroblock most similar to the current macroblock X 510 is referred to as the reference macroblock 530, reference numeral 550 denotes a motion vector of the current macroblock X 510. When the reference macroblock most similar to the current macroblock X 510 is referred to as the reference macroblock 540, reference numeral 560 denotes a motion vector of the current macroblock X 510. That is, the motion vector of the current macroblock X 510 can be represented by the above-described two kinds. However, since the motion vector indicating the macroblock outside the predetermined search range for motion detection cannot be transmitted to the decoder side, the motion vector 550 representing the reference macroblock 530 is determined as the motion vector of the current macroblock X 510. do.

11 is a flowchart of a panoramic image motion estimation method for sub-pixel search according to an embodiment of the present invention.

The panoramic image motion estimator 160 predicts a motion vector of the current data unit by using motion vectors of a plurality of previous data units adjacent to the current data unit (1101). This motion vector prediction is the same as that of the panoramic image motion estimation method for the refinery search.

The panoramic image motion estimator 160 adds a padding area to the basic input image, which is a basic reference image used for motion estimation of the current panoramic image, using a pixel value of the boundary part (1102). That is, the left boundary pixel value is padded in the left padding region of the basic input image, and the right boundary pixel value is padded in the right padding region of the basic input image.

In operation 1103, the panoramic image motion estimator 160 expands the basic input image to which the padding area is added, as a reference image. For example, it would be doubled in the case of 1/2 pixel and quadrupled in the case of 1/4 pixel.

For example, when the size of the current panorama image is 352 * 288 and the half-pixel search is performed, the size of the current panorama image is doubled so that (352 * 2) * (288 * 2) as shown in FIG. do. Referring to FIG. 13A, in the case of a reference picture, if a padding area having a padding size of 4 is added to the base input picture 352 * 288 for reference, the size of the padded base input picture is (4 + 352 + 4) * (4. + 288 + 4), and the padded base input image is doubled to a size of (4 + 352 + 4) * 2 * (4 + 288 + 4) * 2.

After determining the motion vector of the current data unit, the panoramic image motion estimator 160 determines whether the pixel X of the reference data unit indicated by the predicted motion vector belongs to the basic input image area in the reference image (1104).

If the pixel X of the reference data unit belongs to the basic input image area in the reference image, the pixel value of the pixel X is acquired as it is (1105).

When the pixel X of the reference data unit belongs to the outside of the basic input image area, that is, belongs to a padding area or outside the reference image, the x reference coordinate of the pixel X is close to the subpixel. A pixel value corresponding thereto is obtained by adding or subtracting the distance on the x-axis between one boundary of the sub-pixel and the sub-pixel from the x-coordinate of the other boundary of the basic reference image to the x-coordinate of the sub-pixel, thereby obtaining a pixel value corresponding thereto (1106). ). That is, when the pixel X belongs to the left padding region of the reference image or the left outer region of the reference image, the x coordinate of the pixel X is located on the x-axis between one boundary of the base reference image close to the subpixel and the subpixel. A value obtained by subtracting a distance from an x coordinate of another boundary of the basic reference image is set to an x coordinate of the subpixel. When the pixel X belongs to the right padding region of the reference image or the right outer region of the reference image, the x coordinate of the pixel X is located on the x-axis between one boundary of the base reference image near the subpixel and the subpixel. The distance of is added to the x-coordinate of the subpixel by adding the distance from the x-coordinate of another boundary of the basic reference image.

For example, referring to FIG. 13B, the coordinates of the pixel A in the reference data unit are (30,50) and belong to the area of the basic input image of the reference image, so that the pixel value of the pixel A may be obtained as it is.

The coordinate of the pixel B in the reference data unit is (4,50), which belongs to the padding area of the reference image. The x coordinate of one boundary of the base input image close to the pixel B is 7, and the distance between the pixel B and one boundary of the base input image is 3. Since the x coordinate of the other boundary of the basic input image region is 711 and 711-3 = 708, the x coordinate of the pixel B is set to 708. That is, the pixel value of the pixel B is obtained at the coordinates 708 and 50 of the pixel B '.

The coordinate of the pixel C in the reference data unit is (-5,100), which is located outside the reference image. The x coordinate of one boundary of the base input image close to the pixel C is 7 and the distance between the pixel C and one boundary of the base input image is 12. Since the x coordinate of another boundary of the basic input image region is 711 and 711-12 = 699, the x coordinate of the pixel C is set to 699. That is, the pixel value of the pixel C is obtained at the coordinates 699 and 100 of the pixel C '.

As described above, in the present embodiment, when the pixel value existing in the padding area is obtained, the pixel value existing in the padding area is meaningless, and thus the pixel value of the opposite boundary area should be used. That is, when the padding area is performed in a conventional manner, the pixel value of the opposite boundary area should be used for both the pixel present in the padding area and the pixel outside the reference image.

Next, the panoramic image motion estimator 160 determines whether all pixel values of the reference data unit are acquired (1107), and when all pixel values are acquired, the degree of similarity between the current data unit and the reference data unit using a predetermined evaluation function. If not all pixel values are obtained, the process proceeds to step 1104 to obtain the next pixel value. As the predetermined evaluation function, a sum of absolute difference (SAD) function, a sum of absolute transformed difference (SATD) function, or a sum of squared difference (SSD) function is generally used.

Next, the panoramic image motion estimator 160 changes the position of the reference data unit within the predetermined search range and evaluates the similarity between the changed reference data unit and the current data unit (1109 and 1110). When the similarity evaluation between the plurality of reference data units and the current data unit is completed within the predetermined search range, the panoramic image motion estimator 160 determines a reference data unit that is most similar to the current data unit among the plurality of reference data units, A motion vector representing a reference data unit is generated (1111).

14 is a flowchart of a panoramic image motion estimation method for subpixel search according to another embodiment of the present invention.

The panoramic image motion estimator 160 predicts a motion vector of the current data unit by using motion vectors of a plurality of previous data units adjacent to the current data unit (1401). This motion vector prediction is the same as that of the panoramic image motion estimation method for the refinery search.

The panoramic image motion estimation unit 160 pads the left padding region of the basic input image using pixel values of the right boundary region and pads the right padding region of the basic input image using pixel values of the left boundary region (1402). ).

Next, in operation 1402, the panoramic image motion estimator 160 expands the basic input image to which the padding area is added and uses it as a reference image. For example, it would be doubled in the case of 1/2 pixel and quadrupled in the case of 1/4 pixel.

For example, when the size of the current panorama image is 352 * 288 and the half-pixel search is performed, the size of the current panorama image is doubled so that (352 * 2) * (288 * 2) as shown in FIG. do. Referring to FIG. 15A, when a padding area having a padding size of 4 is added to a base input image 352 * 288 for reference, the size of the padded base input image is (4 + 352 + 4) * (4. + 288 + 4), and the padded base input image is doubled to a size of (4 + 352 + 4) * 2 * (4 + 288 + 4) * 2. Of course, referring to the padding area, the pixel value of the right boundary area is padded in the left padding area of the reference image, and the pixel value of the left boundary area is padded in the right padding area of the reference image.

The panoramic image motion estimator 160 predicts the motion vector of the current data unit and then determines whether the pixel B of the reference data unit indicated by the predicted motion vector belongs to the reference image (1404).

If the pixel X of the reference data unit belongs to the reference image, the pixel value of the pixel X is acquired as it is (1405).

When the pixel X of the reference data unit is located outside of the reference image, the basic reference is referred to the distance on the x-axis between one boundary of the base reference image in which the x coordinate of the pixel X is close to the subpixel and the subpixel. A pixel value is obtained by setting a value added from the x coordinate of another boundary of the image as the x coordinate of the pixel X (1406). That is, when the pixel X belongs to the left outer region of the reference image, the basic reference is the distance on the x-axis between one boundary of the base reference image in which the x coordinate of the pixel X is close to the subpixel and the subpixel. The value subtracted from the x coordinate of another boundary of the image is set to the x coordinate of the subpixel. When the pixel X belongs to the right outer region of the reference image, the distance between the boundary of the base reference image in which the x coordinate of the pixel X is close to the subpixel and the distance on the x-axis between the subpixel is determined. The value added from the x coordinate of the other boundary of is set to the x coordinate of the subpixel.

For example, referring to FIG. 15B, the reference data unit includes a pixel A having coordinates 30 and 50, a pixel B having coordinates 4 and 50, and a pixel D having coordinates (-5 and 100).

Pixels A and B belong to the reference picture, and pixel C does not belong to the reference picture but outside the reference picture. In addition, the pixel A belongs to the basic input image region among the reference image, and the pixel B belongs to the padding region. In the case of the pixel A, since the pixel belongs to the basic input image, the pixel value of the position may be acquired.

The pixel B is a value belonging to an area padded with the pixel value of the right boundary area in consideration of the spatial correlation of the panoramic image when generating the reference image in advance, and thus, the pixel value of the pixel B may be acquired.

The coordinate of the pixel D in the reference data unit is (-5,100), which is located outside the reference image. The x coordinate of one boundary of the base input image close to the pixel D is 7, and the distance between the pixel D and one boundary of the base input image is 12. Since the x coordinate of another boundary of the basic input image region is 711 and 711-12 = 699, the x coordinate of the pixel D is set to 699. That is, the pixel value of the pixel D is obtained at the coordinates 699 and 100 of the pixel D '. As described above, in the present embodiment, when the pixel value existing in the padding area is obtained, the pixel value existing in the padding area is already padded using the pixel value of the opposite boundary area and may be used as it is. It is only necessary to calculate and use the pixel value of the opposite boundary region for the small cows.

Next, the panoramic image motion estimator 160 determines whether all pixel values of the reference data unit are acquired (1407), and when all pixel values are acquired, the degree of similarity between the current data unit and the reference data unit using a predetermined evaluation function. In operation 1408, if all pixel values are not obtained, the process proceeds to step 1404 to obtain the next pixel value. As the predetermined evaluation function, a sum of absolute difference (SAD) function, a sum of absolute transformed difference (SATD) function, or a sum of squared difference (SSD) function is generally used.

Next, the panoramic image motion estimator 160 changes the position of the reference data unit within the predetermined search range and evaluates the similarity between the changed reference data unit and the current data unit (1409 and 1410). When the similarity evaluation between the plurality of reference data units and the current data unit is completed within the predetermined search range, the panoramic image motion estimator 160 determines a reference data unit that is most similar to the current data unit among the plurality of reference data units, A motion vector representing the reference data unit is generated (1411).

Hereinafter, a method and apparatus for compensating for motion of a panoramara image according to an embodiment of the present invention will be described.

16 is a block diagram of a panoramic image motion vector decoder according to an embodiment of the present invention. Referring to FIG. 16, a panoramic image motion vector decoder according to an embodiment of the present invention includes a variable length decoder (VLD) 710, an inverse quantizing unit 720, and an inverse transform unit ( an inverse transforming unit 730, an adding unit 740, a panorama image motion compensation unit 750, a clipping unit 760, and a frame memory 770. Include.

The variable length decoder 710 variable length decodes the input bitstream. The motion vector of the variable length decoder 710 is output to the panoramic image motion compensator 750, and the residual signal between the current macroblock and the reference macroblock is output to the inverse quantizer 720.

The frame memory 770 stores the reference image reproduced through the inverse quantizer 720, the inverse transformer 730, and the clipping unit 760. The reference image stored in the frame memory 770 is used for motion compensation of the newly input panoramic image.

The panoramic image motion compensator 750 performs motion compensation according to the present invention by using the reference panoramic image stored in the frame memory 770. A motion vector of the current macroblock transmitted from the panoramic motion vector encoder as shown in FIG. 5 is input, and a reference macroblock corresponding to the current macroblock is read from the frame memory 770 and output to the adder 740. . In addition, the adder 740 receives a residual signal between the current macroblock and the reference macroblock, which are inversely quantized and inversely transformed by the inverse quantizer 720 and the inverse transformer 730, respectively.

The adder 740 receives the residual signal between the current macroblock and the reference macroblock and the reference macroblock from the panoramic image motion compensator 750 to reproduce the current macroblock. The clipping unit 760 normalizes the reproduced current macroblock output from the adder 740.

Hereinafter, the operation of the panoramic image motion compensator 750 will be described in more detail with reference to the accompanying drawings. 17 is a flowchart illustrating a panoramic image motion compensation method according to an embodiment of the present invention.

The panoramic image motion compensator 750 receives a motion vector of a current data unit to perform motion compensation from the VLD 710 (910). In this embodiment, the data unit is a 16 × 16 macroblock.

The panoramic image motion compensator 750 determines whether a reference macroblock indicated by the motion vector of the current macroblock exists in the reference image (920). The reference image is stored in the frame memory 770.

In the case of a refinement search, if all the pixels of the reference macroblock represented by the motion vector of the current macroblock exist in the reference image, after all pixel values of the reference macroblocks are obtained from the frame memory 770 (950), the current macroblock is extracted. Play (960).

The adder 740 receives the residual signal between the current macroblock and the reference macroblock output from the inverse transformer 730 and the reference macroblock output from the panoramic image motion compensator 750 to reproduce the current macroblock. do.

However, if some or all pixels of the reference macroblock indicated by the motion vector of the current macroblock exist outside the boundary of either the left boundary or the right boundary of the reference image, as shown in FIGS. 8A or 8B, An image from one boundary to a predetermined range is padded outside the one boundary (930). In the present invention, the spatial correlation between the right boundary portion and the left boundary portion of the panoramic image including the 360 degree omnidirectional image is padded to the outer region of the reference image as shown in FIG. 9 using a very high property.

The panoramic image motion compensator 750 pads the reference image according to operation 930 and then imports all pixel values of the reference macroblock from the frame memory 770 in the padded reference image (940).

On the other hand, without performing the above-mentioned padding process, it is assumed that the reference image is a cylindrical image by connecting the left boundary and the right boundary of the reference image, thereby obtaining all pixel values of the reference data unit in the cylindrical image. It may be. In detail, the reference image is a two-dimensional planar image as illustrated in FIG. 4B. When the left boundary and the right boundary of the 2D plane image are connected, a cylindrical image as shown in FIG. 4A is obtained. If it is assumed that the reference image is a cylindrical image, all pixel values of the reference data unit may be obtained from the cylindrical image.

Finally, the adder 740 receives the residual signal between the current macroblock and the reference macroblock and the reference macroblock output from the panoramic image motion compensator 750 to reproduce the current macroblock (960).

In the case of half-pixel search, the left padding region of the basic reference image used for motion compensation of the current panoramic image is padded using a right boundary region, and the right padding region of the basic reference image is scanned according to an embodiment of the present invention. After padding using the left boundary area, the reference image generated by expanding the padded image is stored in the frame memory, and the sub-pixel which is a motion search target included in the reference data unit represented by the input motion vector belongs to the reference image. In this case, the pixel value of the subpixel is used as it is for motion compensation, and the distance on the x-axis between one boundary of the base reference image and the subpixel close to the subpixel is x of the other boundary of the base reference image. A pixel value is obtained by setting the value added or subtracted from the coordinates to the x coordinate value of the subpixel.

In the case of half-pixel search, the left padding area of the base reference image used for motion compensation of the current panoramic image is padded using a left boundary pixel value, and the right side padding area of the base reference image according to another embodiment of the present invention. After the padding is performed using the right boundary pixel value, the padded image is stored in the frame memory, and the sub-pixel which is a motion search target is included in the reference data unit indicated by the motion vector of the current data unit. In case of belonging to the image area, the pixel value of the pixel is acquired as it is, and when the subpixel belongs to the padding area of the reference image or the outside of the reference image, one boundary of the base reference image near the subpixel and the The distance on the x-axis between the subpixels is determined from the x-coordinate of the other boundary of the base reference image. Setting a value or by subtracting the x-coordinate value of the sub-pixels and, to obtain a pixel value.

In the above description, the pixel value of the left boundary region of the reference image is mainly used as the pixel value of the right region, and the pixel value of the right boundary region of the reference image is used as the pixel value of the left region. Although only the horizontal direction of the image has been mainly described, this has been described for the horizontal direction because of the high spatial correlation between the left boundary and the right boundary due to the characteristics of the panoramic image. However, it will be understood by those skilled in the art that the method according to the present invention can be applied in the vertical direction if the spatial correlation between the upper boundary region and the lower boundary region of the image is high as well as the horizontal direction.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet). It includes being. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

So far I looked at the center of the preferred embodiment for the present invention. Those skilled in the art will appreciate that the present invention can be implemented in a modified form without departing from the essential features of the present invention. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope will be construed as being included in the present invention.

As described above, according to the present invention, the spatial correlation between the right boundary portion and the left boundary portion of the panoramic image including the 360 degree omnidirectional image is very high, thereby performing motion prediction and compensation of the panoramic image. This improves the image quality by improving the efficiency and accuracy of motion estimation and compensation. In particular, according to the present invention, the image quality of the right and left boundary parts can be improved by improving the efficiency and accuracy of motion prediction and compensation of the right and left boundary parts of the panoramic image including the 360 degree omnidirectional image. .

Claims (15)

In the motion estimation method of a panoramic image including 360-degree omnidirectional image information, The left padding region of the base reference image for motion estimation of the current panoramic image is padded using the right boundary region, the right padding region of the basic reference image is padded using the left boundary region, and then the padded image is expanded. Setting as a reference image, Predicting a motion vector of the current data unit by using motion vectors of a plurality of previous data units adjacent to a current data unit included in the current panoramic image; When the subpixel included in the reference data unit indicated by the predicted motion vector belongs to the reference image, the pixel value of the subpixel is acquired as it is, and when the subpixel is located outside the reference image, the subpixel The distance on the x-axis between one boundary of the base reference image and a subpixel adjacent to a pixel is added to or subtracted from the x-coordinate of another boundary of the base reference image by the x-coordinate of the sub-pixel. Obtaining all pixel values of the reference data unit in the reference image by obtaining pixel values of the pixel; Determining a similarity degree between the current data unit and the reference data unit using a predetermined evaluation function. According to claim 1, Predicting the motion vector of the current data unit, If at least one of the plurality of previous data units exists outside the left boundary or the right boundary of the current panorama image, it is assumed that the current panorama image is a cylindrical image by connecting the left boundary and the right boundary of the current panorama image. And determining the plurality of previous data units in the cylindrical image. The method of claim 1, Determining a reference data unit most similar to the current data unit within a predetermined search range; Determining a motion vector representing the determined reference data unit. In the motion estimation method of a panoramic image including 360-degree omnidirectional image information, The left padding area of the base reference image for motion estimation of the current panoramic image is padded using a left boundary pixel value, the right padding area of the base reference image is padded using a right boundary pixel value, and then the padded image is padded. To expand and set it as a reference image, Predicting a motion vector of the current data unit by using motion vectors of a plurality of previous data units adjacent to a current data unit included in the current panoramic image; If the subpixel included in the reference data unit indicated by the predicted motion vector belongs to the base reference image of the reference image, the pixel value of the subpixel is obtained, and the subpixel is outside the base reference image of the reference image. In the case of, the x coordinate of the subpixel is obtained by subtracting or subtracting a distance on the x-axis between one boundary of the base reference image and the subpixel adjacent to the subpixel from the x coordinate of the other boundary of the base reference image. Obtaining all pixel values of the reference data unit in the reference image by acquiring the pixel value of the subpixel by setting to a value; Determining a similarity degree between the current data unit and the reference data unit using a predetermined evaluation function. The method of claim 4, wherein Predicting the motion vector of the current data unit, If at least one of the plurality of previous data units exists outside the left boundary or the right boundary of the current panorama image, it is assumed that the current panorama image is a cylindrical image by connecting the left boundary and the right boundary of the current panorama image. And determining the plurality of previous data units in the cylindrical image. The method of claim 4, wherein Determining a reference data unit most similar to the current data unit within a predetermined search range, Determining a motion vector representing the determined reference data unit. In the motion estimation apparatus of a panoramic image including 360-degree omnidirectional image information, The left padding region of the base reference image for motion estimation of the current panoramic image is padded using the right boundary region, the right padding region of the basic reference image is padded using the left boundary region, and then the padded image is expanded. A memory for storing motion vectors of a plurality of previous data units adjacent to the current data unit included in the generated reference image and the current panorama image; The subpixel is predicted by using the motion vectors of the plurality of previous data units, and when the subpixel included in the reference data unit represented by the predicted motion vector belongs to the reference image, If the pixel value of is obtained as it is and the subpixel is located outside the reference image, the basic reference value is the distance on the x-axis between one boundary of the base reference image near the subpixel and the subpixel. After all the pixel values of the reference data unit in the reference image are obtained by setting a value added or subtracted from the x coordinate of another boundary of the image to the x coordinate of the subpixel to obtain pixel values of the subpixel, Motion estimation for determining similarity between the current data unit and the reference data unit using a function Motion estimation apparatus comprising a unit. The method of claim 7, wherein The motion estimation unit, And determining a reference data unit most similar to the current data unit within a predetermined search range, and determining a motion vector representing the determined reference data unit. In the motion estimation apparatus of a panoramic image including 360-degree omnidirectional image information, The left padding area of the base reference image for motion estimation of the current panoramic image is padded using a left boundary pixel value, the right padding area of the base reference image is padded using a right boundary pixel value, and then the padded image is padded. A memory for storing motion vectors of a plurality of previous data units adjacent to a current data unit included in the extended reference image and the current panoramic image; The motion vector of the current data unit is predicted using the motion vectors of the plurality of previous data units, and a subpixel included in a reference data unit represented by the predicted motion vector is added to the basic reference image of the reference image. A pixel value of the subpixel when the subpixel belongs, and a distance on the x-axis between one boundary of the base reference image near the subpixel and the subpixel when the subpixel belongs to the outside of the base reference image. Set the value added or subtracted from the x coordinate of another boundary of the base reference image to the x coordinate value of the subpixel to obtain the pixel value of the subpixel, thereby obtaining all the pixel values of the reference data unit in the reference image. After obtaining, the similarity of the current data unit and the reference data unit using a predetermined evaluation function Motion Estimation Motion estimation apparatus comprising: a unit for determining. The method of claim 9, The motion estimation unit, If at least one of the plurality of previous data units exists outside the left boundary or the right boundary of the current panorama image, it is assumed that the current panorama image is a cylindrical image by connecting the left boundary and the right boundary of the current panorama image. And determine the plurality of previous data units in the cylindrical image. The method of claim 9, The motion estimation unit, And determining a reference data unit most similar to the current data unit within a predetermined search range, and determining a motion vector representing the determined reference data unit. In the motion compensation method of a panoramic image including 360-degree omnidirectional image information, The left padding region of the basic reference image for motion compensation of the current panoramic image is padded using the right boundary region, the right padding region of the basic reference image is padded using the left boundary region, and then the padded image is expanded. Generating as a reference image, Receiving a motion vector of a current data unit included in a current panorama image; When the subpixel included in the reference data unit indicated by the motion vector of the current data unit belongs to the reference image, the pixel value of the subpixel is acquired as it is, and the subpixel is located outside the reference image. The distance on the x-axis between one boundary of the base reference image and the subpixel adjacent to the subpixel is added or subtracted from the x coordinate of the other boundary of the base reference image to be the x coordinate of the subpixel, Obtaining all pixel values of the reference data unit in the reference image by obtaining pixel values of the subpixel; And reproducing the current data unit using the pixel values of the reference data unit. In the motion compensation method of a panoramic image including 360-degree omnidirectional image information, The left padding area of the base reference image for motion compensation of the current panoramic image is padded using a left boundary pixel value, the right padding area of the base reference image is padded using a right boundary pixel value, and then the padded image is padded. Generating a reference image by expanding; Receiving a motion vector of a current data unit included in a current panorama image; When the subpixel included in the reference data unit indicated by the motion vector of the current data unit belongs to the basic reference image of the reference image, the pixel value of the subpixel is obtained, and the subpixel is a basic reference image of the reference image. When the value falls outside of, the value on the x-axis between one boundary of the base reference image near the subpixel and the subpixel is added to or subtracted from the x coordinate of the other boundary of the base reference image. obtaining all pixel values of the reference data unit in the reference image by setting the x coordinate value to obtain pixel values of the subpixel; And reproducing the current data unit using the pixel values of the reference data unit. In the motion compensation device of a panoramic image including 360-degree omnidirectional image information, The left padding region of the basic reference image for motion compensation of the current panoramic image is padded using the right boundary region, the right padding region of the basic reference image is padded using the left boundary region, and then the padded image is expanded. A memory for storing the generated reference image; If the subpixel included in the reference data unit indicated by the input motion vector belongs to the reference image, the motion vector of the current data unit included in the current panoramic image is received, and the pixel value of the subpixel is acquired as it is. If the subpixel is located outside the reference image, the distance on the x-axis between one boundary of the base reference image near the subpixel and the subpixel is determined by the x coordinate of the other boundary of the base reference image. By setting the value added or subtracted from to the x-coordinate of the sub-pixel to obtain the pixel value of the sub-pixel to obtain all the pixel values of the reference data unit in the reference image, and then use the pixel values of the reference data unit And a motion compensator for reproducing the current data unit. In a motion compensation device of a panoramic image including 360-degree omnidirectional image information, The left padding area of the base reference image for motion compensation of the current panoramic image is padded using a left boundary pixel value, the right padding area of the base reference image is padded using a right boundary pixel value, and then the padded image is padded. A memory for storing the reference image generated by extension; When the subpixel included in the reference data unit indicated by the motion vector of the current data unit belongs to the basic reference image of the reference image, the motion vector of the current data unit included in the current panoramic image is input. When a pixel value is acquired and the subpixel belongs to the outside of the base reference image, the distance on the x-axis between one boundary of the base reference image near the subpixel and the subpixel is determined by the value of the base reference image. The reference data is obtained after all the pixel values of the reference data unit are obtained from the reference image by setting a value added or subtracted from the x coordinate of another boundary to the x coordinate value of the subpixel to obtain the pixel value of the subpixel. And a motion compensator for reproducing the current data unit using pixel values of the unit. A motion compensation device.

Images