WO2006016783A1 - Method and device for motion estimation and compensation for panorama image - Google Patents
Method and device for motion estimation and compensation for panorama image Download PDFInfo
- Publication number
- WO2006016783A1 WO2006016783A1 PCT/KR2005/002639 KR2005002639W WO2006016783A1 WO 2006016783 A1 WO2006016783 A1 WO 2006016783A1 KR 2005002639 W KR2005002639 W KR 2005002639W WO 2006016783 A1 WO2006016783 A1 WO 2006016783A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- data unit
- current
- motion vector
- reference image
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 239000013598 vector Substances 0.000 claims description 170
- 230000006870 function Effects 0.000 claims description 22
- 238000011156 evaluation Methods 0.000 claims description 17
- 238000010586 diagram Methods 0.000 description 16
- 230000001131 transforming effect Effects 0.000 description 12
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
Definitions
- the present general inventive concept relates to motion estimation and com ⁇ pensation for a panorama image, and more particularly, to a method and apparatus to estimatea motion of a panorama image containing 360 ° omni-directional image in ⁇ formation, and a method and apparatus to compensate for the motion of the panorama image.
- An omni-directional video camera system is capable of acquiring a 360 ° omni ⁇ directional view from a single viewpoint.
- the omni-directional video camera system includes a camera to which a special mirror, such as a hyperboloid mirror, or a special lens, such as a fish-eye lens, is installed, or a plurality of cameras.
- Three-dimensional (3D) realistic broadcasting may be applied to omni-directional video coding.
- a viewer's terminal receives all image information regarding scenes viewed from diverse viewpoints, such as the viewpoints of a pitcher, a catcher, a hitter, and an audience on a first base side in a baseball game, and the viewer can select a desired viewpoint to view one of the scenes from the desired viewpoint.
- An image captured by the omni-directional camera system has characteristics cor ⁇ responding to a 3D cylindrical environment and thus is transformed into a two- dimensional (2D) plane image.
- the 2D plane image is a panorama image with a 360 ° omni-directional view, and omni-directional video coding is performed on the 2D panorama image.
- a motion vector is computed by detecting a data unit, which is most similar to a data unit in a current frame, from a previous frame using a predetermined evaluation function, the motion vector represents a position difference between the data units, and, in general, 16 ' 16 macro blocks are used as the data blocks but the sizes of macro blocks are not limited, and for instance, the data units may be 16x 8, 8x16, or 8x8 macro blocks.
- a motion vector of a current macro block of a current frame is predicted using a plurality of previous macro blocks of a previous frame adjacent to a position corresponding to the current macro block of the current frame.
- FlG. 1 illustrates a plurality of previous macro blocks A, B, C, and Dof the previous frame used to estimate the motion vector of a current macro block X of the current frame.
- the previous macro blocks A through D are encoded before coding of the current macro block X.
- FlG. 2B illustrates a case where the previous macro blocks A and D are not present.
- the motion vectors of the previous macro blocks A and D are set to 0, and the motion vector of the current macro block X is set to a median value of the motion vectors of the previous macro blocks A through D.
- FlG. 2C illustrates a case where the previous macro block C is not present.
- the motion vector of the previous macro block C is set to 0, and the motion vector of the current macro block X is set to the median value of the motion vectors of the previous macro blocks A through D.
- the similarity between each reference macro block in a reference frame indicated by the predicted motion vector and the current macro block X is computed using a predetermined evaluation function.
- a reference macro block that is most similar to the current macro block X is detected from the reference frame within a predetermined search range.
- a sum of absolute differences (SAD) function, a sum of absolute transformed differences (SATD) function, or a sum of squared differences (SSD) function is used as the predetermined evaluation function.
- FlG. 4A illustrates a cylindrical image with a 360 ° omni-directional view.
- FlG. 4B illustrates a panorama image with a 360 ° omni-directional view, taken along with a line X of the cylindrical image of FlG. 4 A.
- a left side A and a right side B of a human-shaped object shown in FlG. 4A are positioned at the right and left borders of the panorama image, respectively. That is, a spatial relation between the right and left borders of the panorama image with the 360 ° omni-directional view is very high.
- the present general inventive concept provides a method and apparatus to ef ⁇ fectively and precisely estimatea motion of a panorama image containing omni ⁇ directional image information.
- the present general inventive concept also provides a method and apparatus to ef ⁇ fectively and precisely compensate for a motion of a panorama image containing omni ⁇ directional image information.
- motion estimation and com ⁇ pensation are performed on a panorama image with a 360 ° omni-directional view based on that the spatial relation between right and left borders of the panorama image is very high, thereby increasing the efficiency and precision of motion estimation and compensation. Accordingly, it is possible to improve image quality, in particular, the image quality at the right and left borders of the panorama image.
- FlG. 1 is a diagram illustrating a plurality of previous macro blocks available for conventional estimation of a motion vector for a current macro block
- FIGS. 2 A through 2C are diagrams illustrating cases where previous macro blocks to be used for estimation of a motion vector of a current macro block are not present;
- FlG. 3 is a diagram illustrating a conventional method of padding a reference image
- FlG. 4A is a diagram illustrating a cylindrical image with a 360 ° omni-directional view
- FlG. 4B is a diagram illustrating a two-dimensional (2D) image corresponding to the cylindrical image of FlG. 4A;
- FIG. 5 is a block diagram illustrating an encoding unit that encodes a motion vector of a panorama image according to an embodiment of the presentgeneral inventive concept
- FIGS. 6A and 6B are flowcharts illustrating a method of estimating the motion of a panorama image according to an embodiment of the presentgeneral inventive concept
- FlG. 7A is a diagram illustrating selection of previous macro blocks to be used for estimation of a motion vector of a current macro block according to an embodiment of the presentgeneral inventive concept
- FlG. 7B is a diagram illustrating selection of previous macro blocks to be used for estimation of a motion vector of a current macro block according to another embodiment of the presentgeneral inventive concept
- FlG. 8A is a diagram illustrating a case where a reference macro block partially overlaps with a reference image
- FlG. 8B is a diagram illustrating a case where a reference macro block is positioned outside a reference image
- FlG. 9 is a diagram illustrating a method of padding a reference image according to an embodiment of the presentgeneral inventive concept
- FlG. 10 is a diagram illustrating a motion vector of a current macro block
- FlG. 11 is a block diagram illustrating a decoding unit that decodes a motion vector of a panorama image according to an embodiment of the presentgeneralinven- tiveconcept.
- FlG. 12 is a flowchart illustrating a method of compensating for a motion of a panorama image according to an embodiment of the presentgeneral inventive concept.
- the foregoing and/or other aspects of the present general inventive concept may be achieved by providing a method of estimating a motion of a current panorama image containing 360 ° omni-directional view information, the method comprising estimating a motion vector of a current data unit of the panorama image using motion vectors of a plurality of previous reference data units of a reference image adjacent to the current data unit, when one or more pixels of one of the reference data units indicated by the estimated motion vector are present outside one of left and right borders of a reference image, padding an image in a predetermined range from the other border of the reference image outside the one of the left and right borders; obtaining values of all pixels of the reference data unit from the padded reference image; and determining a similarity between the current data unit and the reference data unit using a pre ⁇ determined evaluation function.
- the foregoing and/or other aspects of the present general inventive concept may also be achieved by providing a method of estimating a motion of a current panorama image containing 360 ° omni-directional view information, the method comprising estimating a motion vector of a current data unit of the panorama image using motion vectors of a plurality of previous reference data units adjacent to the current data unit, when one or more pixels of one of reference data unit indicated by the estimated motion vector are present outside one of left and right borders of the reference image, obtaining values of all pixels of the one of the reference data units of the reference image from a cylindrical image which is obtained by connecting the left and right borders of the reference image when the reference image is the cylindrical image, and determining a similarity between the current data unit and the reference data unit using a predetermined evaluation function.
- an apparatus to compensate for a motion of a panorama image containing 360 ° omni-directional view information comprising a memory to store a reference image to be used for motion estimation of the panorama image, and motion vectors of a plurality of previous reference data units adjacent to a current data unit of the panorama image, and a motion estimating unit to estimate a motion vector of the current data unit using the motion vectors of the plurality of the previous data units, when one or more pixels of one of the reference data units indicated by the estimated motion vector are present outside one of left and right borders of the reference image, padding an image in a predetermined range from the other border of the reference image outside the one of the left and right borders,to obtain values of all pixels of the reference data unit from the padded reference image, and to determine a similarity between the current data unit and the reference data unit using a predetermined evaluation function.
- an apparatus to estimate a motion of a panorama image containing 360 ° omni-directional view information comprising a memory to store a reference image to be used for motion estimation of the panorama image, and motion vectors of a plurality of previous reference data units of the reference image adjacent to a current data unit of the panorama image, and a motion estimating unit to estimate a motion vector of the current data unit using the motion vectors of the plurality of the previous data units, when one or more pixels of one of the reference data units indicated by the estimated motion vector are present outside one of left and right borders of the reference image, to obtain values of all pixels of the reference data unit from a cylindrical image obtained by connecting the left and right borders of the reference image on an assumption that the reference image is the cylindrical image, and to determine a similarity between the current data unit and the reference data unit using a predetermined evaluation function.
- the foregoing and/or other aspects of the present general inventive concept may also be achieved by providing a method of compensating for a motion of a panorama image containing 360 ° omni-directional view information, the method comprising receiving a motion vector of a current data unit of the panorama image, when one or more pixels of one of reference data units of a reference image indicated by the motion vector of the current data unit are present outside one of left and right borders of the reference image, padding the reference image in a predetermined range from the other border of the reference image outside the one of the left and right borders of the reference image, obtaining values of all pixels of the reference data unit from the padded reference image, and reproducing the current data unit using the values of the pixels of the reference data unit.
- the foregoing and/or other aspects of the present general inventive concept may also be achieved by providing a method of compensating for a motion of a panorama image containing 360 ° omni-directional view information, the method comprising receiving a motion vector of a current data unit of the panorama image, when one or more pixels of one of reference data units of a reference image indicated by the motion vector of the current data unit are present outside one of left and right borders of the reference image, obtaining values of all pixels of the reference data unit from a cylindrical image which is obtained by connecting the left and right borders of the reference image when the reference image is the cylindrical image; and reproducing the current data unit using the values of the pixels of the reference data unit.
- an apparatus to compensate for a motion of a panorama image containing 360 ° omni-directional view information comprising a memory to store a reference image to be used for motion estimation of the panorama image, and a motion compensating unit to receive a motion vector of a current data unit of the panorama image, when one or more pixels of one of reference data units of a reference image indicated by the motion vector of the current data unit are present outside one of left and right borders of the reference image, to padthe reference image in a predetermined range from the other border of the reference image outside the one of the left and right borders of the reference image,to obtain values of all pixels of the reference data unit from the padded reference image, and to reproduce the current data unit using the values of the pixels of the reference data unit.
- an apparatus to compensate for the motion of a panorama image containing 360 ° omni-directional view information comprising a memory to store a reference image to be used for motion estimation of the panorama image; and a motion compensating unit to receive a motion vector of a current data unit of the panorama image, when one or more pixels of reference data units of a reference image indicated by the motion vector of the current data unit are present outside one of left and right borders of the reference image, to obtain values of all pixels of the reference data unit from a cylindrical image which is obtained by connecting the left and rights borders of the reference image when the reference image is the cylindrical image; and reproducing the current data unit using the values of the pixels of the reference data unit.
- a n apparatus to estimate a motionvectorof a panorama image containing 360 ° omni-directional view information comprising a memory to store a reference image having first and second borders and first and second reference data units disposed adjacent to the first border and the second border, re ⁇ spectively, within the reference image,anda motion estimating unit to receive a current data unit of a current image and the reference data units of the reference image from the memory, and to estimate a motion vector of the current data unit using one of the first and second reference data units of the reference image which is not included in a search area when the other one of the first and second reference data units is included in the search area.
- a n apparatus to generate a panorama image containing 360 ° omni-directional view information
- the apparatus comprising a decoding unit to decode a bitstream having data corresponding to a current image and a reference image, and togenerate a motion vector of a current data unit of the current image to correspond to a search area of the reference image which includes a first reference data unit disposed on a first border of the reference image, a panorama image motion com ⁇ pensating unit to generate a reference macro blockofthe first reference data unit of the reference image using a second reference data unit disposed on a second border of the reference image which is not included in the search area according to the motion vector,and an output unit to generate the current image according to the reference macro block and data corresponding to the decoded bitstream.
- the foregoing and/or other aspects of the present general inventive concept may also be achieved by providing a n apparatus having an encoder and a decoder to estimate a motion vector of a panorama image containing 360 ° omni-directional view information.
- the encoder comprises a memory to store a reference image having first and second borders and first and second reference data units disposed adjacent to the first border and the second border, respectively, within the reference image, a motion estimating unit to receive a current data unit of a current image and the reference data units of the reference image from the memory, and to estimate a motion vector of the current data unit using one ofthe first and second reference data units of the reference image which is not included in a search area when the other one of the first and second reference data units is included in the search area, a panorama image motion com- pensating unit to generate a reference macro block according to the motion vector and the reference image,anda coding unit togenerate an bitstream according to the current image and the reference macro block.
- the decoder comprises a decoding unit to decode the bitstream having data corresponding tothecurrent image and the reference image, and to generate the motion vector ofthe current data unit of the current image to correspond to the search area of the reference image which includes the first reference data unit disposed on the first border of the reference image, a second panorama image motion compensating unit to generate the reference macro block of the first reference data unit of the reference image using the second reference data unit disposed on the second border of the reference image which is not included in the search area according to the motion vector,and an output unit to generate the current image according to the reference macro block and data corresponding to the decoded bitstream.
- the foregoing and/or other aspects of the present general inventive concept may also be achieved by providing a method of estimating a motion vector of a panorama image containing 360 ° omni-directional view information, the method comprising storing a reference image having first and second borders and first and second reference data units disposed adjacent to the first border and the second border, respectively, within the reference image,and receiving a current data unit of a current image and the reference data units of the reference image from the memory, and estimating a motion vector of the current data unit using one of the first and second reference data units of the reference image which is not included in a search area when the other one of the first and second reference data units is included in the search area.
- the foregoing and/or other aspects of the present general inventive concept may also be achieved by providing a method of generating a panorama image containing 360 ° omni-directional view information, the method comprising decoding a bitstream having data corresponding to a current image and a reference image, generatinga motion vector of a current data unit of the current image to correspond to a search area of the reference image which includes a first reference data unit disposed on a first border of the reference image, generatinga reference macro block of the first reference data unit of the reference image using a second reference data unit disposed on a second border of the reference image which is not included in the search area according to the motion vector,and generating the current image according to the reference macro block and data corresponding to the decoded bitstream.
- a method of estimating a motion vector of a panorama image containing 360 ° omni-directional view information comprising storing a reference image having first and second borders and first and second reference data units disposed adjacent to the first border and the second border, respectively, within the reference image,receiving a current data unit of a current image and the reference data units of the reference image from the memory, estimating a motion vector of the current data unit using one of the first and second reference data units of the reference image which is not included in a search area when the other one of the first and second reference data units is included in the search area,generating a reference macro block according to the motion vector and the reference image,generating a bitstream according to the current image and the reference macro block,decoding the bitstream having data corresponding to the current image and the reference image, generating the motion vector of the current data unit of the current image to correspond to the search area of the reference image which includes the first reference
- FlG. 5 is a block diagram illustrating an encoding unit that encodes a motion vector of a panorama image according to an embodiment of the presentgeneral inventive concept.
- the encoding unit includes a transforming unit 110, a quantizing unit 115, an inverse quantizing unit 120, an inverse transforming unit 125, an adding unit 130, a clipping unit 140, a frame memory 150, a panorama image motion estimation unit 160, a panorama image motion compensation unit 170, a subtraction unit 180, and a variable-length coder (VLC) 190.
- VLC variable-length coder
- the transforming unit 110 receives an input panorama image and transforms the received panorama image through predetermined transformation to output transform coefficients.
- the input panorama image is a panorama image with a 360 ° omni ⁇ directional view as shown in FlG. 4B, taken along a line X of a cylindrical image shown in FlG. 4A.
- the predetermined transform performed by the transforming unit 110 may be a discrete cosine transform (DCT) in units of 8 ' 8 blocks.
- DCT discrete cosine transform
- the quantizing unit 115 quantizes the transform coefficients received from the transforming unit 110. After the quantized transform coefficients are inversely quantized by the inverse quantizing unit 120 and inversely transformed by the inverse transforming unit 125, the input panorama image is reproduced. The reproduced panorama image is normalized by the clipping unit 140 and stored in the frame memory 150. The panorama image stored in the frame memory 150 is used as a reference panorama image in motion estimation and compensation of a newly input panorama image.
- the adding unit 130 may have a predetermined value, receivethe reproduced panorama image,modify the reproduced panorama imageusingthepredeter- minedvalue, andoutput one of the reproduced panorama image and the modified panorama image to the clipping unit 140 and the panorama image motion compensation unit 170 as the reproduced panorama image. It is possible that the modified panorama image is the same as the reproduced panorama image according to the predetermined value.
- the panorama image motion estimation unit 160 performs motion estimation, using the reference panorama image stored in the frame memory 150. Specifically, the panorama image motion estimation unit 160 receives information regarding the current panorama image, obtains a motion vector of the current panorama image by performing motion estimation on the current panorama image using the reference panorama image stored in the frame memory 150, and outputs the motion vector to the VLC 190. Motion estimation and compensation are performed in units of pre ⁇ determined blocks referred to as data units. In this embodiment, the data units may be 16x16 macro blocks.
- the panorama image motion compensation unit 170 performs the motion com ⁇ pensation.
- the panorama image motion compensation unit 170 receives the motion vector of a current macro block ofthe current panorama image from the panorama image motion estimation unit 160 and thereference panorama image of the frame memory 150, and outputs a reference macro block corresponding to the current macro block to the subtraction unit 180 using themotion vector ofthe current macro block of the current panorama image and the reference panorama image of the frame memory 150.
- the panorama image motion compensation unit 170 may use the reproduced panorama image and the motion vector togeneratethe reference macro block.
- the subtraction unit 180 outputs a residual signal between the current macro block and the reference macro block to the transforming unit 110.
- the residual signal is transformed by the transforming unit 110, quantized by the quantizing unit 115, and variable-length coded by the VLC 190.
- the motion vector of the current macro block generated by the panorama image motion estimation unit 160 is input directly to and variable-length coded by the VLC 190.
- FIGS. 6 A and 6B are flowcharts illustrating a method of estimating the motion of a panorama image according to an embodiment of the presentgeneral inventive concept.
- the panorama image motion estimation unit 160 estimates a motion vector of a current data unit using motion vectors of a plurality of previous data units adjacent to the current data unit (S310).
- a data unitZ is a current data unit
- the data units A, B, C and D are previous data units required for estimation of a motion vector of the current data unit X.
- the data units may be 16x16 macro blocks.
- the current data unit X is included in a current frame, and the plurality of previous data units A, B, C and D are included in a previous frame.
- the panorama image motion estimation unit 160 detects the motion vectors of the previous macro blocksA, B, C, and D stored in an internal memory (not shown). When all the previous macro blocksA through D are present, the motion vector of the current macro block X is estimated according to predetermined or conventional motion estimation, using the detected motion vectors.
- FIG. 7A illustrates a case where the previous macro blocks A and D are not present in a panorama image, and thus, their motion vectors are unavailable for motion estimation of the current macro block X.
- FlG. 7B illustrates a case where the previous macro block C is not present in a panorama image, and thus, its motion vector is un ⁇ available for motion estimation of the current macro block X.
- a spatial relation between the right and left borders of a panorama image with a 360 ° omni-directional view is very high. That is, a distance between the right and left borders of the panorama image is substantially 0.
- the motion vectors of the previous macro blocks required for motion estimation are determined using the above characteristics of the panorama image. For instance, referring to FlG. 7 A, a previous macro block D' at a right side of the panorama image and on a Y-axis on which the previous macro block D is positioned is substantially the same as the previous macro block D.
- a motion vector of the previous macro block D' is considered to be the same as that of the previous macro block D and can be used in estimation of the motion vector of the current macro block X.
- the motion vector of a previous macro block at a right side of the panorama image and on an Y-axis on which the previous macro block A is positioned is predicted after motion estimation of the current macro block X , there is no available motion vector for the previous macro block A. Accordingly, the motion vector of the previous macro block A required for estimation of the motion vector of the current macro block X is set to 0.
- a previous macro block C" at a left side of the panorama image and on an Y-axis on which the previous macro block C is positioned is sub ⁇ stantially the same as the previous macro block C.
- a motion vector of the previous macro block C" is considered the same as that of the previous macro block C and thus is used in estimation of the motion vector of the current macro block X.
- the panorama image motion estimation unit 160 determines whether the reference macro block indicated by the estimated motion vector is present in a reference image (or reference panorama image)in operation S315.
- the reference image is stored in the frame memory 150.
- the pixels of the reference macro block are fetched from the frame memory 150 (S335), and the similarity between the current macro block X and the reference macro block is determined using a predetermined evaluation function (S335).
- FlG. 8A illustrates a case where the reference macro block is positioned at a border of the reference image.
- FlG. 8B illustrates a case where the reference macro block is positioned outside the reference image.
- the panorama image motion estimation unit 160 fetches all pixel values of the reference macro block from the padded reference image in the frame memory 150 (S325). Thereafter, the similarity between the current macro block X and the reference macro block is evaluated using a predetermined evaluation function (S335).
- a predetermined evaluation function S335
- a sum of absolute differences (S AD) function, a sum of absolute transformed differences (SATD) function, or a sum of squared differences (SSD) function is used as the predetermined evaluation function.
- the reference image is a cylindrical image obtained by connecting the right and left borders of the reference image
- the reference image is a two-dimensional (2D) plane image such as that shown in FlG. 4B
- the cylindrical image such as that shown in FlG. 4A is obtained by connecting the right and left borders of the 2D plane image. That is, when the reference image is a cylindrical image, the values of all pixel values of the reference data unit can be obtained from the cylindrical image.
- the panorama image motion estimation unit 160 changes a position of the reference macro block in a predetermined search range and determines the similarity between the changed reference macro block and the current macro block X (S 340 and S345). After the evaluation of the similarity between the current macro block X and each of a plurality of reference macro blocks in the predetermined search range, the panorama image motion estimation unit 160 determines a reference macro block that is the most similar to the current macro block X from the plurality of reference macro blocks, and generates a motion vector of the determined reference macro block (S350).
- FlG. 10 is a diagram illustrating a motion vector of a current macro block 510.
- a reference numeral 530 denotes the macro block that is most similar to the current macro block 510 and present on the padded reference image
- a reference numeral 540 denotes the macro block that corresponds to the macro block 530 and is present on the non-padded image 500.
- a reference numeral 550 denotes the motion vector of the current macro block 510.
- a reference numeral 560 denotes the motion vector of the current macro block 510. That is, the motion vector of the current macro block 510 may be one of the motion vectors 550 and 560.
- the motion vector 550 of the reference macro block 530 maybe determined as the motion vector of the current macro block 510.
- FlG. 11 is a block diagram of a decoding unit that decodes a motion vector of a panorama image according to an embodiment of the present invention.
- the decoder includes a variable-length decoder (VLD) 710, an inverse quantizing unit 720, an inverse transforming unit 730, an adding unit 740, a panorama image motion compensating unit 750, a clipping unit 760, and a frame memory 770.
- VLD variable-length decoder
- the VLD 710 decodes an input bitstream using a variable-length coding/ decodingmethod.
- a motion vector and a residual signal between a macro block and a reference macro block output from the VLD 710 are input to the panorama image m otion compensating unit 750 and the inverse quantizing unit 720, respectively.
- the frame memory 770 stores a reference panorama image obtained by sequentially inputting the input bitstream to the inverse quantizing unit 720, the inverse transforming unit 730, and the clipping unit 760.
- the reference panorama image stored in the frame memory 770 is used for compensation for the motion of a newly input panorama image (current panorama image).
- the panorama image motion compensating unit 750 performs motion compensation using the reference panorama image stored in the frame memory 770.
- the panorama image motion compensating unit 750 receives a motion vector of a current macro block of the panorama image from an encoder such as that shown in FlG. 5, reads a reference macro block of a previous frame corresponding to the current macro block in the frame memory 770, and outputs the read reference macro block to the adding unit 740.
- the adding unit 740 receives the residual signal between the current macro block and the reference macro block that are inversely quantized by the inverse quantizing unit 720 and inversely transformed by the inverse transforming 730.
- the adding unit 740 reproduces the current macro block using the residual signal between the current macro block and the reference macro block, and the reference macro block input from the panorama image motion compensating unit 750.
- the clipping unit 760 normalizes the reproduced current macro block output from the adding unit 740.
- FIG. 12 is a flowchart illustrating a method of com ⁇ pensating for the motion of a panorama image according to an embodiment of the pre- sentgeneral inventive concept.
- the panorama image motion compensating unit 750 receives a motion vector of a current data unit on which motion estimation is to be performed from the VLD 710 (S910).
- data units may be 16 ' 16 macro blocks.
- the panorama image motion compensating unit 750 determines whether a reference macro block indicated by the motion vector of the current macro block is present in a reference image (S920).
- the reference image is stored in the frame memory 770.
- the values of all pixels of the reference macro block are read from the frame memory 770 (S950), and the current macro block is reproduced (S960).
- the adding unit 740 reproduces the current macro block, using the residual signal between the current macro block and the reference macro block output from the inversely transforming unit 730 and the reference macro block output from the panorama image motion compensating unit 750.
- the panorama image motion compensating unit 750 reads the values of all pixels of the reference macro block from the padded reference image from the frame memory 770 (S940).
- the reference image is a cylindrical image obtained by connecting the left and right borders of the reference image
- the reference image is a 2D plane image such as that shown in FlG. 4B
- the cylindrical image such as that shown in FlG. 4B is obtained by connecting the left and right borders of the 2D plane image. That is, if the reference image is the cylindrical image, the values of all pixels of the reference data unit can be obtained from the cylindrical image.
- the adding unit 740 reproduces the current macro block using the residual signal between the current macro block and the reference macro block and the reference macro block input from the panorama image motion compensating unit 750 (S960).
- the present general inventive concept may be embodied as computer readable code in a computer readable medium.
- the computer readable medium may be any recording apparatus capable of storing data that is read by a computer system, e.g., a read-only memory (ROM), a random access memory (RAM), a compact disc (CD)-ROM, a magnetic tape, a floppy disk, an optical data storage device, and so on.
- the computer readable medium may be a carrier wave that transmits data via the Internet, for example.
- the computer readable medium can be distributed among computer systems that are interconnected through a network, and the present invention may be stored and implemented as a computer readable code in the distributed system.
- motion estimation and compensation are performed on a panorama image with a 360 ° omni ⁇ directional view based on that the spatial relation between right and left borders of the panorama image is very high, thereby increasing the efficiency and precision of motion estimation and compensation. Accordingly, it is possible to improve image quality, in particular, the image quality at the right and left borders of the panorama image.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2007525548A JP4842945B2 (en) | 2004-08-13 | 2005-08-12 | Panorama motion estimation and compensation |
EP05780526A EP1779672A4 (en) | 2004-08-13 | 2005-08-12 | Method and device for motion estimation and compensation for panorama image |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US60113704P | 2004-08-13 | 2004-08-13 | |
US60/601,137 | 2004-08-13 | ||
KR10-2004-0081353 | 2004-10-12 | ||
KR1020040081353A KR100677142B1 (en) | 2004-08-13 | 2004-10-12 | Motion estimation and compensation for panorama image |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2006016783A1 true WO2006016783A1 (en) | 2006-02-16 |
Family
ID=35839505
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/KR2005/002639 WO2006016783A1 (en) | 2004-08-13 | 2005-08-12 | Method and device for motion estimation and compensation for panorama image |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP1779672A4 (en) |
WO (1) | WO2006016783A1 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108777798A (en) * | 2017-04-01 | 2018-11-09 | 英特尔公司 | The mass selector based on 360 neighbours, range modulator, view manager and exercise estimator for figure |
US10609379B1 (en) * | 2015-09-01 | 2020-03-31 | Amazon Technologies, Inc. | Video compression across continuous frame edges |
CN112913250A (en) * | 2019-03-04 | 2021-06-04 | 华为技术有限公司 | Encoder, decoder and corresponding method using IBC search range optimization for arbitrary CTU sizes |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5510830A (en) * | 1992-01-09 | 1996-04-23 | Sony Corporation | Apparatus and method for producing a panorama image using a motion vector of an image in an image signal |
US6026195A (en) | 1997-03-07 | 2000-02-15 | General Instrument Corporation | Motion estimation and compensation of video object planes for interlaced digital video |
JP2000261808A (en) | 1999-03-08 | 2000-09-22 | Matsushita Electric Ind Co Ltd | Picture encoding method picture decoding method, observing system motion estimating method and its device |
JP2001086451A (en) * | 1999-09-13 | 2001-03-30 | Victor Co Of Japan Ltd | Panoramic image creating device |
WO2004049721A1 (en) * | 2002-11-26 | 2004-06-10 | British Telecommunications Public Limited Company | Method and system for estimating global motion in video sequences |
Family Cites Families (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11122521A (en) * | 1997-08-11 | 1999-04-30 | Sony Corp | Digital image pickup device and image pickup method |
US20040001546A1 (en) * | 2002-06-03 | 2004-01-01 | Alexandros Tourapis | Spatiotemporal prediction for bidirectionally predictive (B) pictures and motion vector prediction for multi-picture reference motion compensation |
-
2005
- 2005-08-12 EP EP05780526A patent/EP1779672A4/en not_active Ceased
- 2005-08-12 WO PCT/KR2005/002639 patent/WO2006016783A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5510830A (en) * | 1992-01-09 | 1996-04-23 | Sony Corporation | Apparatus and method for producing a panorama image using a motion vector of an image in an image signal |
US6026195A (en) | 1997-03-07 | 2000-02-15 | General Instrument Corporation | Motion estimation and compensation of video object planes for interlaced digital video |
JP2000261808A (en) | 1999-03-08 | 2000-09-22 | Matsushita Electric Ind Co Ltd | Picture encoding method picture decoding method, observing system motion estimating method and its device |
JP2001086451A (en) * | 1999-09-13 | 2001-03-30 | Victor Co Of Japan Ltd | Panoramic image creating device |
WO2004049721A1 (en) * | 2002-11-26 | 2004-06-10 | British Telecommunications Public Limited Company | Method and system for estimating global motion in video sequences |
Non-Patent Citations (1)
Title |
---|
See also references of EP1779672A4 |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10609379B1 (en) * | 2015-09-01 | 2020-03-31 | Amazon Technologies, Inc. | Video compression across continuous frame edges |
CN108777798A (en) * | 2017-04-01 | 2018-11-09 | 英特尔公司 | The mass selector based on 360 neighbours, range modulator, view manager and exercise estimator for figure |
CN108777798B (en) * | 2017-04-01 | 2024-08-13 | 英特尔公司 | 360 Neighbor-based quality selector, range adjuster, viewport manager, and motion estimator for graphics |
CN112913250A (en) * | 2019-03-04 | 2021-06-04 | 华为技术有限公司 | Encoder, decoder and corresponding method using IBC search range optimization for arbitrary CTU sizes |
EP3868111A4 (en) * | 2019-03-04 | 2022-03-02 | Huawei Technologies Co., Ltd. | An encoder, a decoder and corresponding methods using ibc search range optimization for abitrary ctu size |
CN112913250B (en) * | 2019-03-04 | 2023-10-20 | 华为技术有限公司 | Encoder, decoder and corresponding methods using IBC search range optimization for arbitrary CTU sizes |
Also Published As
Publication number | Publication date |
---|---|
EP1779672A1 (en) | 2007-05-02 |
EP1779672A4 (en) | 2010-10-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4842945B2 (en) | Panorama motion estimation and compensation | |
US7623682B2 (en) | Method and device for motion estimation and compensation for panorama image | |
JP4846719B2 (en) | Panorama motion estimation and compensation | |
JP4575447B2 (en) | Method and apparatus for generating virtual image regions for motion estimation and compensation of panoramic images | |
KR100700218B1 (en) | Method and Device for making virtual image region for motion estimation and compensation of panorama image | |
KR102162856B1 (en) | Apparatus and method for video motion compensation | |
US9113170B2 (en) | Motion vector decision apparatus, motion vector decision method and computer readable storage medium | |
KR101456491B1 (en) | Method and apparatus for encoding and decoding image based on plurality of reference pictures | |
CN114080812A (en) | Inter prediction based image or video coding using SBTMVP | |
KR101360279B1 (en) | Method and apparatus for sharing motion information using global disparity estimation by macroblock unit, and method and apparatus for encoding/decoding multi-view video image using it | |
US20100329336A1 (en) | Method and apparatus for encoding and decoding based on inter prediction using image inpainting | |
WO2006126694A2 (en) | Video coding using an alternative reference frame for motion compensated prediction | |
JP7248013B2 (en) | Decoding method, decoding device, encoding device and program | |
JP4898415B2 (en) | Moving picture coding apparatus and moving picture coding method | |
WO2006016783A1 (en) | Method and device for motion estimation and compensation for panorama image | |
CN111630862B (en) | Method and apparatus for encoding and decoding a multi-view video sequence representing omni-directional video | |
KR101407719B1 (en) | Multi-view image coding method and apparatus using variable GOP prediction structure, multi-view image decoding apparatus and recording medium storing program for performing the method thereof | |
CN114080809A (en) | Image or video coding of sub-block units based on temporal motion vector predictor candidates | |
CN109714601B (en) | Video encoding apparatus using depth information and method thereof | |
RU2793902C1 (en) | Image decoding method to perform inter frame prediction when prediction mode for current block cannot be selected and device for this method | |
RU2794519C2 (en) | Method and device for video encoding/decoding using in-block copying mode and method for bit stream transmission | |
RU2776664C2 (en) | Methods and devices for encoding and decoding of multi-view video sequence of omnidirectional video | |
JP4131287B2 (en) | Motion vector detection apparatus and motion vector detection method | |
GB2585022A (en) | Video coding and decoding | |
JP2009253384A (en) | Moving image encoder and moving image coding method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DPEN | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed from 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2005780526 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 183/MUMNP/2007 Country of ref document: IN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 200580027187.3 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2007525548 Country of ref document: JP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWP | Wipo information: published in national office |
Ref document number: 2005780526 Country of ref document: EP |