WO2009069997A2 - Method for image geometric transformation - Google Patents
Method for image geometric transformation Download PDFInfo
- Publication number
- WO2009069997A2 WO2009069997A2 PCT/MY2008/000168 MY2008000168W WO2009069997A2 WO 2009069997 A2 WO2009069997 A2 WO 2009069997A2 MY 2008000168 W MY2008000168 W MY 2008000168W WO 2009069997 A2 WO2009069997 A2 WO 2009069997A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- axis
- destination
- arc
- rectangular
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000009466 transformation Effects 0.000 title claims abstract description 18
- 238000013507 mapping Methods 0.000 claims abstract description 3
- 238000005070 sampling Methods 0.000 claims description 12
- 238000012952 Resampling Methods 0.000 claims description 5
- 230000001419 dependent effect Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 2
- 230000001131 transforming effect Effects 0.000 claims 1
- 238000003384 imaging method Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
Definitions
- the present invention relates to a method for geometric transformation of video images captured by wide-angle lens. More particularly, the present invention relates to a method for correcting distorted video images using arc-based remapping transformation.
- First method is by using high resolution imaging and second method is by using wide coverage imaging.
- wide coverage imaging to maximize the coverage of the surveillance area, one needs to use a wide angle lenses such as fisheye lens to cover a wide field of view.
- images captured would be highly distorted and difficult for human interpretation. This effect is particularly apparent as the diameter increases from the centroid due to the nature of the wide angle lenses.
- the object to image space sampling ratio reduces as it moves from the centroid towards the edge of the circular images.
- U.S. Patent No. 6,005,611 also relates to a method and apparatus of performing perspective correction of digital videos captured by wide angle lenses.
- the method starts by capturing a wide angle digital video input by any suitable means and storing the captured image in a suitable memory means so portions of the image may be selected at a later time.
- a portion of the stored video is selected for viewing, a plurality of discrete viewing vectors in three dimensional space are chosen on the video input and transformed to a plurality of control points in a two-dimensional plane or any other suitable surface.
- the area between these points which is still warped from the original wide angle image capture is then transformed to a perspective corrected field of view and displayed on a suitable displaying apparatus, such as a monitor or head mounted display.
- control points and pre-computation of inverse matrix is needed to calculate the polynomial transfer functions.
- U.S. Patent No. 7,184,609 which relates to an apparatus and method for alleviating distortion and perception problems in images captured by omni-directional cameras.
- a warp table from pixel coordinates of a panoramic image and applying the warp table to the panoramic image to create a corrected panoramic image using a parametric class of warping functions that include Spatially Varying Uniform (SVU) scaling functions.
- the warp table is concatenated with a stitching table used to create the panoramic image.
- the present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, the present invention relates to a method for geometric transformation of video captured by a wide angle lens. An image captured by the lens is circular in shape due to the optics of the lens. The geometric transformation technique allows the restoration of the original view that has been captured by the lens.
- the method for geometric transformation of video images comprises:
- the lens is preferably a fisheye lens.
- Fig. 1 illustrates a method for geometric transformation of video images captured by fisheye lens camera using arc-based remapping transformation according to the present invention
- Fig. 2 illustrates a non-linear stretching method in horizontal axis to achieve linear distribution of horizontal lines across the x-axis according to the present invention
- Fig. 3 illustrates a non-linear stretching method in vertical axis to achieve a corrected rectangular image according to the present invention.
- a scene 200 in object space is captured by a fisheye lens camera, wherein the captured scene produced is a distorted video image in a circular form as a circular video image 210.
- the fisheye lens camera transforms the scene 200 into an image space representation and since the scene to image space sampling is non-linear, the straight lines of the scene 200 appears as arc image data 230 after captured by the fisheye lens, resulting into a circular video image 210.
- the circular video image 210 is decomposed 220 using arc-based remapping transformation by extracting pixel data from the arc image data 230 that forms the circular video image 210. Center of rotation and the radius of the circular video image 210 are detected at this stage to initialize destination maps from position of x-axis and y-axis of the arc image data 230.
- the coordinate of the leftmost arc image data 230 is estimated which enables image detection and of the content of the circular video image 210 until all pixels in the arc image data 230 are read.
- the arc image data 230 of the whole circular video image 210 is collected and stored in memory buffer for later processing.
- the extracted decomposed arc image data are then straighten up and mapped 240 to straight vertical lines or columns in the rectangular image buffer 250 according to the relative position from the center of the circular image 230.
- the pixel reading and writing process finishes when the entire pixel data in the circular video image 210 has been addressed.
- the rectangular image 250 is corrected using non-linear stretching method in horizontal 260 and non-linear stretching method in vertical axis 280 to achieve a correct rectangular image 300, which is explained in detail in Fig. 2 and Fig. 3.
- Fig. 2 illustrates the non-linear stretching method in horizontal axis (x-axis) 260 to achieve linear distribution of horizontal lines across the x-axis.
- the rectangular image 250 from the arc-based remapping transformation is fed to the horizontal sampling correction transformation 260 to correct the rectangular image 250 and remapped it to an equally distributed line sampled image.
- Radius and center point of the circular video image 210 has to be identified first to initialize destination maps from position of x-axis and y-axis of the arc image data 230 (as shown in Fig. 1).
- the non-linear stretching method in x-axis method 260 comprises the following steps:
- the horizontal stretching factor i.e. the level of stretching is dependent upon the destination image dimension and the rectangular image 250.
- the rectangular image 270 that has gone through the non-linear stretching method in horizontal axis 275 is further fed to the vertical sampling correction transformation 280 to correct the rectangular image 270 and remapped it to an equally distributed line sampled image.
- the non-linear resampling of the rectangular image in y-axis 280 comprises the following steps:
- the vertical stretching factor i.e. the level of stretching is defined and is dependent on the destination image and the rectangular image 270.
- the vertical stretching is applied to the rectangular image 270 only defining the vertical stretching factor.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The present invention relates to a method for geometric transformation of video captured by a wide angle lens. The geometric transformation technique allows the restoration of the original view that has been captured by the lens. Accordingly, the method for geometric transformation of video images, comprises: capturing a scene (200) from a lens in a form of circular video image (210), decomposing (220) the circular video image (210) into arc image data (230), straightening and mapping (240) the decomposed lines (230) into straight vertical lines forming a rectangular image (250), and correcting the rectangular image using a non-linear stretching method in horizontal (260) and a non-linear stretching method vertical axes (280) to achieve a corrected rectangular image (300).
Description
METHOD FOR IMAGE GEOMETRIC TRANSFORMATION
The present invention relates to a method for geometric transformation of video images captured by wide-angle lens. More particularly, the present invention relates to a method for correcting distorted video images using arc-based remapping transformation.
BACKGROUND TO THE INVENTION
In visual surveillance, there are two (2) preferred methods in capturing and viewing surveillance videos. First method is by using high resolution imaging and second method is by using wide coverage imaging. In wide coverage imaging, to maximize the coverage of the surveillance area, one needs to use a wide angle lenses such as fisheye lens to cover a wide field of view. However, images captured would be highly distorted and difficult for human interpretation. This effect is particularly apparent as the diameter increases from the centroid due to the nature of the wide angle lenses. The object to image space sampling ratio reduces as it moves from the centroid towards the edge of the circular images.
U.S. Patent No. 6,005,611 also relates to a method and apparatus of performing perspective correction of digital videos captured by wide angle lenses. The method starts by capturing a wide angle digital video input by any suitable means and storing the captured image in a suitable memory means so portions of the image may be selected at a later time. When a portion of the stored video is selected for viewing, a plurality of discrete viewing vectors in three dimensional space are chosen on the video input and transformed to a plurality of control points in a two-dimensional plane or any other suitable surface. The area between these points which is still warped from the original wide angle image capture is then transformed to a perspective corrected field of view and displayed on a suitable displaying apparatus, such as a monitor or head mounted display. In this method, control points and pre-computation of inverse matrix is needed to calculate the polynomial transfer functions.
Another example is U.S. Patent No. 7,184,609 which relates to an apparatus and method for alleviating distortion and perception problems in images captured by omni-directional cameras. Generally, involves generating a warp table from pixel coordinates of a panoramic image and applying the warp table to the panoramic image to create a corrected panoramic image using a parametric class of warping
functions that include Spatially Varying Uniform (SVU) scaling functions. In one embodiment the warp table is concatenated with a stitching table used to create the panoramic image.
SUMMARY OF THE INVENTION
The present invention is directed to overcoming one or more of the problems set forth above. Briefly summarized, the present invention relates to a method for geometric transformation of video captured by a wide angle lens. An image captured by the lens is circular in shape due to the optics of the lens. The geometric transformation technique allows the restoration of the original view that has been captured by the lens.
Accordingly, the method for geometric transformation of video images, comprises:
capturing a scene from a lens in a form of circular video image;
decomposing the circular video image into arc image data;
straightening and mapping the decomposed lines into straight vertical lines forming a rectangular image; and
correcting the rectangular image using a non-linear stretching method in horizontal and vertical axes to achieve a corrected rectangular image.
The lens is preferably a fisheye lens.
It is an advantage of the present invention to provide a new transformation method to restore the captured circular video images into rectangular video by correcting all the distortion curves in the circular video images that have been captured using a fisheye lens.
It is another advantage of the present invention to transform distorted video images into a rectangular view of the scene that allows human to interpret and understand the content of the image easily and naturally.
It is yet another advantage of the present invention for panoramic imaging applications using only one camera, that is a fisheye lens camera instead of multiple cameras.
These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiment and appended claims, and by reference to the accompanying drawings.
BRiEF DESCRIPTION OF THE DRAWINGS
The present invention will now be described in greater detail, by way of an example, with reference to the accompanying drawings, in which:
Fig. 1 illustrates a method for geometric transformation of video images captured by fisheye lens camera using arc-based remapping transformation according to the present invention;
Fig. 2 illustrates a non-linear stretching method in horizontal axis to achieve linear distribution of horizontal lines across the x-axis according to the present invention; and
Fig. 3 illustrates a non-linear stretching method in vertical axis to achieve a corrected rectangular image according to the present invention.
DETAILED DESCRIPTIONS OF THE INVENTION
In the following description of the preferred embodiments of the present invention, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.
Referring to Fig. 1 , a scene 200 in object space is captured by a fisheye lens camera, wherein the captured scene produced is a distorted video image in a circular form as a circular video image 210.
In other words, the fisheye lens camera transforms the scene 200 into an image space representation and since the scene to image space sampling is non-linear, the straight lines of the scene 200 appears as arc image data 230 after captured by the fisheye lens, resulting into a circular video image 210.
Further, the circular video image 210 is decomposed 220 using arc-based remapping transformation by extracting pixel data from the arc image data 230 that forms the circular video image 210. Center of rotation and the radius of the circular video image 210 are detected at this stage to initialize destination maps from position of x-axis and y-axis of the arc image data 230.
With the radius and center of rotation information, the coordinate of the leftmost arc image data 230 is estimated which enables image detection and of the content of the circular video image 210 until all pixels in the arc image data 230 are read.
The arc image data 230 of the whole circular video image 210 is collected and stored in memory buffer for later processing. The extracted decomposed arc image data are then straighten up and mapped 240 to straight vertical lines or columns in the rectangular image buffer 250 according to the relative position from the center of the circular image 230.
The pixel reading and writing process finishes when the entire pixel data in the circular video image 210 has been addressed. After arc-based remapping transformation, the rectangular image 250 is corrected using non-linear stretching method in horizontal 260 and non-linear stretching method in vertical axis 280 to achieve a correct rectangular image 300, which is explained in detail in Fig. 2 and Fig. 3.
Fig. 2 illustrates the non-linear stretching method in horizontal axis (x-axis) 260 to achieve linear distribution of horizontal lines across the x-axis. The rectangular image 250 from the arc-based remapping transformation is fed to the horizontal sampling correction transformation 260 to correct the rectangular image 250 and remapped it to an equally distributed line sampled image.
Radius and center point of the circular video image 210 has to be identified first to initialize destination maps from position of x-axis and y-axis of the arc image data 230 (as shown in Fig. 1).
The non-linear stretching method in x-axis method 260 comprises the following steps:
determining destination image dimension for non-linear x-axis resampling;
determining horizontal stretching factors for the x-axis according to the respective position and destination size 255;
determining the center point of image to increase the sampling of the image from the center towards the left edge of the image 250, and
increasing the sampling of the image 250 from the center toward the right edge until the end point destination of the x-axis is reached to achieve linear distribution of horizontal lines across the x-axis in the rectangular image 270 (as shown in Fig. 1).
The horizontal stretching factor, i.e. the level of stretching is dependent upon the destination image dimension and the rectangular image 250.
In Fig. 3, the rectangular image 270 that has gone through the non-linear stretching method in horizontal axis 275 is further fed to the vertical sampling correction transformation 280 to correct the rectangular image 270 and remapped it to an equally distributed line sampled image.
The non-linear resampling of the rectangular image in y-axis 280 comprises the following steps:
determining destination image dimension for non-linear y-axis resampling,
determining vertical stretching factors for the y-axis according to the respective position and destination size 277,
determining the center point of image to increase the sampling of the image from the center towards the top edge of the image 270, and
increasing the sampling of the image 270 from the center toward the bottom edge until the end point destination of the y-axis is reached.
The vertical stretching factor, i.e. the level of stretching is defined and is dependent on the destination image and the rectangular image 270. The vertical stretching is applied to the rectangular image 270 only defining the vertical stretching factor. After the two non-linear stretching method in horizontal axis 260 and non-linear stretching method in vertical axis 280, the rectangular image 300 becomes horizontally and vertically corrected and looks natural as if it is captured from a standard camera.
Claims
1. A method for transforming video images to rectangular video using arc-based transformation, comprising the steps of:
capturing a scene (200) by a lens in a form of distorted or circular video images (210);
decomposing (220) the captured circular video (210) image into arc image data (230);
straightening and mapping the decomposed lines (240) into straight vertical lines forming a rectangular image (250) according to the level of distortion; and
correcting the rectangular image (250) using a non-linear stretching method in horizontal axis (260) and non-linear stretching method in vertical axes (280) to achieve a corrected rectangular image (300)
2. A method according to claim 1 , wherein the lens is a fisheye lens.
3. A method according to claim 1, wherein the circular video image (210) is decomposed (220) by extracting pixel data from the arc image data (230).
4. A method according to claim 1, wherein to decompose (220) the circular video image (210) into the arc image data (230), radius and center point of the circular video (210) is identified to initialize destination maps from position of x-axis and y-axis of the arc image data (230).
5. A method according to claim 4, wherein coordinate of the leftmost arc image data (230) is estimated after radius and center point of the circular video image (210) is identified which enables image detection and reading of the content of the circular video image (210) until all the pixels in the arc image data (230) are read.
6. A method according to claim 1 , wherein the non-linear stretching method in x- axis (260) comprises:
determining destination image dimension for non-linear x-axis resampling,
determining horizontal stretching factors for the x-axis according to the respective position and destination size (255),
determining center point of image (250) to increase the sampling of the image from the center towards the left edge of the image (250), and
increasing sampling of the image (250) from the center toward the right edge until the end point destination of the x-axis is reached (260).
7. A method according to claim 6, wherein the horizontal stretching factor is dependent upon the destination image dimension and the rectangular image (250).
8. A method according to claim 1 , wherein the non-linear stretching method in y- axis (280) comprises:
determining destination image dimension for non-linear y-axis resampling,
determining vertical stretching factors for the y-axis according to the respective position and destination size (277),
determining center point of image (275) to increase the sampling of the image from the center towards the top edge of the image (275), and
increasing sampling of the image (275) from the center toward the bottom edge until the end point destination of the y-axis is reached (280).
9. A method according to claim 8, wherein the vertical stretching factor is dependent on the destination image and rectangular image (275).
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| MYPI20072105A MY146158A (en) | 2007-11-27 | 2007-11-27 | Method for image geometric transformation |
| MYPI20072105 | 2007-11-27 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| WO2009069997A2 true WO2009069997A2 (en) | 2009-06-04 |
| WO2009069997A3 WO2009069997A3 (en) | 2009-07-30 |
Family
ID=40679169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/MY2008/000168 WO2009069997A2 (en) | 2007-11-27 | 2008-11-26 | Method for image geometric transformation |
Country Status (2)
| Country | Link |
|---|---|
| MY (1) | MY146158A (en) |
| WO (1) | WO2009069997A2 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102291527A (en) * | 2011-08-11 | 2011-12-21 | 杭州海康威视软件有限公司 | Panoramic video roaming method and device based on single fisheye lens |
| CN104574289A (en) * | 2013-10-29 | 2015-04-29 | 深圳市中航比特通讯技术有限公司 | Fish-eye image distortion correction algorithm based on ellipsoidal model |
| CN115439365A (en) * | 2022-09-07 | 2022-12-06 | 生态环境部卫星环境应用中心 | Geometric correction method and device for image of high-tower camera |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002312778A (en) * | 2001-04-09 | 2002-10-25 | Be Here Corp | Method and device for electronically distributing motion panoramic image |
| US20040247173A1 (en) * | 2001-10-29 | 2004-12-09 | Frank Nielsen | Non-flat image processing apparatus, image processing method, recording medium, and computer program |
| JP2007192832A (en) * | 2007-03-06 | 2007-08-02 | Iwate Univ | Calibrating method of fish eye camera |
-
2007
- 2007-11-27 MY MYPI20072105A patent/MY146158A/en unknown
-
2008
- 2008-11-26 WO PCT/MY2008/000168 patent/WO2009069997A2/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002312778A (en) * | 2001-04-09 | 2002-10-25 | Be Here Corp | Method and device for electronically distributing motion panoramic image |
| US20040247173A1 (en) * | 2001-10-29 | 2004-12-09 | Frank Nielsen | Non-flat image processing apparatus, image processing method, recording medium, and computer program |
| JP2007192832A (en) * | 2007-03-06 | 2007-08-02 | Iwate Univ | Calibrating method of fish eye camera |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102291527A (en) * | 2011-08-11 | 2011-12-21 | 杭州海康威视软件有限公司 | Panoramic video roaming method and device based on single fisheye lens |
| CN104574289A (en) * | 2013-10-29 | 2015-04-29 | 深圳市中航比特通讯技术有限公司 | Fish-eye image distortion correction algorithm based on ellipsoidal model |
| CN104574289B (en) * | 2013-10-29 | 2017-09-05 | 深圳市中航比特通讯技术有限公司 | A kind of fish eye images aberration correction algorithm based on ellipsoid surface model |
| CN115439365A (en) * | 2022-09-07 | 2022-12-06 | 生态环境部卫星环境应用中心 | Geometric correction method and device for image of high-tower camera |
| CN115439365B (en) * | 2022-09-07 | 2023-02-17 | 生态环境部卫星环境应用中心 | Geometric correction method and device for image of high-tower camera |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2009069997A3 (en) | 2009-07-30 |
| MY146158A (en) | 2012-06-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI423659B (en) | Image corretion method and related image corretion system thereof | |
| CA3019163C (en) | Generating intermediate views using optical flow | |
| US10748243B2 (en) | Image distortion transformation method and apparatus | |
| EP2820593B1 (en) | Method and system for adaptive perspective correction of ultra wide-angle lens images | |
| US9196022B2 (en) | Image transformation and multi-view output systems and methods | |
| US9706135B2 (en) | Method and apparatus for generating an image cut-out | |
| WO2018235163A1 (en) | Calibration device, calibration chart, chart pattern generation device, and calibration method | |
| CN108805807B (en) | Splicing method and system for ring scene images | |
| JP2013009050A (en) | Image processing apparatus and image processing method | |
| CN105741233B (en) | Video image spherical surface splicing method and system | |
| JP2008048443A (en) | Fisheye lens camera apparatus and image extraction method thereof | |
| CN111866523B (en) | Panoramic video synthesis method and device, electronic equipment and computer storage medium | |
| US10805534B2 (en) | Image processing apparatus and method using video signal of planar coordinate system and spherical coordinate system | |
| KR20110073386A (en) | Method and device for the invariant-affine recognition of shapes | |
| KR100948872B1 (en) | Camera image correction method and apparatus | |
| JP2019029721A (en) | Image processing apparatus, image processing method, and program | |
| WO2009069997A2 (en) | Method for image geometric transformation | |
| WO2018234817A1 (en) | Apparatus and method for stitching together multiple images | |
| WO2009069996A2 (en) | Apparatus and method for panoramic imaging | |
| WO2011158344A1 (en) | Image processing method, program, image processing device, and imaging device | |
| KR102480209B1 (en) | Multi-camera image synthesis method and multi-camera image synthesis apparatus | |
| JP5787930B2 (en) | Virtual navigation system for video | |
| Araki et al. | High quality panoramic image generation using multiple panoramic annular lens images | |
| KR20060015460A (en) | Stereoscopic panoramic image capture device | |
| JP2022077221A (en) | Image processing apparatus, image processing system, image processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 08855570 Country of ref document: EP Kind code of ref document: A2 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 08855570 Country of ref document: EP Kind code of ref document: A2 |