SG11201808923QA - Fisheye rendering with lens distortion correction for 360-degree video - Google Patents
Fisheye rendering with lens distortion correction for 360-degree videoInfo
- Publication number
- SG11201808923QA SG11201808923QA SG11201808923QA SG11201808923QA SG11201808923QA SG 11201808923Q A SG11201808923Q A SG 11201808923QA SG 11201808923Q A SG11201808923Q A SG 11201808923QA SG 11201808923Q A SG11201808923Q A SG 11201808923QA SG 11201808923Q A SG11201808923Q A SG 11201808923QA
- Authority
- SG
- Singapore
- Prior art keywords
- international
- california
- san diego
- degree video
- pct
- Prior art date
Links
- 238000009877 rendering Methods 0.000 title abstract 3
- 238000013507 mapping Methods 0.000 abstract 1
- 238000000034 method Methods 0.000 abstract 1
- 230000008520 organization Effects 0.000 abstract 1
- XOIQMTLWECTKJL-HXPDMXKUSA-M sodium;(3r,4s)-4-[(2s,5r,7s,8r,9s)-2-[(2r,5s)-5-ethyl-5-[(2r,3s,5r)-5-[(2s,3s,5r,6r)-6-hydroxy-6-(hydroxymethyl)-3,5-dimethyloxan-2-yl]-3-methyloxolan-2-yl]oxolan-2-yl]-7-hydroxy-2,8-dimethyl-1,10-dioxaspiro[4.5]decan-9-yl]-3-methoxy-2-methylpentanoate Chemical compound [Na+].C([C@@](O1)(C)[C@H]2CC[C@@](O2)(CC)[C@H]2[C@H](C[C@@H](O2)[C@@H]2[C@H](C[C@@H](C)[C@](O)(CO)O2)C)C)C[C@@]21C[C@H](O)[C@@H](C)[C@@H]([C@@H](C)[C@@H](OC)C(C)C([O-])=O)O2 XOIQMTLWECTKJL-HXPDMXKUSA-M 0.000 abstract 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/08—Projecting images onto non-planar surfaces, e.g. geodetic screens
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
- Geometry (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) (19) World Intellectual Property 11111111111111111110 11111110 11E111110 1111111111111111111110 HOED Organization International Bureau (10) International Publication Number (43) International Publication Date ......0\"\"1 WO 2017/204971 Al 30 November 2017 (30.11.2017) WIPO I PCT (51) International Patent Classification: California 92121-1714 (US). KARCZEWICZ, Marta; G06T 3/00 (2006.01) 5775 Morehouse Drive, San Diego, California 92121-1714 (21) International Application Number: (US). PCT/US2017/029441 (74) Agent: WONG, Claudine et al.; Kilpatrick Townsend & (22) International Filing Date: Stockton LLP, Mailstop: IP Docketing - 22, 1100 Peachtree 25 April 2017 (25.04.2017) Street, N.E., Suite 2800, Atlanta, Georgia 30309 (US). (25) Filing Language: English (81) Designated States (unless otherwise indicated, for every kind of national protection available): AE, AG, AL, AM, (26) Publication Language: English AO, AT, AU, AZ, BA, BB, BG, BH, BN, BR, BW, BY, BZ, (30) Priority Data: CA, CH, CL, CN, CO, CR, CU, CZ, DE, DJ, DK, DM, DO, 62/340,914 24 May 2016 (24.05.2016) US DZ, EC, EE, EG, ES, FI, GB, GD, GE, GH, GM, GT, HN, 15/495,709 24 April 2017 (24.04.2017) US HR, HU, ID, IL, IN, IR, IS, JP, KE, KG, KH, KN, KP, KR, KW, KZ, LA, LC, LK, LR, LS, LU, LY, MA, MD, ME, MG, (71) Applicant: QUALCOMM INCORPORATED [US/US]; MK, MN, MW, MX, MY, MZ, NA, NG, NI, NO, NZ, OM, Attn: International IP Administration, 5775 Morehouse Dri- PA, PE, PG, PH, PL, PT, QA, RO, RS, RU, RW, SA, SC, ye, San Diego, California 92121-1714 (US). SD, SE, SG, SK, SL, SM, ST, SV, SY, TH, TJ, TM, TN, TR, TT, TZ, UA, UG, US, UZ, VC, VN, ZA, ZM, ZW. (72) Inventors: VAN DER AUWERA, Geert; 5775 More- house Drive, San Diego, California 92121-1714 (US). (84) Designated States (unless otherwise indicated, for every COBAN, Muhammed; 5775 Morehouse Drive, San Diego, kind of regional protection available): ARIPO (BW, GH, = (54) Title: FISHEYE RENDERING WITH LENS DISTORTION CORRECTION FOR 360-DEGREE VIDEO = io% 10 2 _ 1134- \_ = = = = = (- Z j -io4. = ‘ • — -- -- .4- . •-•-•-• fr.% .•- . 7::: ......, ........\". P .- j \".\"'' EZCZirr . \" C e-' = - --- 1 E:=1 ,--,] E= \" 0=11=000 . A\" : or-i =loop. = = = • • ,.. ,...---.\" , --, .4k 1-----.11:=]=1 (r -----,_,_ \"I lie A ' = oo =coo • ' IN , ni g a t g.tr.1- .1.-.... • — V --::;,_ `7. - -- , _ - - > = Ib ik v\" YPI' 104- N. _ %IN 0 j -104 1-1 1-1 FIG. .1 IN 01 'V (57) : In various implementations, provided are systems and methods for correcting the distortion present in a fisheye image, and rendering the image for display as 360-degree video. In various implementations, a computing device can receive 2-dimensional ei ----. video data captured by an omnidirectional camera. The computing device can map an image from each video frame to a 3-dimensional IN hemispherical representation. In various implementations, this mapping can be executed using a polynomial model. The 3-dimensional © hemispherical representation can then be used in a 360-degree video presentation, to provide a virtual reality experience. ei O [Continued on next page] WO 2017/204971 Al MIDEDIMOMOIDEIROIDEM001010110MOMHIEMOVOIMIE GM, KE, LR, LS, MW, MZ, NA, RW, SD, SL, ST, SZ, TZ, UG, ZM, ZW), Eurasian (AM, AZ, BY, KG, KZ, RU, TJ, TM), European (AL, AT, BE, BG, CH, CY, CZ, DE, DK, EE, ES, FI, FR, GB, GR, HR, HU, IE, IS, IT, LT, LU, LV, MC, MK, MT, NL, NO, PL, PT, RO, RS, SE, SI, SK, SM, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, GW, KM, ML, MR, NE, SN, TD, TG). Published: — with international search report (Art. 21(3))
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201662340914P | 2016-05-24 | 2016-05-24 | |
| US15/495,709 US10699389B2 (en) | 2016-05-24 | 2017-04-24 | Fisheye rendering with lens distortion correction for 360-degree video |
| PCT/US2017/029441 WO2017204971A1 (en) | 2016-05-24 | 2017-04-25 | Fisheye rendering with lens distortion correction for 360-degree video |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| SG11201808923QA true SG11201808923QA (en) | 2018-12-28 |
Family
ID=58800890
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| SG11201808923QA SG11201808923QA (en) | 2016-05-24 | 2017-04-25 | Fisheye rendering with lens distortion correction for 360-degree video |
Country Status (9)
| Country | Link |
|---|---|
| US (1) | US10699389B2 (en) |
| EP (1) | EP3465607B1 (en) |
| JP (1) | JP2019523921A (en) |
| KR (1) | KR102373921B1 (en) |
| CN (1) | CN109155056B (en) |
| BR (1) | BR112018073986A2 (en) |
| SG (1) | SG11201808923QA (en) |
| TW (1) | TW201742435A (en) |
| WO (1) | WO2017204971A1 (en) |
Families Citing this family (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10200672B2 (en) * | 2016-08-17 | 2019-02-05 | Nextvr Inc. | Methods and apparatus for capturing images of an environment |
| KR20190052129A (en) | 2016-10-04 | 2019-05-15 | 김기백 | Image data encoding / decoding method and apparatus |
| US20190238888A1 (en) * | 2017-07-17 | 2019-08-01 | Ki Baek Kim | Image data encoding/decoding method and apparatus |
| CN107967665B (en) * | 2016-10-20 | 2021-07-13 | 株式会社理光 | Image processing method and image processing apparatus |
| US10332242B2 (en) | 2017-02-02 | 2019-06-25 | OrbViu Inc. | Method and system for reconstructing 360-degree video |
| US10915986B2 (en) * | 2017-03-20 | 2021-02-09 | Qualcomm Incorporated | Adaptive perturbed cube map projection |
| US10467775B1 (en) * | 2017-05-03 | 2019-11-05 | Amazon Technologies, Inc. | Identifying pixel locations using a transformation function |
| US10469751B2 (en) * | 2017-05-12 | 2019-11-05 | Garmin Switzerland Gmbh | Smart stabilization and reorientation for an omnidirectional camera |
| TWI640747B (en) * | 2017-11-28 | 2018-11-11 | 寶成工業股份有限公司 | Visual detection device for shoes |
| EP3745719A4 (en) * | 2018-01-23 | 2021-09-15 | LG Electronics Inc. | Method and apparatus for transmitting or receiving 360-degree video including camera lens information |
| US20190236416A1 (en) * | 2018-01-31 | 2019-08-01 | Microsoft Technology Licensing, Llc | Artificial intelligence system utilizing microphone array and fisheye camera |
| JP7310794B2 (en) * | 2018-03-16 | 2023-07-19 | ソニーグループ株式会社 | Image processing device and image processing method |
| CN110351492B (en) * | 2018-04-06 | 2021-11-19 | 中兴通讯股份有限公司 | Video data processing method, device and medium |
| TWI681366B (en) * | 2018-05-31 | 2020-01-01 | 廣達電腦股份有限公司 | Method and system for non-linearly stretching a cropped image |
| US10878595B2 (en) * | 2018-07-02 | 2020-12-29 | Intel Corporation | Dual model for fisheye lens distortion and an algorithm for calibrating model parameters |
| WO2020068960A1 (en) * | 2018-09-26 | 2020-04-02 | Coherent Logix, Inc. | Any world view generation |
| EP3651119A1 (en) * | 2018-11-06 | 2020-05-13 | Koninklijke Philips N.V. | Disparity estimation from a wide angle image |
| CN111263037B (en) * | 2018-11-30 | 2023-04-18 | 株式会社理光 | Image processing device, imaging device, video playback system, method, and program |
| US11875796B2 (en) | 2019-04-30 | 2024-01-16 | Microsoft Technology Licensing, Llc | Audio-visual diarization to identify meeting attendees |
| US10812921B1 (en) | 2019-04-30 | 2020-10-20 | Microsoft Technology Licensing, Llc | Audio stream processing for distributed device meeting |
| US11822081B2 (en) * | 2019-08-29 | 2023-11-21 | Apple Inc. | Optical module for head-mounted device |
| US11885965B1 (en) | 2019-09-23 | 2024-01-30 | Apple Inc. | Head-mounted display and display modules thereof |
| KR102263540B1 (en) * | 2019-09-27 | 2021-06-14 | (주)베이다스 | Method and apparatus for setting gt region on distored image |
| US11127111B2 (en) * | 2019-11-14 | 2021-09-21 | Qualcomm Incorporated | Selective allocation of processing resources for processing image data |
| CN110930299A (en) * | 2019-12-06 | 2020-03-27 | 杭州视洞科技有限公司 | Circular fisheye video display scheme based on hemisphere expansion |
| CN111787300B (en) * | 2020-07-29 | 2022-07-01 | 北京金山云网络技术有限公司 | VR video processing method and device and electronic equipment |
| CN112669429A (en) * | 2021-01-07 | 2021-04-16 | 稿定（厦门）科技有限公司 | Image distortion rendering method and device |
| EP4300963A4 (en) * | 2021-03-30 | 2024-05-08 | Panasonic Ip Corp America | Image encoding method, image decoding method, image processing method, image encoding device, and image decoding device |
| US11720995B2 (en) | 2021-06-04 | 2023-08-08 | Ford Global Technologies, Llc | Image rectification |
| EP4328847A1 (en) | 2021-07-09 | 2024-02-28 | Samsung Electronics Co., Ltd. | Electronic device and operation method of electronic device |
| CN115100026B (en) * | 2022-06-15 | 2023-07-14 | 佳都科技集团股份有限公司 | Label coordinate conversion method, device, equipment and storage medium based on target object |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5359363A (en) * | 1991-05-13 | 1994-10-25 | Telerobotics International, Inc. | Omniview motionless camera surveillance system |
| US6002430A (en) | 1994-01-31 | 1999-12-14 | Interactive Pictures Corporation | Method and apparatus for simultaneous capture of a spherical image |
| US5185667A (en) * | 1991-05-13 | 1993-02-09 | Telerobotics International, Inc. | Omniview motionless camera orientation system |
| US5796426A (en) * | 1994-05-27 | 1998-08-18 | Warp, Ltd. | Wide-angle image dewarping method and apparatus |
| US6415056B1 (en) | 1996-01-22 | 2002-07-02 | Matsushita Electric Industrial, Co., Ltd. | Digital image encoding and decoding method and digital image encoding and decoding device using the same |
| AU4453200A (en) * | 1999-04-08 | 2000-10-23 | Internet Pictures Corporation | Perspective-corrected video presentations |
| US6833843B2 (en) * | 2001-12-03 | 2004-12-21 | Tempest Microsystems | Panoramic imaging and display system with canonical magnifier |
| US7308131B2 (en) * | 2002-12-03 | 2007-12-11 | Ntt Docomo, Inc. | Representation and coding of panoramic and omnidirectional images |
| US7336299B2 (en) * | 2003-07-03 | 2008-02-26 | Physical Optics Corporation | Panoramic video system with real-time distortion-free imaging |
| US8594180B2 (en) * | 2007-02-21 | 2013-11-26 | Qualcomm Incorporated | 3D video encoding |
| JP4975592B2 (en) * | 2007-11-19 | 2012-07-11 | アルプス電気株式会社 | Imaging device |
| JP4629131B2 (en) * | 2008-09-03 | 2011-02-09 | 大日本印刷株式会社 | Image converter |
| JP5046132B2 (en) * | 2008-12-24 | 2012-10-10 | 株式会社富士通ゼネラル | Image data converter |
| US9124874B2 (en) * | 2009-06-05 | 2015-09-01 | Qualcomm Incorporated | Encoding of three-dimensional conversion information with two-dimensional video sequence |
| US9648299B2 (en) * | 2013-01-04 | 2017-05-09 | Qualcomm Incorporated | Indication of presence of texture and depth views in tracks for multiview coding plus depth |
| JP5843033B1 (en) * | 2014-05-15 | 2016-01-13 | 株式会社リコー | Imaging system, imaging apparatus, program, and system |
| US10104361B2 (en) * | 2014-11-14 | 2018-10-16 | Samsung Electronics Co., Ltd. | Coding of 360 degree videos using region adaptive smoothing |
-
2017
- 2017-04-24 US US15/495,709 patent/US10699389B2/en active Active
- 2017-04-25 BR BR112018073986-3A patent/BR112018073986A2/en not_active IP Right Cessation
- 2017-04-25 WO PCT/US2017/029441 patent/WO2017204971A1/en unknown
- 2017-04-25 EP EP17726733.3A patent/EP3465607B1/en active Active
- 2017-04-25 SG SG11201808923QA patent/SG11201808923QA/en unknown
- 2017-04-25 JP JP2018560066A patent/JP2019523921A/en active Pending
- 2017-04-25 KR KR1020187033550A patent/KR102373921B1/en active IP Right Grant
- 2017-04-25 CN CN201780028837.9A patent/CN109155056B/en active Active
- 2017-05-10 TW TW106115402A patent/TW201742435A/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| EP3465607A1 (en) | 2019-04-10 |
| US10699389B2 (en) | 2020-06-30 |
| CN109155056A (en) | 2019-01-04 |
| JP2019523921A (en) | 2019-08-29 |
| WO2017204971A1 (en) | 2017-11-30 |
| BR112018073986A2 (en) | 2019-02-26 |
| CN109155056B (en) | 2023-07-21 |
| KR102373921B1 (en) | 2022-03-11 |
| KR20190009753A (en) | 2019-01-29 |
| US20170345136A1 (en) | 2017-11-30 |
| TW201742435A (en) | 2017-12-01 |
| EP3465607B1 (en) | 2021-02-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| SG11201808923QA (en) | Fisheye rendering with lens distortion correction for 360-degree video | |
| SG11201907262QA (en) | Sphere pole projections for efficient compression of 360-degree video | |
| SG11201907264UA (en) | Sphere equator projection for efficient compression of 360-degree video | |
| SG11201901862YA (en) | Method and system for processing 360 degree video data | |
| SG11201907294PA (en) | Systems and methods for layered virtual features in an amusement park environment | |
| SG11201903452SA (en) | User location determination based on augmented reality | |
| SG11201906486SA (en) | Coding mcts-nested sei messages | |
| SG11201907090WA (en) | Affine motion information derivation | |
| SG11201909450WA (en) | 360-degree video coding using face continuities | |
| SG11201809299QA (en) | Formulations of an lsd1 inhibitor | |
| SG11201809219XA (en) | Systems and methods for generating stereoscopic, augmented, and virtual reality images | |
| SG11201807401RA (en) | Therapeutic membrane vesicles | |
| SG11201811001WA (en) | Vehicle navigation system using pose estimation based on point cloud | |
| SG11201910191RA (en) | A peptide capable of forming a gel for use in tissue engineering and bioprinting | |
| SG11201408112XA (en) | Random access and signaling of long-term reference pictures in video coding | |
| SG11201804506RA (en) | Systems and methods for rendering multiple levels of detail | |
| SG11201811318XA (en) | Respiratory interface | |
| SG11201408364VA (en) | Coding sei nal units for video coding | |
| SG11201901020RA (en) | Anti-siglec-7 antibodies for the treatment of cancer | |
| EP3331239A1 (en) | Moving image playback device, moving image playback method, moving image playback program, moving image playback system, and moving image transmitting device | |
| SG11201804841VA (en) | Hardware integrity check | |
| SG11201901860WA (en) | Systems and methods for signaling missing or corrupted video data | |
| SG11201809908TA (en) | Stabilized glycopeptide antibiotic formulations | |
| SG11201901644TA (en) | Optical detection of vibrations | |
| SG11201901901VA (en) | Techniques for new radio frame structures for data transmission |