CN105164728B - 用于混合实境的设备和方法 - Google Patents

用于混合实境的设备和方法 Download PDF

Info

Publication number
CN105164728B
CN105164728B CN201480024088.9A CN201480024088A CN105164728B CN 105164728 B CN105164728 B CN 105164728B CN 201480024088 A CN201480024088 A CN 201480024088A CN 105164728 B CN105164728 B CN 105164728B
Authority
CN
China
Prior art keywords
image
mask
pixels
pixel
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201480024088.9A
Other languages
English (en)
Chinese (zh)
Other versions
CN105164728A (zh
Inventor
迪特尔·施马尔施蒂格
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qualcomm Inc
Original Assignee
Qualcomm Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qualcomm Inc filed Critical Qualcomm Inc
Publication of CN105164728A publication Critical patent/CN105164728A/zh
Application granted granted Critical
Publication of CN105164728B publication Critical patent/CN105164728B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)
  • Telephone Function (AREA)
  • Image Analysis (AREA)
CN201480024088.9A 2013-04-30 2014-03-26 用于混合实境的设备和方法 Active CN105164728B (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201361817792P 2013-04-30 2013-04-30
US61/817,792 2013-04-30
US14/225,157 2014-03-25
US14/225,157 US9269003B2 (en) 2013-04-30 2014-03-25 Diminished and mediated reality effects from reconstruction
PCT/US2014/031926 WO2014178966A1 (en) 2013-04-30 2014-03-26 Diminished and mediated reality effects from reconstruction

Publications (2)

Publication Number Publication Date
CN105164728A CN105164728A (zh) 2015-12-16
CN105164728B true CN105164728B (zh) 2018-02-06

Family

ID=51789286

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201480024088.9A Active CN105164728B (zh) 2013-04-30 2014-03-26 用于混合实境的设备和方法

Country Status (5)

Country Link
US (1) US9269003B2 (enExample)
EP (1) EP2992508B1 (enExample)
JP (1) JP2016522485A (enExample)
CN (1) CN105164728B (enExample)
WO (1) WO2014178966A1 (enExample)

Families Citing this family (98)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10262462B2 (en) 2014-04-18 2019-04-16 Magic Leap, Inc. Systems and methods for augmented and virtual reality
US9811731B2 (en) * 2013-10-04 2017-11-07 Qualcomm Incorporated Dynamic extension of map data for object detection and tracking
US9607437B2 (en) * 2013-10-04 2017-03-28 Qualcomm Incorporated Generating augmented reality content for unknown objects
US9704295B2 (en) * 2013-11-05 2017-07-11 Microsoft Technology Licensing, Llc Construction of synthetic augmented reality environment
US10484697B2 (en) 2014-09-09 2019-11-19 Qualcomm Incorporated Simultaneous localization and mapping for video coding
US9607388B2 (en) * 2014-09-19 2017-03-28 Qualcomm Incorporated System and method of pose estimation
US9710960B2 (en) 2014-12-04 2017-07-18 Vangogh Imaging, Inc. Closed-form 3D model generation of non-rigid complex objects from incomplete and noisy scans
US10180734B2 (en) 2015-03-05 2019-01-15 Magic Leap, Inc. Systems and methods for augmented reality
US10838207B2 (en) * 2015-03-05 2020-11-17 Magic Leap, Inc. Systems and methods for augmented reality
KR102331164B1 (ko) 2015-03-05 2021-11-24 매직 립, 인코포레이티드 증강 현실을 위한 시스템들 및 방법들
US9911242B2 (en) 2015-05-14 2018-03-06 Qualcomm Incorporated Three-dimensional model generation
US10373366B2 (en) 2015-05-14 2019-08-06 Qualcomm Incorporated Three-dimensional model generation
US10304203B2 (en) 2015-05-14 2019-05-28 Qualcomm Incorporated Three-dimensional model generation
US20160358382A1 (en) * 2015-06-04 2016-12-08 Vangogh Imaging, Inc. Augmented Reality Using 3D Depth Sensor and 3D Projection
EP3115969B1 (en) * 2015-07-09 2021-01-06 Nokia Technologies Oy Mediated reality
US10388069B2 (en) * 2015-09-09 2019-08-20 Futurewei Technologies, Inc. Methods and systems for light field augmented reality/virtual reality on mobile devices
US20170083790A1 (en) 2015-09-23 2017-03-23 Behavioral Recognition Systems, Inc. Detected object tracker for a video analytics system
KR20180090355A (ko) 2015-12-04 2018-08-10 매직 립, 인코포레이티드 리로컬리제이션 시스템들 및 방법들
US10165171B2 (en) 2016-01-22 2018-12-25 Coban Technologies, Inc. Systems, apparatuses, and methods for controlling audiovisual apparatuses
KR102721084B1 (ko) * 2016-02-05 2024-10-22 매직 립, 인코포레이티드 증강 현실을 위한 시스템들 및 방법들
US10789840B2 (en) 2016-05-09 2020-09-29 Coban Technologies, Inc. Systems, apparatuses and methods for detecting driving behavior and triggering actions based on detected driving behavior
US10370102B2 (en) 2016-05-09 2019-08-06 Coban Technologies, Inc. Systems, apparatuses and methods for unmanned aerial vehicle
US10152858B2 (en) * 2016-05-09 2018-12-11 Coban Technologies, Inc. Systems, apparatuses and methods for triggering actions based on data capture and characterization
CN106023179A (zh) * 2016-05-13 2016-10-12 江苏科技大学 基于几何活动轮廓模型的sar图像海岸线提取方法
US11024088B2 (en) 2016-05-27 2021-06-01 HoloBuilder, Inc. Augmented and virtual reality
KR102471301B1 (ko) * 2016-06-24 2022-11-29 로베르트 보쉬 게엠베하 Rgb-d 카메라 기반 추적 시스템 및 그 방법
WO2018026737A1 (en) 2016-08-02 2018-02-08 Magic Leap, Inc. Fixed-distance virtual and augmented reality systems and methods
WO2018045076A1 (en) * 2016-08-30 2018-03-08 C3D Augmented Reality Solutions Ltd Systems and methods for simultaneous localization and mapping
US10380762B2 (en) 2016-10-07 2019-08-13 Vangogh Imaging, Inc. Real-time remote collaboration and virtual presence using simultaneous localization and mapping to construct a 3D model and update a scene based on sparse data
US10341568B2 (en) 2016-10-10 2019-07-02 Qualcomm Incorporated User interface to assist three dimensional scanning of objects
US10593116B2 (en) 2016-10-24 2020-03-17 Snap Inc. Augmented reality object manipulation
IL297846B2 (en) * 2016-11-15 2023-12-01 Magic Leap Inc A deep learning system for discovering a cube
US10210662B2 (en) 2016-12-09 2019-02-19 Fyusion, Inc. Live augmented reality using tracking
US10665024B2 (en) 2016-12-12 2020-05-26 Fyusion, Inc. Providing recording guidance in generating a multi-view interactive digital media representation
US9996945B1 (en) * 2016-12-12 2018-06-12 Fyusion, Inc. Live augmented reality guides
US10242503B2 (en) 2017-01-09 2019-03-26 Snap Inc. Surface aware lens
US20180210628A1 (en) 2017-01-23 2018-07-26 Snap Inc. Three-dimensional interaction system
US10812936B2 (en) 2017-01-23 2020-10-20 Magic Leap, Inc. Localization determination for mixed reality systems
JP6756679B2 (ja) * 2017-02-07 2020-09-16 日本電信電話株式会社 決定装置、画像処理装置、決定方法及び決定プログラム
CA3054162A1 (en) * 2017-02-27 2018-08-30 Thomson Licensing Method, system and apparatus for visual effects
WO2018170490A1 (en) * 2017-03-17 2018-09-20 Magic Leap, Inc. Technique for recording augmented reality data
AU2018236457B2 (en) * 2017-03-17 2021-04-29 Magic Leap, Inc. Mixed reality system with virtual content warping and method of generating virtual content using same
IL298822B2 (en) 2017-03-17 2025-05-01 Magic Leap Inc Mixed reality system with color virtual content distortion and method for creating virtual content therewith
EP3596702A4 (en) 2017-03-17 2020-07-22 Magic Leap, Inc. MIXED REALITY SYSTEM WITH MULTI-SOURCE VIRTUAL CONTENT COMPOSITION AND ASSOCIATED VIRTUAL CONTENT GENERATION PROCESS
CN107170011B (zh) * 2017-04-24 2019-12-17 杭州艾芯智能科技有限公司 一种机器人视觉跟踪方法及系统
TWI660156B (zh) * 2017-04-28 2019-05-21 宏達國際電子股份有限公司 追蹤系統及方法
WO2018213131A1 (en) * 2017-05-18 2018-11-22 Pcms Holdings, Inc. System and method for distributing and rendering content as spherical video and 3d asset combination
WO2019032677A1 (en) * 2017-08-09 2019-02-14 Apple Inc. LOW-LATENCY INTEGRATED COLOR INCRUST IN A MIXED REALITY VISIOCASQUE
US10474900B2 (en) 2017-09-15 2019-11-12 Snap Inc. Real-time tracking-compensated image effects
KR101971791B1 (ko) * 2017-09-28 2019-04-23 한국과학기술원 가시광 통신을 이용한 실내공간의 증강현실 제공 시스템 및 그 방법
US10817983B1 (en) * 2017-09-28 2020-10-27 Apple Inc. Method and device for combining real and virtual images
EP3474185B1 (en) * 2017-10-18 2023-06-28 Dassault Systèmes Classification of 2d images according to types of 3d arrangement
EP3701355A1 (en) * 2017-10-23 2020-09-02 Koninklijke Philips N.V. Self-expanding augmented reality-based service instructions library
US10839585B2 (en) 2018-01-05 2020-11-17 Vangogh Imaging, Inc. 4D hologram: real-time remote avatar creation and animation control
JP2019152899A (ja) * 2018-02-28 2019-09-12 株式会社バンダイナムコスタジオ シミュレーションシステム及びプログラム
US11080540B2 (en) 2018-03-20 2021-08-03 Vangogh Imaging, Inc. 3D vision processing using an IP block
US10810783B2 (en) 2018-04-03 2020-10-20 Vangogh Imaging, Inc. Dynamic real-time texture alignment for 3D models
US11170224B2 (en) 2018-05-25 2021-11-09 Vangogh Imaging, Inc. Keyframe-based object scanning and tracking
EP3827299B1 (en) 2018-07-23 2025-07-09 Magic Leap, Inc. Mixed reality system with virtual content warping and method of generating virtual content using same
CN112470464B (zh) 2018-07-23 2023-11-28 奇跃公司 场顺序显示器中的场内子码时序
CN109272457B (zh) * 2018-08-09 2022-07-22 腾讯科技（深圳）有限公司 一种图像掩膜生成方法、装置及服务器
US11030813B2 (en) 2018-08-30 2021-06-08 Snap Inc. Video clip object tracking
CN111182174B (zh) * 2018-11-09 2020-12-04 珠海格力电器股份有限公司 一种扫地机器人补光的方法及装置
US11410398B2 (en) * 2018-11-21 2022-08-09 Hewlett-Packard Development Company, L.P. Augmenting live images of a scene for occlusion
US11176737B2 (en) 2018-11-27 2021-11-16 Snap Inc. Textured mesh building
KR102867793B1 (ko) * 2018-12-20 2025-10-13 스냅 인코포레이티드 가상 표면 수정
US10984575B2 (en) 2019-02-06 2021-04-20 Snap Inc. Body pose estimation
EP3948840A4 (en) * 2019-03-18 2023-07-19 Geomagical Labs, Inc. VIRTUAL INTERACTION WITH THREE-DIMENSIONAL INTERIOR ROOM IMAGING
US11232633B2 (en) 2019-05-06 2022-01-25 Vangogh Imaging, Inc. 3D object capture and object reconstruction using edge cloud computing resources
US11170552B2 (en) 2019-05-06 2021-11-09 Vangogh Imaging, Inc. Remote visualization of three-dimensional (3D) animation with synchronized voice in real-time
US11189098B2 (en) 2019-06-28 2021-11-30 Snap Inc. 3D object camera customization system
US11164384B2 (en) * 2019-07-24 2021-11-02 Houzz, Inc. Mobile device image item replacements
CN110446092B (zh) * 2019-07-25 2023-06-20 北京拉近众博科技有限公司 用于体育比赛的虚拟观众席生成方法、系统、装置和介质
US11232646B2 (en) 2019-09-06 2022-01-25 Snap Inc. Context-based virtual object rendering
CN112734797B (zh) * 2019-10-29 2024-12-10 浙江商汤科技开发有限公司 图像特征跟踪方法、装置及电子设备
US11227442B1 (en) 2019-12-19 2022-01-18 Snap Inc. 3D captions with semantic graphical elements
US11263817B1 (en) 2019-12-19 2022-03-01 Snap Inc. 3D captions with face tracking
US10976806B1 (en) 2019-12-27 2021-04-13 GE Precision Healthcare LLC Methods and systems for immersive reality in a medical environment
US11335063B2 (en) 2020-01-03 2022-05-17 Vangogh Imaging, Inc. Multiple maps for 3D object scanning and reconstruction
US12020379B2 (en) 2020-04-17 2024-06-25 Apple Inc. Virtual anchoring systems and methods for extended reality
US11783550B2 (en) 2020-09-17 2023-10-10 Apple Inc. Image composition for extended reality systems
JP7543836B2 (ja) * 2020-10-22 2024-09-03 富士フイルムビジネスイノベーション株式会社 情報処理装置及び情報処理プログラム
US11615592B2 (en) 2020-10-27 2023-03-28 Snap Inc. Side-by-side character animation from realtime 3D body motion capture
US11660022B2 (en) 2020-10-27 2023-05-30 Snap Inc. Adaptive skeletal joint smoothing
US11734894B2 (en) 2020-11-18 2023-08-22 Snap Inc. Real-time motion transfer for prosthetic limbs
US11450051B2 (en) 2020-11-18 2022-09-20 Snap Inc. Personalized avatar real-time motion capture
US11748931B2 (en) 2020-11-18 2023-09-05 Snap Inc. Body animation sharing and remixing
JP7372230B2 (ja) * 2020-12-21 2023-10-31 トヨタ自動車株式会社 表示システム及び表示装置
US20230260240A1 (en) * 2021-03-11 2023-08-17 Quintar, Inc. Alignment of 3d graphics extending beyond frame in augmented reality system with remote presentation
US11715495B2 (en) * 2021-05-26 2023-08-01 Flawless Holdings Limited Modification of objects in film
US11398255B1 (en) 2021-05-26 2022-07-26 Flawless Holdings Limited Modification of objects in film
US11822713B2 (en) * 2021-06-11 2023-11-21 Rockwell Collins, Inc. Pose augmentation of head worn display video for latency reduction
US11568552B2 (en) * 2021-06-28 2023-01-31 Varjo Technologies Oy Imaging systems and methods incorporating improved culling of virtual objects
US12223672B2 (en) 2021-12-21 2025-02-11 Snap Inc. Real-time garment exchange
US12198398B2 (en) 2021-12-21 2025-01-14 Snap Inc. Real-time motion and appearance transfer
US11880947B2 (en) 2021-12-21 2024-01-23 Snap Inc. Real-time upper-body garment exchange
WO2025163772A1 (ja) * 2024-01-30 2025-08-07 Ｎｔｔ株式会社 映像処理装置
WO2025238724A1 (ja) * 2024-05-14 2025-11-20 Ｎｔｔ株式会社 情報提示装置、情報提示方法およびプログラム

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7168953B1 (en) * 2003-01-27 2007-01-30 Massachusetts Institute Of Technology Trainable videorealistic speech animation
US8150101B2 (en) * 2006-11-13 2012-04-03 Cybernet Systems Corporation Orientation invariant object identification using model-based image processing
US8369579B2 (en) * 2006-12-21 2013-02-05 Massachusetts Institute Of Technology Methods and apparatus for 3D surface imaging using active wave-front sampling
US8175413B1 (en) * 2009-03-05 2012-05-08 Google Inc. Video identification through detection of proprietary rights logos in media
US8768313B2 (en) * 2009-08-17 2014-07-01 Digimarc Corporation Methods and systems for image or audio recognition processing
US9230300B2 (en) 2009-10-22 2016-01-05 Tim Bekaert Method for creating a mosaic image using masks
EP2383696A1 (en) * 2010-04-30 2011-11-02 LiberoVision AG Method for estimating a pose of an articulated object model
KR20120113058A (ko) 2011-04-04 2012-10-12 한국전자통신연구원 현실-가상 융합 공간 기반의 교육 장치 및 방법
US9053571B2 (en) 2011-06-06 2015-06-09 Microsoft Corporation Generating computer models of 3D objects
US9094606B2 (en) 2011-07-04 2015-07-28 Waikatolink Limited Motion compensation in range imaging
US9255813B2 (en) 2011-10-14 2016-02-09 Microsoft Technology Licensing, Llc User controlled real object disappearance in a mixed reality display

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
Full-frame Video Stabilization;Yasuyuki Matsushita等;《PROCEEDINGS/2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION,CVPR 2005：[20-25 JUNE 2005，SAN DIEGO,CA],IEEE,PISCATAWAY,NJ,USA》;20050620;第1卷;全文 *
Multiview Paraperspective Projection Model for Diminished Reality;ZOKAI S等;《PROCEEDINGS/THE SECOND IEEE AND ACM INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY:7-10 OCTOBER 2003,TOKYO,JAPAN,IEEE COMPUTER SOCIETY,LOS ALAMITOS,CALIF.[U.A.]》;20031007;全文 *
Using Photographs to Enhance Videos of a Static Scene;PRAVIN BAT等;《RENDERING TECHNIQUES 2007:18TH EUROGRAPHICS SYMPOSIUMON RENDERING》;20070627;全文 *
基于SLAM的虚实配准方法研究;邹承明等;《计算机与数字工程》;20100131;第38卷(第1期);全文 *
移动增强现实中的跟踪技术研究;谭雪婷;《中国优秀硕士学位论文全文数据库 信息科技辑》;20101215;第2010年卷(第12期);全文 *

Also Published As

Publication number Publication date
WO2014178966A1 (en) 2014-11-06
EP2992508B1 (en) 2019-01-16
JP2016522485A (ja) 2016-07-28
CN105164728A (zh) 2015-12-16
EP2992508A1 (en) 2016-03-09
US20140321702A1 (en) 2014-10-30
US9269003B2 (en) 2016-02-23

Similar Documents

Publication Publication Date Title
CN105164728B (zh) 用于混合实境的设备和方法
US9872010B2 (en) Lidar stereo fusion live action 3D model video reconstruction for six degrees of freedom 360° volumetric virtual reality video
US11050994B2 (en) Virtual reality parallax correction
US9237330B2 (en) Forming a stereoscopic video
TWI583176B (zh) 具有功率有效深度感測器運用之即時三維重建
US20130095920A1 (en) Generating free viewpoint video using stereo imaging
US9041819B2 (en) Method for stabilizing a digital video
US20130127988A1 (en) Modifying the viewpoint of a digital image
US8611642B2 (en) Forming a steroscopic image using range map
JP2021535466A (ja) シーンの色及び深度の情報を再構成するための方法及びシステム
EP3646244A1 (en) Method and system for performing simultaneous localization and mapping using convolutional image transformation
US20130129192A1 (en) Range map determination for a video frame
WO2015142446A1 (en) Augmented reality lighting with dynamic geometry
JP2019534511A (ja) 自由動作ｆｖｖアプリケーションのためのマルチレイヤｕｖマップに基づくテクスチャレンダリング
CN115409949A (zh) 模型训练方法、视角图像生成方法、装置、设备及介质
WO2024032165A1 (zh) 3d模型生成方法、系统及电子设备
WO2024250160A1 (en) System and method for depth completion and three-dimensional reconstruction of an image area
KR20120118462A (ko) 이미지 기반의 비주얼 헐에서의 오목 표면 모델링
HK1182248B (en) Generating free viewpoint video using stereo imaging

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant