CN114026606B - 用于动态遮挡的快速手部网格划分 - Google Patents

用于动态遮挡的快速手部网格划分 Download PDF

Info

Publication number
CN114026606B
CN114026606B CN202080046371.7A CN202080046371A CN114026606B CN 114026606 B CN114026606 B CN 114026606B CN 202080046371 A CN202080046371 A CN 202080046371A CN 114026606 B CN114026606 B CN 114026606B
Authority
CN
China
Prior art keywords
hand
scene
user
information
depth
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080046371.7A
Other languages
English (en)
Chinese (zh)
Other versions
CN114026606A (zh
Inventor
D·G·莫利纳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of CN114026606A publication Critical patent/CN114026606A/zh
Application granted granted Critical
Publication of CN114026606B publication Critical patent/CN114026606B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/10—Geometric effects
    • G06T15/20—Perspective computation
    • G06T15/205—Image-based rendering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/107—Static hand or arm
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2215/00—Indexing scheme for image rendering
    • G06T2215/16—Using real world measurements to influence rendering

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Social Psychology (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Computer Hardware Design (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Generation (AREA)
CN202080046371.7A 2019-06-28 2020-06-25 用于动态遮挡的快速手部网格划分 Active CN114026606B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962868640P 2019-06-28 2019-06-28
US62/868,640 2019-06-28
PCT/US2020/039601 WO2020264149A1 (en) 2019-06-28 2020-06-25 Fast hand meshing for dynamic occlusion

Publications (2)

Publication Number Publication Date
CN114026606A CN114026606A (zh) 2022-02-08
CN114026606B true CN114026606B (zh) 2025-06-17

Family

ID=74042905

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080046371.7A Active CN114026606B (zh) 2019-06-28 2020-06-25 用于动态遮挡的快速手部网格划分

Country Status (5)

Country Link
US (2) US11238651B2 (https=)
EP (1) EP3991142B1 (https=)
JP (2) JP7711002B2 (https=)
CN (1) CN114026606B (https=)
WO (1) WO2020264149A1 (https=)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11710310B2 (en) * 2019-06-19 2023-07-25 Apple Inc. Virtual content positioned based on detected object
JP7711002B2 (ja) 2019-06-28 2025-07-22 マジック リープ， インコーポレイテッド 動的オクルージョンのための高速の手のメッシュ化
US11373339B2 (en) * 2020-03-18 2022-06-28 Sony Group Corporation Projection-based mesh compression
US11836871B2 (en) * 2021-03-22 2023-12-05 Apple Inc. Indicating a position of an occluded physical object
GB2613840B (en) * 2021-12-16 2025-03-12 Sony Interactive Entertainment Europe Ltd Feature tracking system and method
JP7740048B2 (ja) * 2022-02-14 2025-09-17 トヨタ自動車株式会社 システム、情報処理装置、及び方法
US12154231B2 (en) * 2022-07-20 2024-11-26 Qualcomm Incorporated Late stage occlusion based rendering for extended reality (XR)
CN115439171A (zh) * 2022-08-04 2022-12-06 阿里巴巴（中国）有限公司 商品信息展示方法、装置及电子设备
WO2024191175A1 (en) * 2023-03-15 2024-09-19 Samsung Electronics Co., Ltd. Method and electronic device for estimating a landmark point of body part of subject
US20250148701A1 (en) * 2023-11-06 2025-05-08 Samsung Electronics Co., Ltd. Dynamic overlapping of moving objects with real and virtual scenes for video see-through (vst) extended reality (xr)
US20250173978A1 (en) * 2023-11-27 2025-05-29 Meta Platforms Technologies, Llc Real-time Rendering Of Animated Objects

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103472909A (zh) * 2012-04-10 2013-12-25 微软公司 用于头戴式、增强现实显示器的逼真遮挡
CN109471521A (zh) * 2018-09-05 2019-03-15 华东计算技术研究所（中国电子科技集团公司第三十二研究所） Ar环境下的虚实遮挡交互方法及系统

Family Cites Families (40)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5886702A (en) * 1996-10-16 1999-03-23 Real-Time Geometry Corporation System and method for computer modeling of 3D objects or surfaces by mesh constructions having optimal quality characteristics and dynamic resolution capabilities
US6879324B1 (en) * 1998-07-14 2005-04-12 Microsoft Corporation Regional progressive meshes
JP4745724B2 (ja) * 2005-06-08 2011-08-10 キヤノン株式会社 画像処理方法、画像処理装置
US7961186B2 (en) * 2005-09-26 2011-06-14 Siemens Medical Solutions Usa, Inc. Brick-based fusion renderer
FR2897455A1 (fr) * 2006-02-13 2007-08-17 Univ Hokkaido Nat Univ Corp Dispositif, procede et programme de segmentation de donnees de modele en treillis
US8774512B2 (en) * 2009-02-11 2014-07-08 Thomson Licensing Filling holes in depth maps
US8933925B2 (en) * 2009-06-15 2015-01-13 Microsoft Corporation Piecewise planar reconstruction of three-dimensional scenes
US9110194B2 (en) * 2009-09-14 2015-08-18 Exxonmobil Upstream Research Company System and method for providing a time-based representation of data
KR101697184B1 (ko) 2010-04-20 2017-01-17 삼성전자주식회사 메쉬 생성 장치 및 그 방법, 그리고, 영상 처리 장치 및 그 방법
US20120154400A1 (en) * 2010-12-20 2012-06-21 General Electric Company Method of reducing noise in a volume-rendered image
US8587583B2 (en) * 2011-01-31 2013-11-19 Microsoft Corporation Three-dimensional environment reconstruction
JP5784818B2 (ja) * 2011-03-29 2015-09-24 クアルコム，インコーポレイテッド 拡張現実システムにおける実世界表面への仮想画像のアンカリング
US8840466B2 (en) * 2011-04-25 2014-09-23 Aquifi, Inc. Method and system to create three-dimensional mapping in a two-dimensional game
US8879828B2 (en) * 2011-06-29 2014-11-04 Matterport, Inc. Capturing and aligning multiple 3-dimensional scenes
US9255813B2 (en) * 2011-10-14 2016-02-09 Microsoft Technology Licensing, Llc User controlled real object disappearance in a mixed reality display
US8638989B2 (en) * 2012-01-17 2014-01-28 Leap Motion, Inc. Systems and methods for capturing motion in three-dimensional space
US9477303B2 (en) * 2012-04-09 2016-10-25 Intel Corporation System and method for combining three-dimensional tracking with a three-dimensional display for a user interface
US9645394B2 (en) * 2012-06-25 2017-05-09 Microsoft Technology Licensing, Llc Configured virtual environments
WO2014128749A1 (ja) * 2013-02-19 2014-08-28 株式会社ブリリアントサービス 形状認識装置、形状認識プログラム、および形状認識方法
US20150024337A1 (en) * 2013-07-18 2015-01-22 A.Tron3D Gmbh Voxel level new information updates using intelligent weighting
US9996974B2 (en) * 2013-08-30 2018-06-12 Qualcomm Incorporated Method and apparatus for representing a physical scene
KR102184766B1 (ko) * 2013-10-17 2020-11-30 삼성전자주식회사 3차원 모델 복원 시스템 및 방법
US10203762B2 (en) * 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
US10852838B2 (en) * 2014-06-14 2020-12-01 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
AU2015274283B2 (en) * 2014-06-14 2020-09-10 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
US9865089B2 (en) * 2014-07-25 2018-01-09 Microsoft Technology Licensing, Llc Virtual reality environment with real world objects
WO2016077798A1 (en) * 2014-11-16 2016-05-19 Eonite Perception Inc. Systems and methods for augmented reality preparation, processing, and application
JP6357412B2 (ja) * 2014-12-15 2018-07-11 キヤノンマーケティングジャパン株式会社 情報処理装置、情報処理システム、情報処理方法、及びプログラム
KR102317247B1 (ko) * 2015-06-15 2021-10-26 한국전자통신연구원 영상정보를 이용한 증강현실 기반 손 인터랙션 장치 및 방법
US20170185141A1 (en) * 2015-12-29 2017-06-29 Microsoft Technology Licensing, Llc Hand tracking for interaction feedback
US10565791B2 (en) * 2015-12-29 2020-02-18 Microsoft Technology Licensing, Llc Tracking rigged polygon-mesh models of articulated objects
US10665019B2 (en) * 2016-03-24 2020-05-26 Qualcomm Incorporated Spatial relationships for integration of visual images of physical environment into virtual reality
JP6857980B2 (ja) * 2016-08-02 2021-04-14 キヤノン株式会社 情報処理装置、情報処理装置の制御方法およびプログラム
CN106373085A (zh) * 2016-09-20 2017-02-01 福州大学 一种基于增强现实的智能终端3d手表试戴方法及系统
US10297085B2 (en) * 2016-09-28 2019-05-21 Intel Corporation Augmented reality creations with interactive behavior and modality assignments
US10803616B1 (en) * 2017-04-13 2020-10-13 Facebook Technologies, Llc Hand calibration using single depth camera
CN114332332B (zh) * 2017-12-22 2023-08-18 奇跃公司 用于生成场景中的表面的三维重建的方法和设备
US10929654B2 (en) 2018-03-12 2021-02-23 Nvidia Corporation Three-dimensional (3D) pose estimation from a monocular camera
US11775836B2 (en) 2019-05-21 2023-10-03 Magic Leap, Inc. Hand pose estimation
JP7711002B2 (ja) 2019-06-28 2025-07-22 マジック リープ， インコーポレイテッド 動的オクルージョンのための高速の手のメッシュ化

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103472909A (zh) * 2012-04-10 2013-12-25 微软公司 用于头戴式、增强现实显示器的逼真遮挡
CN109471521A (zh) * 2018-09-05 2019-03-15 华东计算技术研究所（中国电子科技集团公司第三十二研究所） Ar环境下的虚实遮挡交互方法及系统

Also Published As

Publication number Publication date
WO2020264149A1 (en) 2020-12-30
EP3991142A4 (en) 2023-06-28
JP2022537817A (ja) 2022-08-30
US20220114788A1 (en) 2022-04-14
EP3991142A1 (en) 2022-05-04
US20200410753A1 (en) 2020-12-31
CN114026606A (zh) 2022-02-08
JP7711002B2 (ja) 2025-07-22
JP2025107314A (ja) 2025-07-17
US11238651B2 (en) 2022-02-01
EP3991142B1 (en) 2026-04-08
US11620792B2 (en) 2023-04-04

Similar Documents

Publication Publication Date Title
CN114026606B (zh) 用于动态遮挡的快速手部网格划分
CN114245908B (zh) 用深度信息快速3d重建
US12100098B2 (en) Simple environment solver using planar extraction
US11587298B2 (en) Caching and updating of dense 3D reconstruction data
US10902679B2 (en) Method of occlusion rendering using raycast and live depth
CN119205860A (zh) 跨现实系统
US12444136B2 (en) Scene understanding using occupancy grids

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant