CA3062541C - Contextual applications in a mixed reality environment - Google Patents

Contextual applications in a mixed reality environment Download PDF

Info

Publication number
CA3062541C
CA3062541C CA3062541A CA3062541A CA3062541C CA 3062541 C CA3062541 C CA 3062541C CA 3062541 A CA3062541 A CA 3062541A CA 3062541 A CA3062541 A CA 3062541A CA 3062541 C CA3062541 C CA 3062541C
Authority
CA
Canada
Prior art keywords
contextual
application
user
real
environment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CA3062541A
Other languages
English (en)
French (fr)
Other versions
CA3062541A1 (en
Inventor
Sylvio Herve Drouin
Gregory Lionel Xavier Jean Palmaro
Dioselin Alejandra Gonzalez ROSILLO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Unity IPR ApS
Original Assignee
Unity IPR ApS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Unity IPR ApS filed Critical Unity IPR ApS
Publication of CA3062541A1 publication Critical patent/CA3062541A1/en
Application granted granted Critical
Publication of CA3062541C publication Critical patent/CA3062541C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Image Analysis (AREA)
CA3062541A 2017-05-05 2018-05-04 Contextual applications in a mixed reality environment Active CA3062541C (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201762502349P 2017-05-05 2017-05-05
US62/502,349 2017-05-05
US201762561017P 2017-09-20 2017-09-20
US62/561,017 2017-09-20
PCT/US2018/031245 WO2018204879A1 (en) 2017-05-05 2018-05-04 Contextual applications in a mixed reality environment

Publications (2)

Publication Number Publication Date
CA3062541A1 CA3062541A1 (en) 2018-11-08
CA3062541C true CA3062541C (en) 2022-01-11

Family

ID=64014833

Family Applications (1)

Application Number Title Priority Date Filing Date
CA3062541A Active CA3062541C (en) 2017-05-05 2018-05-04 Contextual applications in a mixed reality environment

Country Status (7)

Country Link
US (3) US10360734B2 (enExample)
EP (1) EP3639261B1 (enExample)
JP (1) JP6828189B2 (enExample)
KR (1) KR102220949B1 (enExample)
CN (1) CN111033605A (enExample)
CA (1) CA3062541C (enExample)
WO (1) WO2018204879A1 (enExample)

Families Citing this family (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104740869B (zh) * 2015-03-26 2018-04-03 北京小小牛创意科技有限公司 一种融合真实环境的虚实结合的交互方法及系统
JP2018005091A (ja) * 2016-07-06 2018-01-11 富士通株式会社 表示制御プログラム、表示制御方法および表示制御装置
US10867445B1 (en) * 2016-11-16 2020-12-15 Amazon Technologies, Inc. Content segmentation and navigation
JP6828189B2 (ja) 2017-05-05 2021-02-10 ユニティ アイピーアール エイピーエスＵｎｉｔｙ Ｉｐｒ Ａｐｓ 混合現実環境でのコンテキストアプリケーション
US10559133B2 (en) * 2018-02-07 2020-02-11 Dell Products L.P. Visual space management across information handling system and augmented reality
CN108769814B (zh) * 2018-06-01 2022-02-01 腾讯科技（深圳）有限公司 视频互动方法、装置、终端及可读存储介质
WO2019237085A1 (en) 2018-06-08 2019-12-12 Vulcan Inc. Session-based information exchange
US11195336B2 (en) 2018-06-08 2021-12-07 Vulcan Inc. Framework for augmented reality applications
US20190385372A1 (en) * 2018-06-15 2019-12-19 Microsoft Technology Licensing, Llc Positioning a virtual reality passthrough region at a known distance
US10712901B2 (en) 2018-06-27 2020-07-14 Facebook Technologies, Llc Gesture-based content sharing in artificial reality environments
US10783712B2 (en) * 2018-06-27 2020-09-22 Facebook Technologies, Llc Visual flairs for emphasizing gestures in artificial-reality environments
US10635895B2 (en) 2018-06-27 2020-04-28 Facebook Technologies, Llc Gesture-based casting and manipulation of virtual content in artificial-reality environments
US10996831B2 (en) 2018-06-29 2021-05-04 Vulcan Inc. Augmented reality cursors
WO2020152828A1 (ja) * 2019-01-24 2020-07-30 マクセル株式会社 表示端末、アプリケーション制御システムおよびアプリケーション制御方法
US11054896B1 (en) * 2019-02-07 2021-07-06 Facebook, Inc. Displaying virtual interaction objects to a user on a reference plane
US11836978B2 (en) * 2019-03-08 2023-12-05 Ntt Docomo, Inc. Related information output device
DE102019106281A1 (de) * 2019-03-12 2020-09-17 tooz technologies GmbH System, Verfahren, Computerprogramm, mobile Vorrichtung und Kit zum Betreiben einer Vorrichtung mit niedriger Rechenleistung
US10810782B1 (en) * 2019-04-01 2020-10-20 Snap Inc. Semantic texture mapping system
US11804014B1 (en) * 2019-04-22 2023-10-31 Apple Inc. Context-based application placement
US11163997B2 (en) * 2019-05-05 2021-11-02 Google Llc Methods and apparatus for venue based augmented reality
US10897564B1 (en) 2019-06-17 2021-01-19 Snap Inc. Shared control of camera device by multiple devices
WO2020261028A1 (ja) * 2019-06-25 2020-12-30 株式会社半導体エネルギー研究所 情報処理システム、および情報処理方法
US11100715B2 (en) * 2019-06-26 2021-08-24 International Business Machines Corporation Establishment of positional timers in an augmented reality environment
US11340857B1 (en) 2019-07-19 2022-05-24 Snap Inc. Shared control of a virtual object by multiple devices
US11176744B2 (en) * 2019-07-22 2021-11-16 Microsoft Technology Licensing, Llc Mapping sensor data using a mixed-reality cloud
US11461867B2 (en) * 2019-09-19 2022-10-04 Sanctuary Cognitive Systems Corporation Visual interface and communications techniques for use with robots
CN110716645A (zh) * 2019-10-15 2020-01-21 北京市商汤科技开发有限公司 一种增强现实数据呈现方法、装置、电子设备及存储介质
US11270115B2 (en) * 2019-11-18 2022-03-08 Lenovo (Singapore) Pte. Ltd. Presentation of augmented reality content based on identification of trigger accompanying video content
US11461115B2 (en) * 2020-01-28 2022-10-04 Sap Se Third party executable asset bundle deployment
US11354867B2 (en) * 2020-03-04 2022-06-07 Apple Inc. Environment application model
US11985175B2 (en) 2020-03-25 2024-05-14 Snap Inc. Virtual interaction session to facilitate time limited augmented reality based communication between multiple users
US12101360B2 (en) 2020-03-25 2024-09-24 Snap Inc. Virtual interaction session to facilitate augmented reality based communication between multiple users
US12182903B2 (en) 2020-03-25 2024-12-31 Snap Inc. Augmented reality based communication between multiple users
US11593997B2 (en) * 2020-03-31 2023-02-28 Snap Inc. Context based augmented reality communication
US11804039B2 (en) * 2020-05-28 2023-10-31 Science House LLC Systems, methods, and apparatus for enhanced cameras
CN111462663B (zh) * 2020-06-19 2020-10-27 南京新研协同定位导航研究院有限公司 一种基于mr眼镜的导游方法
CN111539662A (zh) * 2020-07-10 2020-08-14 南京新研协同定位导航研究院有限公司 基于mr眼镜租赁业务的动态库存分配方法
EP3964929A1 (en) * 2020-09-08 2022-03-09 Koninklijke Philips N.V. Controlling a 2d screen interface application in a mixed reality application
KR20230062857A (ko) 2020-09-09 2023-05-09 스냅 인코포레이티드 증강 현실 메신저 시스템
EP4637108A3 (en) 2020-09-16 2025-12-17 Snap Inc. Augmented reality auto reactions
CN116171566B (zh) * 2020-09-16 2025-05-09 斯纳普公司 上下文触发的增强现实
US12439145B1 (en) 2020-09-25 2025-10-07 Apple Inc. Determining an intended gesture when hands are occupied
US11617953B2 (en) 2020-10-09 2023-04-04 Contact Control Interfaces, Llc. Virtual object interaction scripts
US11676355B2 (en) 2020-11-04 2023-06-13 Unity IPR ApS Method and system for merging distant spaces
JP6975489B1 (ja) * 2020-12-18 2021-12-01 株式会社Gatari 情報処理システム、情報処理方法および情報処理プログラム
KR20220121990A (ko) * 2021-02-26 2022-09-02 삼성전자주식회사 증강 현실 장치 및 증강 현실 장치와 상호작용하는 전자 장치
US12056416B2 (en) 2021-02-26 2024-08-06 Samsung Electronics Co., Ltd. Augmented reality device and electronic device interacting with augmented reality device
US20240152948A1 (en) * 2021-03-12 2024-05-09 Christopher Faraguna Systems and methods for user data collection within an augmented reality game
US20220319124A1 (en) * 2021-03-31 2022-10-06 Snap Inc. Auto-filling virtual content
US11935199B2 (en) * 2021-07-26 2024-03-19 Google Llc Augmented reality depth detection through object recognition
WO2023034009A1 (en) 2021-09-01 2023-03-09 Snap Inc. Handcrafted augmented reality experiences
WO2023034012A1 (en) 2021-09-01 2023-03-09 Snap Inc. Handcrafted augmented reality effort evidence
WO2023034010A1 (en) 2021-09-01 2023-03-09 Snap Inc. Physical action-based augmented reality communication exchanges
KR20240050437A (ko) * 2021-09-02 2024-04-18 스냅 인코포레이티드 증강 현실 프로프 상호작용
US11969650B2 (en) * 2021-11-18 2024-04-30 Sony Interactive Entertainment Inc. Feature similarity scoring of physical environment for augmented reality gameplay
US11568616B1 (en) * 2021-11-29 2023-01-31 Varjo Technologies Oy Display apparatuses and methods for facilitating location-based virtual content
CN114971764A (zh) * 2022-04-08 2022-08-30 浙江赟燊商业信息系统科技有限公司 一种基于HoloLens的存储和匹配系统及方法
JP7482943B2 (ja) * 2022-06-17 2024-05-14 ソフトバンク株式会社 情報処理装置、プログラム、システム、及び情報処理方法
US12020388B2 (en) * 2022-07-29 2024-06-25 Maplebear Inc. Displaying an augmented reality element that provides a personalized enhanced experience at a warehouse
US12299827B2 (en) 2022-10-17 2025-05-13 T-Mobile Usa, Inc. Generating mixed reality content based on a location of a wireless device
US12450829B2 (en) * 2022-12-29 2025-10-21 Skonec Entertainment Co., Ltd. Virtual reality control system
WO2024218003A1 (en) * 2023-04-17 2024-10-24 Interdigital Ce Patent Holdings, Sas Graph representation of events for interactive environments
US20250036371A1 (en) * 2023-07-25 2025-01-30 simpleAR, Inc. Serializing and deserializing mixed reality experiences or portions thereof
US20250111563A1 (en) * 2023-09-29 2025-04-03 PlayFusion Limited Video playback recognition apparatus and method
US12437487B2 (en) * 2023-10-03 2025-10-07 Adeia Guides Inc. Systems and methods for virtually displaying content items on real-world objects in an extended reality environment
US20250131632A1 (en) * 2023-10-19 2025-04-24 Niantic, Inc Llm-based animation selection for an ar character
GB2638148A (en) * 2024-02-13 2025-08-20 Elmon Jacob System and method of landmark triggred mobile AR advertising in taxi rides
US12399600B1 (en) * 2024-02-23 2025-08-26 Google Llc Contact space determination for user interface control

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9285589B2 (en) * 2010-02-28 2016-03-15 Microsoft Technology Licensing, Llc AR glasses with event and sensor triggered control of AR eyepiece applications
US20110213664A1 (en) * 2010-02-28 2011-09-01 Osterhout Group, Inc. Local advertising content on an interactive head-mounted eyepiece
JP2013004001A (ja) * 2011-06-21 2013-01-07 Sony Corp 表示制御装置、表示制御方法、およびプログラム
US10019962B2 (en) * 2011-08-17 2018-07-10 Microsoft Technology Licensing, Llc Context adaptive user interface for augmented reality display
KR101874895B1 (ko) * 2012-01-12 2018-07-06 삼성전자 주식회사 증강 현실 제공 방법 및 이를 지원하는 단말기
GB2501929B (en) * 2012-05-11 2015-06-24 Sony Comp Entertainment Europe Apparatus and method for augmented reality
KR102062763B1 (ko) * 2012-12-07 2020-01-07 삼성전자주식회사 상황 정보 기반의 정보 제공 방법 및 시스템과 그 기록 매체
US9924102B2 (en) * 2013-03-14 2018-03-20 Qualcomm Incorporated Image-based application launcher
US9240075B2 (en) * 2013-03-15 2016-01-19 Daqri, Llc Campaign optimization for experience content dataset
CN203250508U (zh) * 2013-04-24 2013-10-23 苏州创捷传媒展览股份有限公司 3d混合现实展示装置
US10025375B2 (en) * 2015-10-01 2018-07-17 Disney Enterprises, Inc. Augmented reality controls for user interactions with a virtual world
US9536355B1 (en) * 2016-03-24 2017-01-03 Daqri, Llc Thermal detection in an augmented reality system
CN105912130A (zh) * 2016-07-06 2016-08-31 深圳云戈科技有限公司 一种基于混合现实的通信方法、装置及系统
JP6828189B2 (ja) 2017-05-05 2021-02-10 ユニティ アイピーアール エイピーエスＵｎｉｔｙ Ｉｐｒ Ａｐｓ 混合現実環境でのコンテキストアプリケーション

Also Published As

Publication number Publication date
KR20190141777A (ko) 2019-12-24
JP6828189B2 (ja) 2021-02-10
JP2020520000A (ja) 2020-07-02
US20180322706A1 (en) 2018-11-08
US20200357187A1 (en) 2020-11-12
US10769857B2 (en) 2020-09-08
US10360734B2 (en) 2019-07-23
CA3062541A1 (en) 2018-11-08
US20190392645A1 (en) 2019-12-26
EP3639261A4 (en) 2020-05-13
EP3639261B1 (en) 2023-08-30
US10984604B2 (en) 2021-04-20
EP3639261A1 (en) 2020-04-22
CN111033605A (zh) 2020-04-17
KR102220949B1 (ko) 2021-03-02
WO2018204879A1 (en) 2018-11-08

Similar Documents

Publication Publication Date Title
US10984604B2 (en) Contextual applications in a mixed reality environment
US12231804B2 (en) Selective identification and order of image modifiers
US10228836B2 (en) System and method for generation of 3D virtual objects
US9928662B2 (en) System and method for temporal manipulation in virtual environments
US10001901B2 (en) System and method for texturing in virtual reality and mixed reality environments
US11676355B2 (en) Method and system for merging distant spaces
US12288298B2 (en) Generating user interfaces displaying augmented reality graphics
CN119301544A (zh) 基于ar的虚拟键盘
CN119137576A (zh) 基于姿势的键盘文本输入
CN118974687A (zh) 基于姿势的应用调用
US12197641B2 (en) Systems and methods for dynamic continuous input in mixed reality environments

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20191105

EEER Examination request

Effective date: 20191105