RU2017102204A - Препятствование спотыканию при погружении в среду виртуальной реальности - Google Patents

Препятствование спотыканию при погружении в среду виртуальной реальности Download PDF

Info

Publication number
RU2017102204A
RU2017102204A RU2017102204A RU2017102204A RU2017102204A RU 2017102204 A RU2017102204 A RU 2017102204A RU 2017102204 A RU2017102204 A RU 2017102204A RU 2017102204 A RU2017102204 A RU 2017102204A RU 2017102204 A RU2017102204 A RU 2017102204A
Authority
RU
Russia
Prior art keywords
user
hmd device
world
real
display device
Prior art date
Application number
RU2017102204A
Other languages
English (en)
Inventor
ВЬЕГА Александр ДЕ
Бенджамин САГДЕН
Лаура МАССИ
Original Assignee
МАЙКРОСОФТ ТЕКНОЛОДЖИ ЛАЙСЕНСИНГ, ЭлЭлСи
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by МАЙКРОСОФТ ТЕКНОЛОДЖИ ЛАЙСЕНСИНГ, ЭлЭлСи filed Critical МАЙКРОСОФТ ТЕКНОЛОДЖИ ЛАЙСЕНСИНГ, ЭлЭлСи
Publication of RU2017102204A publication Critical patent/RU2017102204A/ru

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B27/0172Head mounted characterised by optical features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012Head tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/08Volume rendering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/20Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20Image signal generators
    • H04N13/204Image signal generators using stereoscopic image cameras
    • H04N13/239Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/332Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/332Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • H04N13/344Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/0138Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0101Head-up displays characterised by optical features
    • G02B2027/014Head-up displays characterised by optical features comprising information/image processing systems
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B2027/0178Eyeglass type
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0179Display position adjusting means not related to the information to be displayed
    • G02B2027/0187Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074Stereoscopic image analysis
    • H04N2013/0081Depth or disparity estimation from stereoscopic image signals

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Optics & Photonics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Architecture (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Processing Or Creating Images (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)

Claims (25)

1. Способ, выполняемый закрепляемым на голове устройством отображения (HMD-устройством), поддерживающим визуализацию среды виртуальной реальности, содержащий этапы, на которых:
получают данные датчиков, описывающие физическое пространство, прилегающее к пользователю HMD-устройства;
используя данные датчиков, восстанавливают геометрию предметов реального мира, расположенных в физическом пространстве;
определяют расстояние между пользователем и предметами реального мира с использованием восстановленной геометрии; и
если расстояние равно или меньше предварительно определенной пороговой величины, то задействуют HMD-устройство так, чтобы пользователь мог видеть предметы реального мира беспрепятственно от среды виртуальной реальности.
2. Способ по п.1, в котором данные датчиков включают в себя данные глубины, и способ дополнительно включает в себя этап, на котором генерируют данные датчиков с использованием датчика глубины и применяют методики восстановления поверхности для восстановления геометрии предметов реального мира.
3. Способ по п.1, дополнительно включающий в себя этап, на котором генерируют данные глубины с использованием одного или более анализов глубины из формирования стереоизображений.
4. Способ по п.1, дополнительно включающий в себя этап, на котором делают тусклым, по меньшей мере, участок устройства отображения сквозного видения, встроенного в HMD-устройство, в течение упомянутого задействования.
5. Способ по п.1, дополнительно включающий в себя этап, на котором предписывают, по меньшей мере, участку устройства отображения сквозного видения, встроенного в HMD-устройство, становиться, по существу, прозрачным.
6. Способ по п.1, дополнительно включающий в себя этап, на котором представляют пользователя с помощью объемной формы и используют эту объемную форму при определении расстояния между пользователем и предметами реального мира.
7. Способ по п.6, дополнительно включающий в себя этап, на котором задают размеры объемной формы согласно высоте головы пользователя над основанием физического пространства.
8. Способ по п.7, дополнительно включающий в себя этап, на котором изменяют упомянутую предварительно определенную пороговую величину согласно размеру, форме или другой характеристике объемной формы.
9. Способ по п.8, дополнительно включающий в себя этап, на котором конфигурируют упомянутую предварительно определенную пороговую величину, чтобы она была относительно малой, когда пользователь усажен, по сравнению с тем, когда пользователь стоит.
10. Способ по п.1, дополнительно включающий в себя этап, на котором используют данные датчиков для определения текущей высоты пользователя.
11. Закрепляемое на голове устройство отображения (HMD-устройство), задействуемое пользователем в физической среде, содержащее:
один или более процессоров;
устройство отображения сквозного видения, использующее добавочное смешивание между элементами отображения реального мира и виртуального мира; и
одно или более запоминающих устройств, хранящих считываемые компьютером команды, которыми при их исполнении одним или более процессорами выполняется способ, содержащий этапы:
генерирования данных восстановления поверхности с использованием системы камер или датчиков глубины,
определения, используя данные восстановления поверхности, расстояния до предмета реального мира в физической среде от объемной формы, которая представляет пользователя, и
задействования HMD-устройства так, чтобы, по меньшей мере, участок устройства отображения сквозного видения был виден насквозь пользователем, когда предмет реального мира находится внутри порогового расстояния от объемной формы.
12. HMD-устройство по п.11, дополнительно включающее в себя задействование устройства отображения сквозного видения посредством визуализации темных пикселей для отображения виртуального мира так, чтобы, по меньшей мере, участок виртуального мира не загораживал отображение предмета реального мира.
13. HMD-устройство по п.11, дополнительно включающее в себя датчик глубины, выполненный с возможностью сбора данных восстановления поверхности.
14. HMD-устройство по п.11, дополнительно включающее в себя изменение упомянутого порогового расстояния в зависимости от определения того, стоит ли пользователь или сидит.
15. HMD-устройство по п.11, дополнительно включающее в себя моделирование физической среды с использованием конвейерной обработки данных восстановления поверхности, которая реализует объемный способ, создающий множество перекрывающихся поверхностей, которые объединяются.
RU2017102204A 2014-07-25 2015-07-24 Препятствование спотыканию при погружении в среду виртуальной реальности RU2017102204A (ru)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201462029351P 2014-07-25 2014-07-25
US62/029,351 2014-07-25
US14/600,774 2015-01-20
US14/600,774 US10311638B2 (en) 2014-07-25 2015-01-20 Anti-trip when immersed in a virtual reality environment
PCT/US2015/041867 WO2016014877A1 (en) 2014-07-25 2015-07-24 Anti-trip when immersed in a virtual reality environment

Publications (1)

Publication Number Publication Date
RU2017102204A true RU2017102204A (ru) 2018-07-24

Family

ID=53801186

Family Applications (1)

Application Number Title Priority Date Filing Date
RU2017102204A RU2017102204A (ru) 2014-07-25 2015-07-24 Препятствование спотыканию при погружении в среду виртуальной реальности

Country Status (11)

Country Link
US (1) US10311638B2 (ru)
EP (1) EP3172649B1 (ru)
JP (1) JP2017531221A (ru)
KR (1) KR102385756B1 (ru)
CN (1) CN106575155A (ru)
AU (1) AU2015292490A1 (ru)
BR (1) BR112017000069A2 (ru)
CA (1) CA2954727A1 (ru)
MX (1) MX2017001129A (ru)
RU (1) RU2017102204A (ru)
WO (1) WO2016014877A1 (ru)

Families Citing this family (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9392212B1 (en) 2014-04-17 2016-07-12 Visionary Vr, Inc. System and method for presenting virtual reality content to a user
US9766460B2 (en) 2014-07-25 2017-09-19 Microsoft Technology Licensing, Llc Ground plane adjustment in a virtual reality environment
US10416760B2 (en) 2014-07-25 2019-09-17 Microsoft Technology Licensing, Llc Gaze-based object placement within a virtual reality environment
US10451875B2 (en) 2014-07-25 2019-10-22 Microsoft Technology Licensing, Llc Smart transparency for virtual objects
US10389992B2 (en) * 2014-08-05 2019-08-20 Utherverse Digital Inc. Immersive display and method of operating immersive display for real-world object alert
US10943395B1 (en) * 2014-10-03 2021-03-09 Virtex Apps, Llc Dynamic integration of a virtual environment with a physical environment
US9665170B1 (en) 2015-06-10 2017-05-30 Visionary Vr, Inc. System and method for presenting virtual reality content to a user based on body posture
EP3333616A4 (en) 2015-08-04 2019-04-03 Sony Interactive Entertainment Inc. VISIOCASQUE, DISPLAY CONTROL METHOD, AND PROGRAM
US10133345B2 (en) * 2016-03-22 2018-11-20 Microsoft Technology Licensing, Llc Virtual-reality navigation
US10665019B2 (en) * 2016-03-24 2020-05-26 Qualcomm Incorporated Spatial relationships for integration of visual images of physical environment into virtual reality
US10722800B2 (en) * 2016-05-16 2020-07-28 Google Llc Co-presence handling in virtual reality
CN108633307B (zh) * 2016-05-17 2021-08-31 谷歌有限责任公司 在虚拟现实环境中投射与真实对象的接触的方法和装置
KR20170135267A (ko) 2016-05-31 2017-12-08 엘지전자 주식회사 글래스 타입의 이동 단말기
US20170372499A1 (en) * 2016-06-27 2017-12-28 Google Inc. Generating visual cues related to virtual objects in an augmented and/or virtual reality environment
CN106200947B (zh) * 2016-06-30 2020-05-26 联想(北京)有限公司 一种电子设备以及虚拟环境图像展示方法
CN106249882B (zh) * 2016-07-26 2022-07-12 华为技术有限公司 一种应用于vr设备的手势操控方法与装置
KR20180021515A (ko) 2016-08-22 2018-03-05 삼성전자주식회사 영상 표시 장치 및 영상 표시 장치의 동작 방법
US10032314B2 (en) 2016-10-11 2018-07-24 Microsoft Technology Licensing, Llc Virtual reality headset
FR3058534B1 (fr) 2016-11-09 2019-02-01 Stereolabs Dispositif individuel d'immersion visuelle pour personne en mouvement avec gestion des obstacles
CN106530620A (zh) * 2016-12-26 2017-03-22 宇龙计算机通信科技(深圳)有限公司 一种安全监测方法、装置、系统及虚拟现实设备
EP3563215A4 (en) 2016-12-29 2020-08-05 Magic Leap, Inc. AUTOMATIC CONTROL OF A WEARABLE DISPLAY DEVICE BASED ON EXTERNAL CONDITIONS
EP3367210A1 (en) 2017-02-24 2018-08-29 Thomson Licensing Method for operating a device and corresponding device, system, computer readable program product and computer readable storage medium
CN106971501B (zh) * 2017-03-09 2019-07-26 广州三星通信技术研究有限公司 用于虚拟现实设备的提醒方法和提醒装置
US10475243B2 (en) * 2017-03-31 2019-11-12 Intel Corporation Transition between virtual reality and real world
US10514801B2 (en) * 2017-06-15 2019-12-24 Microsoft Technology Licensing, Llc Hover-based user-interactions with virtual objects within immersive environments
US10627635B2 (en) * 2017-08-02 2020-04-21 Microsoft Technology Licensing, Llc Transitioning into a VR environment and warning HMD users of real-world physical obstacles
WO2019067470A1 (en) * 2017-09-29 2019-04-04 Zermatt Technologies Llc PHYSICAL LIMIT DETECTOR
DE102017218215A1 (de) 2017-10-12 2019-04-18 Audi Ag Verfahren zum Betreiben einer am Kopf tragbaren elektronischen Anzeigeeinrichtung und Anzeigesystem zum Anzeigen eines virtuellen Inhalts
EP3714438B1 (en) * 2017-11-21 2023-08-16 Volvo Truck Corporation Assistance method for assisting performance of a task on a product, comprising displaying a highlighting image highlighting a monitored part of the product
CN108156669A (zh) * 2017-12-04 2018-06-12 捷开通讯(深圳)有限公司 智能终端通信方法、智能终端及具有存储功能的装置
US11164380B2 (en) 2017-12-05 2021-11-02 Samsung Electronics Co., Ltd. System and method for transition boundaries and distance responsive interfaces in augmented and virtual reality
EP3540570A1 (en) 2018-03-13 2019-09-18 Thomson Licensing Method for generating an augmented representation of a real environment, corresponding device, computer program product, and computer-readable carrier medium
US10853946B2 (en) 2018-05-18 2020-12-01 Ebay Inc. Physical object boundary detection techniques and systems
US20190385372A1 (en) * 2018-06-15 2019-12-19 Microsoft Technology Licensing, Llc Positioning a virtual reality passthrough region at a known distance
CN112005282A (zh) * 2018-06-20 2020-11-27 惠普发展公司,有限责任合伙企业 混合现实设备的警报
WO2019245543A1 (en) * 2018-06-20 2019-12-26 Hewlett-Packard Development Company, L.P. Alerts of mixed reality devices
US11816886B1 (en) * 2018-06-28 2023-11-14 Meta Platforms Technologies, Llc Apparatus, system, and method for machine perception
WO2020023404A1 (en) * 2018-07-24 2020-01-30 Magic Leap, Inc. Flicker mitigation when toggling eyepiece display illumination in augmented reality systems
US10901081B2 (en) 2018-10-02 2021-01-26 International Business Machines Corporation Virtual reality safety
DE102018217834A1 (de) 2018-10-18 2020-04-23 Robert Bosch Gmbh Verfahren und System zum Betreiben einer Erfassungs- und Anzeigevorrichtung
US11004269B2 (en) * 2019-04-22 2021-05-11 Microsoft Technology Licensing, Llc Blending virtual environments with situated physical reality
JP7356827B2 (ja) 2019-06-26 2023-10-05 株式会社コロプラ プログラム、情報処理方法、及び情報処理装置
US11842449B2 (en) 2019-09-26 2023-12-12 Apple Inc. Presenting an environment based on user movement
US11175730B2 (en) 2019-12-06 2021-11-16 Facebook Technologies, Llc Posture-based virtual space configurations
US11126850B1 (en) 2020-04-09 2021-09-21 Facebook Technologies, Llc Systems and methods for detecting objects within the boundary of a defined space while in artificial reality
US11257280B1 (en) 2020-05-28 2022-02-22 Facebook Technologies, Llc Element-based switching of ray casting rules
US11493764B2 (en) 2020-06-04 2022-11-08 Htc Corporation Method for dynamically displaying real-world scene, electronic device, and computer readable medium
US11256336B2 (en) 2020-06-29 2022-02-22 Facebook Technologies, Llc Integration of artificial reality interaction modes
US11475652B2 (en) 2020-06-30 2022-10-18 Samsung Electronics Co., Ltd. Automatic representation toggling based on depth camera field of view
US11893206B2 (en) * 2020-09-02 2024-02-06 Autodesk, Inc. Transitions between states in a hybrid virtual reality desktop computing environment
US11178376B1 (en) 2020-09-04 2021-11-16 Facebook Technologies, Llc Metering for display modes in artificial reality
US11507246B2 (en) * 2020-10-07 2022-11-22 Htc Corporation Method for dynamically showing virtual boundary, electronic device and computer readable storage medium thereof
CN112363628A (zh) * 2020-11-27 2021-02-12 上海影创信息科技有限公司 虚拟对象互动处理的方法和系统及其vr眼镜
CN112579029A (zh) * 2020-12-11 2021-03-30 上海影创信息科技有限公司 Vr眼镜的显示控制方法和系统
US11232644B1 (en) 2020-12-31 2022-01-25 Facebook Technologies, Llc Systems and methods for providing spatial awareness in virtual reality
US11461986B2 (en) 2021-01-27 2022-10-04 Qualcomm Incorporated Context-aware extended reality systems
US11294475B1 (en) 2021-02-08 2022-04-05 Facebook Technologies, Llc Artificial reality multi-modal input switching model
US11449137B2 (en) * 2021-02-12 2022-09-20 Rockwell Collins, Inc. Soldier and surface vehicle heads-up display imagery compensation system to align imagery with surroundings

Family Cites Families (166)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6061064A (en) 1993-08-31 2000-05-09 Sun Microsystems, Inc. System and method for providing and using a computer user interface with a view space having discrete portions
US5615132A (en) 1994-01-21 1997-03-25 Crossbow Technology, Inc. Method and apparatus for determining position and orientation of a moveable object using accelerometers
US5748189A (en) 1995-09-19 1998-05-05 Sony Corp Method and apparatus for sharing input devices amongst plural independent graphic display devices
US5877748A (en) 1995-11-20 1999-03-02 Redlich; Sanford I. Computer control input interface system
US6127990A (en) 1995-11-28 2000-10-03 Vega Vista, Inc. Wearable display and methods for controlling same
US6012926A (en) 1996-03-27 2000-01-11 Emory University Virtual reality system for treating patients with anxiety disorders
US5880733A (en) 1996-04-30 1999-03-09 Microsoft Corporation Display system and method for displaying windows of an operating system to provide a three-dimensional workspace for a computer system
GB2334643A (en) 1998-02-20 1999-08-25 Discreet Logic Inc Generating registration data for a virtual set
US6329986B1 (en) 1998-02-21 2001-12-11 U.S. Philips Corporation Priority-based virtual environment
GB2336057B (en) 1998-04-02 2002-05-08 Discreet Logic Inc Producing image data in a virtual set
WO2001056007A1 (en) 2000-01-28 2001-08-02 Intersense, Inc. Self-referenced tracking
US6552698B1 (en) 2000-05-02 2003-04-22 Agilent Technologies, Inc. Immersive display system
US20020044152A1 (en) 2000-10-16 2002-04-18 Abbott Kenneth H. Dynamic integration of computer generated and real world images
US20020154214A1 (en) 2000-11-02 2002-10-24 Laurent Scallie Virtual reality game system using pseudo 3D display driver
US6898266B2 (en) 2000-11-13 2005-05-24 Digitome Corporation 3D projection method
US6804607B1 (en) * 2001-04-17 2004-10-12 Derek Wood Collision avoidance system and method utilizing variable surveillance envelope
US6529331B2 (en) 2001-04-20 2003-03-04 Johns Hopkins University Head mounted display with full field of view and high resolution
US7274380B2 (en) 2001-10-04 2007-09-25 Siemens Corporate Research, Inc. Augmented reality system
US7010158B2 (en) 2001-11-13 2006-03-07 Eastman Kodak Company Method and apparatus for three-dimensional scene modeling and reconstruction
US7386799B1 (en) 2002-11-21 2008-06-10 Forterra Systems, Inc. Cinematic techniques in avatar-centric communication during a multi-user online simulation
US20070057946A1 (en) * 2003-07-24 2007-03-15 Dan Albeck Method and system for the three-dimensional surface reconstruction of an object
JP4262011B2 (ja) 2003-07-30 2009-05-13 キヤノン株式会社 画像提示方法及び装置
JP4401727B2 (ja) 2003-09-30 2010-01-20 キヤノン株式会社 画像表示装置及び方法
JP4638143B2 (ja) * 2003-12-26 2011-02-23 富士重工業株式会社 車両用運転支援装置
US7487463B2 (en) 2004-02-17 2009-02-03 Sun Microsystems, Inc. Multiprocess input redirection in a 3D window system
JP4125252B2 (ja) 2004-03-02 2008-07-30 株式会社東芝 画像生成装置、画像生成方法、及び画像生成プログラム
EP1754201A1 (en) 2004-05-27 2007-02-21 Canon Kabushiki Kaisha Information processing method, information processing apparatus, and image sensing apparatus
US20090040233A1 (en) 2004-06-10 2009-02-12 Kakuya Yamamoto Wearable Type Information Presentation Device
US7950026B1 (en) 2004-06-24 2011-05-24 Julian Michael Urbach Virtual application execution system and method
US7557774B2 (en) 2004-08-13 2009-07-07 Microsoft Corporation Displaying visually correct pointer movements on a multi-monitor display system
US20060050070A1 (en) 2004-09-07 2006-03-09 Canon Kabushiki Kaisha Information processing apparatus and method for presenting image combined with virtual image
US8195386B2 (en) 2004-09-28 2012-06-05 National University Corporation Kumamoto University Movable-body navigation information display method and movable-body navigation information display unit
US7248968B2 (en) * 2004-10-29 2007-07-24 Deere & Company Obstacle detection using stereo vision
US20060092178A1 (en) 2004-10-29 2006-05-04 Tanguay Donald O Jr Method and system for communicating through shared media
JP2006239844A (ja) * 2005-03-04 2006-09-14 Sony Corp 障害物回避装置、障害物回避方法及び障害物回避プログラム並びに移動型ロボット装置
JP4660357B2 (ja) * 2005-11-18 2011-03-30 任天堂株式会社 画像処理プログラムおよび画像処理装置
US8793620B2 (en) 2011-04-21 2014-07-29 Sony Computer Entertainment Inc. Gaze-assisted computer interface
US7542210B2 (en) 2006-06-29 2009-06-02 Chirieleison Sr Anthony Eye tracking head mounted display
JP4789745B2 (ja) 2006-08-11 2011-10-12 キヤノン株式会社 画像処理装置および方法
JP5154775B2 (ja) 2006-08-18 2013-02-27 任天堂株式会社 ゲームプログラムおよびゲーム装置
JP4999408B2 (ja) 2006-09-11 2012-08-15 スカラ株式会社 ヘッドマウント型表示装置、及び画像表示装置
CA2667315A1 (en) * 2006-11-03 2008-05-15 University Of Georgia Research Foundation Interfacing with virtual reality
US20080174659A1 (en) 2007-01-18 2008-07-24 Mcdowall Ian Wide field of view display device and method
JP5255623B2 (ja) * 2007-04-20 2013-08-07 ソフトキネティック エス.エイ. ボリューム認識方法およびシステム
US8605008B1 (en) * 2007-05-04 2013-12-10 Apple Inc. Head-mounted display
US8576247B2 (en) 2007-08-03 2013-11-05 Apple Inc. Method and apparatus for adjusting pixel values
US8049750B2 (en) 2007-11-16 2011-11-01 Sportvision, Inc. Fading techniques for virtual viewpoint animations
US20090160985A1 (en) * 2007-12-10 2009-06-25 The University Of Connecticut Method and system for recognition of a target in a three dimensional scene
US9355493B2 (en) 2007-12-31 2016-05-31 Advanced Micro Devices, Inc. Device and method for compositing video planes
US8786675B2 (en) 2008-01-23 2014-07-22 Michael F. Deering Systems using eye mounted displays
US20090199275A1 (en) 2008-02-06 2009-08-06 David Brock Web-browser based three-dimensional media aggregation social networking application
NL1035303C2 (nl) 2008-04-16 2009-10-19 Virtual Proteins B V Interactieve virtuele reality eenheid.
WO2009128781A1 (en) 2008-04-17 2009-10-22 Lundgren & Nordstrand Ab A method and a device for remote visualization
JP5111312B2 (ja) 2008-09-16 2013-01-09 任天堂株式会社 3次元画像処理プログラム、3次元画像処理装置、3次元画像処理システムおよび3次元画像処理方法
US8266536B2 (en) 2008-11-20 2012-09-11 Palo Alto Research Center Incorporated Physical-virtual environment interface
US8970690B2 (en) 2009-02-13 2015-03-03 Metaio Gmbh Methods and systems for determining the pose of a camera with respect to at least one object of a real environment
US8004769B2 (en) 2009-03-05 2011-08-23 Nabes, Llc Binocular apparatus and system
CN101540020B (zh) 2009-04-01 2012-09-26 江西省交通设计院 一种公路三维选线方法
US20100315413A1 (en) 2009-06-16 2010-12-16 Microsoft Corporation Surface Computer User Interaction
JP5263049B2 (ja) 2009-07-21 2013-08-14 ソニー株式会社 情報処理装置、および情報処理方法、並びにプログラム
US9286720B2 (en) 2009-08-20 2016-03-15 Northrop Grumman Systems Corporation Locative video for situation awareness
KR101627214B1 (ko) 2009-11-12 2016-06-03 엘지전자 주식회사 영상표시장치 및 그 동작방법
JP4679661B1 (ja) 2009-12-15 2011-04-27 株式会社東芝 情報提示装置、情報提示方法及びプログラム
US8754931B2 (en) 2010-01-08 2014-06-17 Kopin Corporation Video eyewear for smart phone games
US8964298B2 (en) 2010-02-28 2015-02-24 Microsoft Corporation Video display modification based on sensor input for a see-through near-to-eye display
CN101813976A (zh) 2010-03-09 2010-08-25 华南理工大学 基于soc的视线跟踪人机交互方法及装置
US8570344B2 (en) 2010-04-02 2013-10-29 Qualcomm Incorporated Augmented reality direction orientation mask
US8405680B1 (en) * 2010-04-19 2013-03-26 YDreams S.A., A Public Limited Liability Company Various methods and apparatuses for achieving augmented reality
US9798436B2 (en) 2010-07-08 2017-10-24 Red Hat Israel, Ltd. Remote computing with a low latency mouse mode
JP5791131B2 (ja) 2010-07-20 2015-10-07 アップル インコーポレイテッド 自然な相互作用のための相互作用的現実拡張
EP2418865A3 (en) 2010-08-09 2014-08-06 LG Electronics Inc. 3D viewing device, image display apparatus, and method for operating the same
US8780014B2 (en) 2010-08-25 2014-07-15 Eastman Kodak Company Switchable head-mounted display
KR101674957B1 (ko) 2010-08-31 2016-11-10 엘지전자 주식회사 이동 단말기 및 그 제어방법
US8675993B2 (en) 2010-09-03 2014-03-18 Adobe Systems Incorporated Methods and apparatus for patch-based shape from shading
US8941559B2 (en) 2010-09-21 2015-01-27 Microsoft Corporation Opacity filter for display device
EP2625845B1 (en) 2010-10-04 2021-03-03 Gerard Dirk Smits System and method for 3-d projection and enhancements for interactivity
US8884984B2 (en) 2010-10-15 2014-11-11 Microsoft Corporation Fusing virtual content into real content
US9292973B2 (en) 2010-11-08 2016-03-22 Microsoft Technology Licensing, Llc Automatic variable virtual focus for augmented reality displays
US8401225B2 (en) 2011-01-31 2013-03-19 Microsoft Corporation Moving object segmentation using depth images
US8606010B2 (en) 2011-03-18 2013-12-10 Seiko Epson Corporation Identifying text pixels in scanned images
KR20160084502A (ko) 2011-03-29 2016-07-13 퀄컴 인코포레이티드 로컬 멀티-사용자 협업을 위한 모듈식 모바일 접속된 피코 프로젝터들
US9030425B2 (en) 2011-04-19 2015-05-12 Sony Computer Entertainment Inc. Detection of interaction with virtual object from finger color change
US9022864B2 (en) 2011-06-03 2015-05-05 Nintendo Co., Ltd. Apparatus and method for controlling objects on a stereoscopic display
US8692738B2 (en) 2011-06-10 2014-04-08 Disney Enterprises, Inc. Advanced Pepper's ghost projection system with a multiview and multiplanar display
US20120327116A1 (en) 2011-06-23 2012-12-27 Microsoft Corporation Total field of view classification for head-mounted display
JP5145444B2 (ja) 2011-06-27 2013-02-20 株式会社コナミデジタルエンタテインメント 画像処理装置、画像処理装置の制御方法、及びプログラム
US9776460B2 (en) 2011-07-27 2017-10-03 Bridgestone Corporation Pneumatic tire with specified bead core and method for manufacturing the same
US10019962B2 (en) 2011-08-17 2018-07-10 Microsoft Technology Licensing, Llc Context adaptive user interface for augmented reality display
US9342610B2 (en) 2011-08-25 2016-05-17 Microsoft Technology Licensing, Llc Portals: registered objects as virtualized, personalized displays
WO2013029097A2 (en) 2011-08-30 2013-03-07 Monash University System and method for processing sensor data for the visually impaired
US9323325B2 (en) 2011-08-30 2016-04-26 Microsoft Technology Licensing, Llc Enhancing an object of interest in a see-through, mixed reality display device
WO2013033842A1 (en) 2011-09-07 2013-03-14 Tandemlaunch Technologies Inc. System and method for using eye gaze information to enhance interactions
US20130083018A1 (en) 2011-09-30 2013-04-04 Kevin A. Geisner Personal audio/visual system with holographic objects
US20130083007A1 (en) 2011-09-30 2013-04-04 Kevin A. Geisner Changing experience using personal a/v system
US20130088413A1 (en) 2011-10-05 2013-04-11 Google Inc. Method to Autofocus on Near-Eye Display
EP2579128B1 (en) 2011-10-05 2017-11-22 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Portable device, virtual reality system and method
US9081177B2 (en) * 2011-10-07 2015-07-14 Google Inc. Wearable computer with nearby object response
IN2014CN03103A (ru) 2011-10-20 2015-07-03 Koninkl Philips Nv
US9454849B2 (en) 2011-11-03 2016-09-27 Microsoft Technology Licensing, Llc Augmented reality playspaces with adaptive game rules
US8611015B2 (en) 2011-11-22 2013-12-17 Google Inc. User interface
US20130137076A1 (en) 2011-11-30 2013-05-30 Kathryn Stone Perez Head-mounted display based education and instruction
US20130141419A1 (en) 2011-12-01 2013-06-06 Brian Mount Augmented reality with realistic occlusion
WO2013085193A1 (ko) 2011-12-06 2013-06-13 경북대학교 산학협력단 사용자 인지 향상 장치 및 그 인지 향상 방법
US20130147686A1 (en) 2011-12-12 2013-06-13 John Clavin Connecting Head Mounted Displays To External Displays And Other Communication Networks
US20160011724A1 (en) 2012-01-06 2016-01-14 Google Inc. Hands-Free Selection Using a Ring-Based User-Interface
US9734633B2 (en) 2012-01-27 2017-08-15 Microsoft Technology Licensing, Llc Virtual environment generating system
CN102646117B (zh) 2012-02-20 2015-07-08 华为技术有限公司 文件数据传送的方法与装置
US10008002B2 (en) * 2012-02-28 2018-06-26 NXP Canada, Inc. Single-camera distance estimation
JP6066037B2 (ja) 2012-03-27 2017-01-25 セイコーエプソン株式会社 頭部装着型表示装置
US8988465B2 (en) 2012-03-30 2015-03-24 Ford Global Technologies, Llc Physical-virtual hybrid representation
CN103472909B (zh) 2012-04-10 2017-04-12 微软技术许可有限责任公司 用于头戴式、增强现实显示器的逼真遮挡
US8855442B2 (en) 2012-04-30 2014-10-07 Yuri Owechko Image registration of multimodal data using 3D-GeoArcs
JP2013238693A (ja) 2012-05-14 2013-11-28 Sharp Corp 画像表示装置、画像表示方法及びプログラム
US20130307855A1 (en) 2012-05-16 2013-11-21 Mathew J. Lamb Holographic story telling
US20130326364A1 (en) 2012-05-31 2013-12-05 Stephen G. Latta Position relative hologram interactions
JP5953963B2 (ja) 2012-06-13 2016-07-20 ソニー株式会社 頭部装着型映像表示装置
US9389420B2 (en) 2012-06-14 2016-07-12 Qualcomm Incorporated User interface interaction for transparent head-mounted displays
US9219901B2 (en) 2012-06-19 2015-12-22 Qualcomm Incorporated Reactive user interface for head-mounted display
US9645394B2 (en) 2012-06-25 2017-05-09 Microsoft Technology Licensing, Llc Configured virtual environments
US9767720B2 (en) 2012-06-25 2017-09-19 Microsoft Technology Licensing, Llc Object-centric mixed reality space
US9292085B2 (en) 2012-06-29 2016-03-22 Microsoft Technology Licensing, Llc Configuring an interaction zone within an augmented reality environment
WO2014009561A2 (en) 2012-07-13 2014-01-16 Softkinetic Software Method and system for human-to-computer gesture based simultaneous interactions using singular points of interest on a hand
US9429912B2 (en) 2012-08-17 2016-08-30 Microsoft Technology Licensing, Llc Mixed reality holographic object development
US8994614B2 (en) 2012-10-11 2015-03-31 Sony Computer Entertainment Europe Limited Head mountable display
US10241638B2 (en) 2012-11-02 2019-03-26 Atheer, Inc. Method and apparatus for a three dimensional interface
US9135052B2 (en) 2012-11-06 2015-09-15 Red Hat Israel, Ltd. Distributed multiple monitor display split using multiple client devices in a virtualization system
GB2499694B8 (en) 2012-11-09 2017-06-07 Sony Computer Entertainment Europe Ltd System and method of image reconstruction
US9619911B2 (en) 2012-11-13 2017-04-11 Qualcomm Incorporated Modifying virtual object display properties
JP5818773B2 (ja) 2012-11-22 2015-11-18 キヤノン株式会社 画像処理装置、画像処理方法及びプログラム
JP6271960B2 (ja) 2012-11-26 2018-01-31 キヤノン株式会社 情報処理システム
US20140168264A1 (en) 2012-12-19 2014-06-19 Lockheed Martin Corporation System, method and computer program product for real-time alignment of an augmented reality device
US9443352B1 (en) 2012-12-21 2016-09-13 Motion Reality, Inc. Navigating through a virtual environment having a real-world elevation characteristics using motion capture
JP6029453B2 (ja) 2012-12-21 2016-11-24 京セラ株式会社 携帯端末、カーソル表示制御プログラム、カーソル表示制御方法、カーソル発生制御プログラム、カーソル発生制御方法、アイコン表示プログラムおよびアイコン表示方法
US9058693B2 (en) 2012-12-21 2015-06-16 Dassault Systemes Americas Corp. Location correction of virtual objects
US9412201B2 (en) 2013-01-22 2016-08-09 Microsoft Technology Licensing, Llc Mixed reality filtering
US9083960B2 (en) * 2013-01-30 2015-07-14 Qualcomm Incorporated Real-time 3D reconstruction with power efficient depth sensor usage
EP2953099B1 (en) 2013-02-01 2019-02-13 Sony Corporation Information processing device, terminal device, information processing method, and programme
CN109656354B (zh) 2013-02-22 2022-05-31 索尼公司 信息处理装置和信息处理方法
US20140240351A1 (en) 2013-02-27 2014-08-28 Michael Scavezze Mixed reality augmentation
US20140253605A1 (en) 2013-03-05 2014-09-11 John N. Border Controlling brightness of a displayed image
US9041741B2 (en) 2013-03-14 2015-05-26 Qualcomm Incorporated User interface for a head mounted display
KR102148809B1 (ko) 2013-04-22 2020-08-27 삼성전자주식회사 단축 아이콘 윈도우 표시 장치, 방법 및 컴퓨터 판독 가능한 기록 매체
WO2014188798A1 (ja) 2013-05-21 2014-11-27 ソニー株式会社 表示制御装置、表示制御方法および記録媒体
US20140363073A1 (en) 2013-06-11 2014-12-11 Microsoft Corporation High-performance plane detection with depth camera data
WO2014204330A1 (en) 2013-06-17 2014-12-24 3Divi Company Methods and systems for determining 6dof location and orientation of head-mounted display and associated user movements
US9329682B2 (en) 2013-06-18 2016-05-03 Microsoft Technology Licensing, Llc Multi-step virtual object selection
US9256072B2 (en) 2013-10-02 2016-02-09 Philip Scott Lyren Wearable electronic glasses that detect movement of a real object copies movement of a virtual object
US9679144B2 (en) 2013-11-15 2017-06-13 Microsoft Technology Licensing, Llc Protecting privacy in web-based immersive augmented reality
JP6119570B2 (ja) 2013-11-19 2017-04-26 ソニー株式会社 表示装置、表示方法およびプログラム
US20150145887A1 (en) 2013-11-25 2015-05-28 Qualcomm Incorporated Persistent head-mounted content display
CN106104635B (zh) 2013-12-06 2019-01-04 惠普发展公司,有限责任合伙企业 遮挡增强现实对象
CN103761085B (zh) 2013-12-18 2018-01-19 微软技术许可有限责任公司 混合现实全息对象开发
US9626776B2 (en) 2013-12-20 2017-04-18 Cognex Corporation Apparatus, systems, and methods for processing a height map
CN104750440B (zh) 2013-12-30 2017-09-29 纬创资通股份有限公司 多屏幕的窗口管理方法、电子装置与计算机程序产品
US10001645B2 (en) 2014-01-17 2018-06-19 Sony Interactive Entertainment America Llc Using a second screen as a private tracking heads-up display
US9691181B2 (en) 2014-02-24 2017-06-27 Sony Interactive Entertainment Inc. Methods and systems for social sharing head mounted display (HMD) content with a second screen
US20150261293A1 (en) 2014-03-12 2015-09-17 Weerapan Wilairat Remote device control via gaze detection
KR20150141461A (ko) 2014-06-10 2015-12-18 엘지전자 주식회사 헤드 마운티드 디스플레이 및 그 제어 방법
US9904055B2 (en) 2014-07-25 2018-02-27 Microsoft Technology Licensing, Llc Smart placement of virtual objects to stay in the field of view of a head mounted display
US20160027218A1 (en) 2014-07-25 2016-01-28 Tom Salter Multi-user gaze projection using head mounted display devices
US9766460B2 (en) 2014-07-25 2017-09-19 Microsoft Technology Licensing, Llc Ground plane adjustment in a virtual reality environment
US20160027214A1 (en) 2014-07-25 2016-01-28 Robert Memmott Mouse sharing between a desktop and a virtual world
US10416760B2 (en) 2014-07-25 2019-09-17 Microsoft Technology Licensing, Llc Gaze-based object placement within a virtual reality environment
US10451875B2 (en) 2014-07-25 2019-10-22 Microsoft Technology Licensing, Llc Smart transparency for virtual objects
US9865089B2 (en) 2014-07-25 2018-01-09 Microsoft Technology Licensing, Llc Virtual reality environment with real world objects
US9858720B2 (en) 2014-07-25 2018-01-02 Microsoft Technology Licensing, Llc Three-dimensional mixed-reality viewport

Also Published As

Publication number Publication date
KR20170035995A (ko) 2017-03-31
US20160027212A1 (en) 2016-01-28
CA2954727A1 (en) 2016-01-28
CN106575155A (zh) 2017-04-19
EP3172649A1 (en) 2017-05-31
BR112017000069A2 (pt) 2017-10-31
MX2017001129A (es) 2017-05-09
KR102385756B1 (ko) 2022-04-11
EP3172649B1 (en) 2019-10-02
AU2015292490A1 (en) 2017-01-12
WO2016014877A1 (en) 2016-01-28
US10311638B2 (en) 2019-06-04
JP2017531221A (ja) 2017-10-19

Similar Documents

Publication Publication Date Title
RU2017102204A (ru) Препятствование спотыканию при погружении в среду виртуальной реальности
JP2018534687A5 (ru)
WO2016100931A8 (en) Method, system and device for navigating in a virtual reality environment
JP2016527583A5 (ru)
GB2507830A8 (en) System and Method of Image Augmentation
JP2015125774A5 (ru)
EP4254334A3 (en) System and method for valve quantification
JP6548967B2 (ja) 画像処理装置、画像処理方法及びプログラム
US10884576B2 (en) Mediated reality
EP3007038A3 (en) Interaction with three-dimensional video
CN106471544A (zh) 三维模型产生的系统和方法
EP3333541A3 (en) Surveying system
JP2014507723A5 (ru)
EP2394715A3 (en) Image display program, system and method
CN110633617B (zh) 使用语义分割的平面检测
JP2016522935A5 (ru)
ES2383976A1 (es) Método para el probado virtual de calzado.
JP2016530581A5 (ru)
WO2016003529A3 (en) Rapid avatar capture and simulation using commodity depth sensors
RU2015152810A (ru) Система выбора интерфейсного устройства пациента и способ, основанный на трехмерном моделировании
EP3062203A3 (en) Three-dimensional virtualization
EP3324619A3 (en) Three-dimensional (3d) rendering method and apparatus for user' eyes
CN108021227B (zh) 在虚拟现实中快速移动的方法及虚拟现实装置
US11302023B2 (en) Planar surface detection
JP2018526726A5 (ru)

Legal Events

Date Code Title Description
FA93 Acknowledgement of application withdrawn (no request for examination)

Effective date: 20180725