JP7030854B2 - 仮想現実システム内の仮想コントローラの位置および向きの追跡 - Google Patents

仮想現実システム内の仮想コントローラの位置および向きの追跡 Download PDF

Info

Publication number
JP7030854B2
JP7030854B2 JP2019570391A JP2019570391A JP7030854B2 JP 7030854 B2 JP7030854 B2 JP 7030854B2 JP 2019570391 A JP2019570391 A JP 2019570391A JP 2019570391 A JP2019570391 A JP 2019570391A JP 7030854 B2 JP7030854 B2 JP 7030854B2
Authority
JP
Japan
Prior art keywords
controller
virtual
segment
joint
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2019570391A
Other languages
English (en)
Japanese (ja)
Other versions
JP2020529646A5 (enExample
JP2020529646A (ja
Inventor
ラジャ，サマンサ
クレメント，マニュエル・クリスチャン
ニシュリー，シャニ―
スターンフェルド，ダニエル
ウェルカー，ステファン
ホロビッツ，ケビン
ファタス，コレンティン
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Publication of JP2020529646A publication Critical patent/JP2020529646A/ja
Priority to JP2021166315A priority Critical patent/JP7382994B2/ja
Publication of JP2020529646A5 publication Critical patent/JP2020529646A5/ja
Application granted granted Critical
Publication of JP7030854B2 publication Critical patent/JP7030854B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G06F3/0383—Signal control means within the pointing device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)
  • Processing Or Creating Images (AREA)
JP2019570391A 2017-09-18 2018-09-17 仮想現実システム内の仮想コントローラの位置および向きの追跡 Active JP7030854B2 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2021166315A JP7382994B2 (ja) 2017-09-18 2021-10-08 仮想現実システム内の仮想コントローラの位置および向きの追跡

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US15/708,040 2017-09-18
US15/708,040 US10386938B2 (en) 2017-09-18 2017-09-18 Tracking of location and orientation of a virtual controller in a virtual reality system
PCT/US2018/051361 WO2019055929A1 (en) 2017-09-18 2018-09-17 MONITORING THE LOCATION AND ORIENTATION OF A VIRTUAL CONTROL DEVICE IN A VIRTUAL REALITY SYSTEM

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2021166315A Division JP7382994B2 (ja) 2017-09-18 2021-10-08 仮想現実システム内の仮想コントローラの位置および向きの追跡

Publications (3)

Publication Number Publication Date
JP2020529646A JP2020529646A (ja) 2020-10-08
JP2020529646A5 JP2020529646A5 (enExample) 2021-12-02
JP7030854B2 true JP7030854B2 (ja) 2022-03-07

Family

ID=63963384

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2019570391A Active JP7030854B2 (ja) 2017-09-18 2018-09-17 仮想現実システム内の仮想コントローラの位置および向きの追跡
JP2021166315A Active JP7382994B2 (ja) 2017-09-18 2021-10-08 仮想現実システム内の仮想コントローラの位置および向きの追跡

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2021166315A Active JP7382994B2 (ja) 2017-09-18 2021-10-08 仮想現実システム内の仮想コントローラの位置および向きの追跡

Country Status (6)

Country Link
US (1) US10386938B2 (enExample)
EP (1) EP3685248B1 (enExample)
JP (2) JP7030854B2 (enExample)
KR (1) KR102338836B1 (enExample)
CN (1) CN110603510B (enExample)
WO (1) WO2019055929A1 (enExample)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12042303B2 (en) * 2018-03-07 2024-07-23 Brown University System for facilitating speech-based communication for individuals unable to speak or write
US20190385372A1 (en) * 2018-06-15 2019-12-19 Microsoft Technology Licensing, Llc Positioning a virtual reality passthrough region at a known distance
US11126341B2 (en) * 2019-02-21 2021-09-21 Htc Corporation Object manipulating method, host device and computer readable storage medium
US11086392B1 (en) * 2019-04-09 2021-08-10 Facebook Technologies, Llc Devices, systems, and methods for virtual representation of user interface devices
US11373374B2 (en) * 2019-11-07 2022-06-28 Volvo Car Corporation Aligning the augmented reality or virtual reality world with the real world using natural position understanding
JP7589696B2 (ja) * 2020-01-17 2024-11-26 ソニーグループ株式会社 情報処理装置及び情報処理方法、コンピュータプログラム、並びに拡張現実感システム
US11435185B2 (en) * 2020-02-21 2022-09-06 Microsoft Technology Licensing, Llc Systems and methods for deep learning-based pedestrian dead reckoning for exteroceptive sensor-enabled devices
WO2022061114A1 (en) * 2020-09-17 2022-03-24 Bogie Inc. System and method for an interactive controller
CN113318435A (zh) * 2021-04-27 2021-08-31 青岛小鸟看看科技有限公司 手柄控制追踪器的控制方法、装置及头戴式显示设备
US11934569B2 (en) * 2021-09-24 2024-03-19 Apple Inc. Devices, methods, and graphical user interfaces for interacting with three-dimensional environments
KR20230100472A (ko) 2021-12-28 2023-07-05 삼성전자주식회사 손의 관절에 관한 위치 정보를 획득하는 증강 현실 디바이스 및 그 동작 방법
JP7719010B2 (ja) * 2022-02-18 2025-08-05 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置およびデバイス位置推定方法
US20230419627A1 (en) * 2022-06-24 2023-12-28 Lowe's Companies, Inc. Object modeling based on properties and images of an object
CN120266080A (zh) * 2022-12-23 2025-07-04 创峰科技 限定虚拟边界的方法和系统
CN116585709A (zh) * 2023-06-06 2023-08-15 北京字跳网络技术有限公司 虚拟对象的运动控制方法、装置、电子设备及存储介质
KR20250087355A (ko) 2023-12-07 2025-06-16 가톨릭대학교 산학협력단 가상 환경에서 사용자 아바타를 통한 대화형 체험 서비스 제공 방법 및 장치
JP7663767B1 (ja) * 2024-09-27 2025-04-16 株式会社コロプラ プログラムおよび情報処理システム
JP7658021B1 (ja) * 2024-09-27 2025-04-07 株式会社コロプラ プログラムおよび情報処理システム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001236520A (ja) 2000-02-21 2001-08-31 Nippon Telegr & Teleph Corp <Ntt> 仮想空間における動作入力方法および装置およびその動作入力プログラムを記録した記録媒体および仮想空間システム
JP2016126500A (ja) 2014-12-26 2016-07-11 Kddi株式会社 ウェアラブル端末装置およびプログラム
JP2017045443A (ja) 2015-08-28 2017-03-02 株式会社タカラトミー 頭部装着型ディスプレイを備えた情報処理装置

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6104379A (en) * 1996-12-11 2000-08-15 Virtual Technologies, Inc. Forearm-supported exoskeleton hand-tracking device
US7472047B2 (en) * 1997-05-12 2008-12-30 Immersion Corporation System and method for constraining a graphical hand from penetrating simulated graphical objects
US6042555A (en) * 1997-05-12 2000-03-28 Virtual Technologies, Inc. Force-feedback interface device for the hand
US7324085B2 (en) 2002-01-25 2008-01-29 Autodesk, Inc. Techniques for pointing to locations within a volumetric display
US10936074B2 (en) 2003-11-20 2021-03-02 National Institute Of Advanced Industrial Science And Technology Haptic information presentation system and method
US20160098095A1 (en) * 2004-01-30 2016-04-07 Electronic Scripting Products, Inc. Deriving Input from Six Degrees of Freedom Interfaces
US20080009771A1 (en) * 2006-03-29 2008-01-10 Joel Perry Exoskeleton
US9082213B2 (en) * 2007-11-07 2015-07-14 Canon Kabushiki Kaisha Image processing apparatus for combining real object and virtual object and processing method therefor
JP5287421B2 (ja) * 2009-03-27 2013-09-11 株式会社Ｊｖｃケンウッド 入力装置及びこれを用いた電子機器
US20110267356A1 (en) * 2010-04-29 2011-11-03 Rennuit Antoine F Lix Robert Animating a virtual object within a virtual world
US20110267357A1 (en) * 2010-04-29 2011-11-03 Rennuit Antoine Felix Robert Animating a virtual object within a virtual world
US9177409B2 (en) * 2010-04-29 2015-11-03 Naturalmotion Ltd Animating a virtual object within a virtual world
US8483877B2 (en) * 2010-09-03 2013-07-09 GM Global Technology Operations LLC Workspace safe operation of a force- or impedance-controlled robot
US9552673B2 (en) 2012-10-17 2017-01-24 Microsoft Technology Licensing, Llc Grasping virtual objects in augmented reality
KR101691633B1 (ko) * 2012-11-01 2017-01-09 아이캠, 엘엘씨 무선 손목 컴퓨팅과 3d 영상화, 매핑, 네트워킹 및 인터페이스를 위한 제어 장치 및 방법
US20140152558A1 (en) * 2012-11-30 2014-06-05 Tom Salter Direct hologram manipulation using imu
US9202313B2 (en) * 2013-01-21 2015-12-01 Microsoft Technology Licensing, Llc Virtual interaction with image projection
WO2014204330A1 (en) 2013-06-17 2014-12-24 3Divi Company Methods and systems for determining 6dof location and orientation of head-mounted display and associated user movements
US9437159B2 (en) * 2014-01-25 2016-09-06 Sony Interactive Entertainment America Llc Environmental interrupt in a head-mounted display and utilization of non field of view real estate
US9649558B2 (en) * 2014-03-14 2017-05-16 Sony Interactive Entertainment Inc. Gaming device with rotatably placed cameras
US10416760B2 (en) * 2014-07-25 2019-09-17 Microsoft Technology Licensing, Llc Gaze-based object placement within a virtual reality environment
KR102299364B1 (ko) * 2014-08-25 2021-09-07 삼성전자주식회사 웨어러블 디바이스의 센서가 측정하는 임피던스 값을 보정하는 방법 및 장치
US9690367B2 (en) 2014-12-10 2017-06-27 Sixense Entertainment, Inc. System and method for assisting a user in locating physical objects while the user is in a virtual reality environment
US10725297B2 (en) * 2015-01-28 2020-07-28 CCP hf. Method and system for implementing a virtual representation of a physical environment using a virtual reality environment
US10180734B2 (en) * 2015-03-05 2019-01-15 Magic Leap, Inc. Systems and methods for augmented reality
US9898091B2 (en) * 2015-06-03 2018-02-20 Oculus Vr, Llc Virtual reality system with head-mounted display, camera and hand-held controllers
WO2016205182A1 (en) * 2015-06-15 2016-12-22 Survios, Inc. Systems and methods for immersive physical interaction with a virtual environment
US10095024B2 (en) * 2015-08-07 2018-10-09 Sony Interactive Entertainment Inc. Systems and methods for using a MEMS projector to determine an orientation of a photosensor of an HMD or another controller
US10010788B2 (en) * 2015-12-21 2018-07-03 Sony Interactive Entertainment Inc. Game controller with lights visible inside and outside the game controller
US10019131B2 (en) * 2016-05-10 2018-07-10 Google Llc Two-handed object manipulations in virtual reality
US10249090B2 (en) * 2016-06-09 2019-04-02 Microsoft Technology Licensing, Llc Robust optical disambiguation and tracking of two or more hand-held controllers with passive optical and inertial tracking
WO2018022657A1 (en) * 2016-07-25 2018-02-01 Ctrl-Labs Corporation System and method for measuring the movements of articulated rigid bodies
US10596700B2 (en) * 2016-09-16 2020-03-24 Carbon Robotics, Inc. System and calibration, registration, and training methods
US10134192B2 (en) * 2016-10-17 2018-11-20 Microsoft Technology Licensing, Llc Generating and displaying a computer generated image on a future pose of a real world object
US10657696B2 (en) * 2016-12-13 2020-05-19 DeepMotion, Inc. Virtual reality system using multiple force arrays for a solver
US10444865B2 (en) * 2017-05-01 2019-10-15 Google Llc Tracking of position and orientation of objects in virtual reality systems
US11054894B2 (en) * 2017-05-05 2021-07-06 Microsoft Technology Licensing, Llc Integrated mixed-input system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001236520A (ja) 2000-02-21 2001-08-31 Nippon Telegr & Teleph Corp <Ntt> 仮想空間における動作入力方法および装置およびその動作入力プログラムを記録した記録媒体および仮想空間システム
JP2016126500A (ja) 2014-12-26 2016-07-11 Kddi株式会社 ウェアラブル端末装置およびプログラム
JP2017045443A (ja) 2015-08-28 2017-03-02 株式会社タカラトミー 頭部装着型ディスプレイを備えた情報処理装置

Also Published As

Publication number Publication date
CN110603510A (zh) 2019-12-20
CN110603510B (zh) 2023-10-27
KR20190136047A (ko) 2019-12-09
WO2019055929A1 (en) 2019-03-21
US20190087019A1 (en) 2019-03-21
JP7382994B2 (ja) 2023-11-17
JP2022008987A (ja) 2022-01-14
EP3685248B1 (en) 2022-03-30
EP3685248A1 (en) 2020-07-29
US10386938B2 (en) 2019-08-20
JP2020529646A (ja) 2020-10-08
KR102338836B1 (ko) 2021-12-13

Similar Documents

Publication Publication Date Title
JP7030854B2 (ja) 仮想現実システム内の仮想コントローラの位置および向きの追跡
CN108139815B (zh) 用于虚拟现实内容的显示的场景和对象的分立时间扭曲
KR102038638B1 (ko) 가상 현실에서 핸드 헬드 디바이스를 트랙킹하는 시스템
CN110794958B (zh) 在增强/虚拟现实环境中使用的输入设备
US10642344B2 (en) Manipulating virtual objects with six degree-of-freedom controllers in an augmented and/or virtual reality environment
EP3314371B1 (en) System for tracking a handheld device in an augmented and/or virtual reality environment
US10147243B2 (en) Generating virtual notation surfaces with gestures in an augmented and/or virtual reality environment
CN109074149B (zh) 用于增强或虚拟现实环境的头戴式参考系中的对象跟踪
EP3414644B1 (en) Control system for navigation in virtual reality environment
CN108604123B (zh) 虚拟现实中的位置仪
CN111771180A (zh) 增强现实环境中对象的混合放置
US20180136744A1 (en) Input controller stabilization techniques for virtual reality systems
WO2024066756A1 (zh) 交互方法、装置和显示设备
CN106681506A (zh) 一种终端设备中非vr应用的交互方法及终端设备
CN116149474A (zh) 近眼显示设备的控制方法、装置及近眼显示设备
CN115617164A (zh) 交互方法、装置和近眼显示设备
CN118244879A (zh) 对象移动控制方法、装置、设备及介质

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200204

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20200204

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20210113

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20210126

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20210426

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20210608

A524 Written submission of copy of amendment under article 19 pct

Free format text: JAPANESE INTERMEDIATE CODE: A524

Effective date: 20211008

C60 Trial request (containing other claim documents, opposition documents)

Free format text: JAPANESE INTERMEDIATE CODE: C60

Effective date: 20211008

C11 Written invitation by the commissioner to file amendments

Free format text: JAPANESE INTERMEDIATE CODE: C11

Effective date: 20211019

A911 Transfer to examiner for re-examination before appeal (zenchi)

Free format text: JAPANESE INTERMEDIATE CODE: A911

Effective date: 20211122

C21 Notice of transfer of a case for reconsideration by examiners before appeal proceedings

Free format text: JAPANESE INTERMEDIATE CODE: C21

Effective date: 20211124

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20220125

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220222

R150 Certificate of patent or registration of utility model

Ref document number: 7030854

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250