JP6591411B2 - 空間的対話における追加モダリティのための顔追跡 - Google Patents

空間的対話における追加モダリティのための顔追跡 Download PDF

Info

Publication number
JP6591411B2
JP6591411B2 JP2016527260A JP2016527260A JP6591411B2 JP 6591411 B2 JP6591411 B2 JP 6591411B2 JP 2016527260 A JP2016527260 A JP 2016527260A JP 2016527260 A JP2016527260 A JP 2016527260A JP 6591411 B2 JP6591411 B2 JP 6591411B2
Authority
JP
Japan
Prior art keywords
user
user device
coordinate system
face
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2016527260A
Other languages
English (en)
Japanese (ja)
Other versions
JP2016536687A5 (enExample
JP2016536687A (ja
Inventor
ハルムット・ザイヒター
Original Assignee
クアルコム，インコーポレイテッド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by クアルコム，インコーポレイテッド filed Critical クアルコム，インコーポレイテッド
Publication of JP2016536687A publication Critical patent/JP2016536687A/ja
Publication of JP2016536687A5 publication Critical patent/JP2016536687A5/ja
Application granted granted Critical
Publication of JP6591411B2 publication Critical patent/JP6591411B2/ja
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G06V40/176—Dynamic expression
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Image Analysis (AREA)
JP2016527260A 2013-11-08 2014-11-07 空間的対話における追加モダリティのための顔追跡 Expired - Fee Related JP6591411B2 (ja)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201361902025P 2013-11-08 2013-11-08
US61/902,025 2013-11-08
US14/535,132 2014-11-06
US14/535,132 US10146299B2 (en) 2013-11-08 2014-11-06 Face tracking for additional modalities in spatial interaction
PCT/US2014/064625 WO2015070063A1 (en) 2013-11-08 2014-11-07 Face tracking for additional modalities in spatial interaction

Publications (3)

Publication Number Publication Date
JP2016536687A JP2016536687A (ja) 2016-11-24
JP2016536687A5 JP2016536687A5 (enExample) 2017-11-30
JP6591411B2 true JP6591411B2 (ja) 2019-10-16

Family

ID=52014350

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2016527260A Expired - Fee Related JP6591411B2 (ja) 2013-11-08 2014-11-07 空間的対話における追加モダリティのための顔追跡

Country Status (6)

Country Link
US (1) US10146299B2 (enExample)
EP (1) EP3066543B1 (enExample)
JP (1) JP6591411B2 (enExample)
CN (2) CN105683868B (enExample)
ES (1) ES2897797T3 (enExample)
WO (1) WO2015070063A1 (enExample)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017136938A1 (en) * 2016-02-10 2017-08-17 Tandemlaunch Inc. A quality adaptive multimodal affect recognition system for user-centric multimedia indexing
US10748450B1 (en) * 2016-11-29 2020-08-18 Sproutel, Inc. System, apparatus, and method for creating an interactive augmented reality experience to simulate medical procedures for pediatric disease education
US11056022B1 (en) * 2016-11-29 2021-07-06 Sproutel, Inc. System, apparatus, and method for creating an interactive augmented reality experience to simulate medical procedures for pediatric disease education
DE102017108194A1 (de) * 2017-04-18 2018-10-18 Vorwerk & Co. Interholding Gmbh Verfahren zum Betrieb eines sich selbsttätig fortbewegenden Fahrzeugs
CN108563327B (zh) * 2018-03-26 2020-12-01 Oppo广东移动通信有限公司 增强现实方法、装置、存储介质及电子设备
CN108932632A (zh) * 2018-06-01 2018-12-04 北京市商汤科技开发有限公司 广告互动方法及装置、电子设备和存储介质
US11733824B2 (en) 2018-06-22 2023-08-22 Apple Inc. User interaction interpreter
FR3137517A1 (fr) * 2022-06-29 2024-01-05 Sagemcom Broadband Procede de selection de portions d’images dans un flux video et systeme executant le procede.
WO2025162741A1 (en) * 2024-01-30 2025-08-07 Roche Diabetes Care Gmbh Method of detecting an analyte in a sample of bodily fluid

Family Cites Families (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6317127B1 (en) * 1996-10-16 2001-11-13 Hughes Electronics Corporation Multi-user real-time augmented reality system and method
US6842175B1 (en) * 1999-04-22 2005-01-11 Fraunhofer Usa, Inc. Tools for interacting with virtual environments
JP2002157607A (ja) * 2000-11-17 2002-05-31 Canon Inc 画像生成システム、画像生成方法および記憶媒体
JP3762309B2 (ja) * 2002-02-18 2006-04-05 キヤノン株式会社 位置方向測定装置および情報処理方法
US7737965B2 (en) * 2005-06-09 2010-06-15 Honeywell International Inc. Handheld synthetic vision device
US8730156B2 (en) * 2010-03-05 2014-05-20 Sony Computer Entertainment America Llc Maintaining multiple views on a shared stable virtual space
US20100159434A1 (en) * 2007-10-11 2010-06-24 Samsun Lampotang Mixed Simulator and Uses Thereof
US20090322671A1 (en) 2008-06-04 2009-12-31 Cybernet Systems Corporation Touch screen augmented reality system and method
US9092053B2 (en) 2008-06-17 2015-07-28 Apple Inc. Systems and methods for adjusting a display based on the user's position
CN102203850A (zh) 2008-09-12 2011-09-28 格斯图尔泰克公司 相对于用户而定向所显示的元素
US8788977B2 (en) 2008-11-20 2014-07-22 Amazon Technologies, Inc. Movement recognition as input mechanism
CN101943982B (zh) * 2009-07-10 2012-12-12 北京大学 基于被跟踪的眼睛运动的图像操作
US8762846B2 (en) * 2009-11-16 2014-06-24 Broadcom Corporation Method and system for adaptive viewport for a mobile device based on viewing angle
US9507418B2 (en) * 2010-01-21 2016-11-29 Tobii Ab Eye tracker based contextual action
CN102884490B (zh) 2010-03-05 2016-05-04 索尼电脑娱乐美国公司 在共享的稳定虚拟空间上维持多视图
US9901828B2 (en) * 2010-03-30 2018-02-27 Sony Interactive Entertainment America Llc Method for an augmented reality character to maintain and exhibit awareness of an observer
US8581905B2 (en) * 2010-04-08 2013-11-12 Disney Enterprises, Inc. Interactive three dimensional displays on handheld devices
JP5651386B2 (ja) * 2010-06-23 2015-01-14 ソフトバンクモバイル株式会社 眼鏡型表示装置
US9524523B2 (en) * 2010-09-01 2016-12-20 Vigor Systems Inc. Fail-safe switch for media insertion server in a broadcast stream
KR101740231B1 (ko) 2010-11-17 2017-05-26 삼성전자주식회사 3차원 얼굴 위치 추정 방법 및 장치
DE102010060942A1 (de) * 2010-12-01 2012-06-06 Sick Ag Sensoranordnung zur Objekterkennung
EP2652537B1 (de) * 2010-12-15 2021-02-03 Carl Zeiss Microscopy GmbH Automatisierte abbildung vorbestimmter bereiche in schnittserien
US9354718B2 (en) * 2010-12-22 2016-05-31 Zspace, Inc. Tightly coupled interactive stereo display
US20120304059A1 (en) * 2011-05-24 2012-11-29 Microsoft Corporation Interactive Build Instructions
US20130063560A1 (en) * 2011-09-12 2013-03-14 Palo Alto Research Center Incorporated Combined stereo camera and stereo display interaction
US9974615B2 (en) * 2011-09-28 2018-05-22 Brainlab Ag Determining a position of a medical device to be localized
US20130121559A1 (en) 2011-11-16 2013-05-16 Sharp Laboratories Of America, Inc. Mobile device with three dimensional augmented reality
US20130137076A1 (en) * 2011-11-30 2013-05-30 Kathryn Stone Perez Head-mounted display based education and instruction
US20130159935A1 (en) * 2011-12-16 2013-06-20 Garrick EVANS Gesture inputs for navigating in a 3d scene via a gui
WO2013111119A1 (en) * 2012-01-27 2013-08-01 Saar Wilf Simulating interaction with a three-dimensional environment
US20140068526A1 (en) * 2012-02-04 2014-03-06 Three Bots Ltd Method and apparatus for user interaction
US20130275873A1 (en) 2012-04-13 2013-10-17 Qualcomm Incorporated Systems and methods for displaying a user interface
US20130286161A1 (en) 2012-04-25 2013-10-31 Futurewei Technologies, Inc. Three-dimensional face recognition for mobile devices
US20140092005A1 (en) * 2012-09-28 2014-04-03 Glen Anderson Implementation of an augmented reality element
TWI466062B (zh) * 2012-10-04 2014-12-21 Ind Tech Res Inst 重建三維模型的方法與三維模型重建裝置
US8928695B2 (en) * 2012-10-05 2015-01-06 Elwha Llc Formatting of one or more persistent augmentations in an augmented view in response to multiple input factors
US20140168261A1 (en) * 2012-12-13 2014-06-19 Jeffrey N. Margolis Direct interaction system mixed reality environments
US9230326B1 (en) * 2012-12-31 2016-01-05 Cognex Corporation System, method and calibration plate employing embedded 2D data codes as self-positioning fiducials
US20140316543A1 (en) * 2013-04-19 2014-10-23 Qualcomm Incorporated Configuring audio for a coordinated display session between a plurality of proximate client devices
EP2800020B1 (en) * 2013-04-30 2020-11-04 Dassault Systèmes A computer-implemented method for manipulating three-dimensional modeled objects of an assembly in a three-dimensional scene.
US20150009238A1 (en) * 2013-07-03 2015-01-08 Nvidia Corporation Method for zooming into and out of an image shown on a display
US9615081B2 (en) * 2013-10-28 2017-04-04 Lateral Reality Kft. Method and multi-camera portable device for producing stereo images

Also Published As

Publication number Publication date
CN110488972B (zh) 2023-06-09
ES2897797T3 (es) 2022-03-02
CN105683868B (zh) 2019-07-30
EP3066543B1 (en) 2021-10-27
CN110488972A (zh) 2019-11-22
US20150130704A1 (en) 2015-05-14
WO2015070063A1 (en) 2015-05-14
EP3066543A1 (en) 2016-09-14
JP2016536687A (ja) 2016-11-24
US10146299B2 (en) 2018-12-04
CN105683868A (zh) 2016-06-15

Similar Documents

Publication Publication Date Title
JP6591411B2 (ja) 空間的対話における追加モダリティのための顔追跡
US11003307B1 (en) Artificial reality systems with drawer simulation gesture for gating user interface elements
EP3218781B1 (en) Spatial interaction in augmented reality
US11043192B2 (en) Corner-identifiying gesture-driven user interface element gating for artificial reality systems
US11023035B1 (en) Virtual pinboard interaction using a peripheral device in artificial reality environments
US10976804B1 (en) Pointer-based interaction with a virtual surface using a peripheral device in artificial reality environments
US10921879B2 (en) Artificial reality systems with personal assistant element for gating user interface elements
US20200387286A1 (en) Arm gaze-driven user interface element gating for artificial reality systems
US10133342B2 (en) Human-body-gesture-based region and volume selection for HMD
CN108469899B (zh) 识别可穿戴显示装置的观察空间中的瞄准点或区域的方法
US10649616B2 (en) Volumetric multi-selection interface for selecting multiple objects in 3D space
WO2020146125A1 (en) Holographic palm raycasting for targeting virtual objects
US20110164032A1 (en) Three-Dimensional User Interface
US10852839B1 (en) Artificial reality systems with detachable personal assistant for gating user interface elements
CN107743604A (zh) 增强和/或虚拟现实环境中的触摸屏悬停检测
KR20140040246A (ko) 컴퓨터 비젼 애플리케이션들에서 상호작용 반경을 확대하기 위한 제스쳐-제어 기법
EP2558924B1 (en) Apparatus, method and computer program for user input using a camera
US11023036B1 (en) Virtual drawing surface interaction using a peripheral device in artificial reality environments
US20150277570A1 (en) Providing Onscreen Visualizations of Gesture Movements
Jain et al. [POSTER] AirGestAR: Leveraging Deep Learning for Complex Hand Gestural Interaction with Frugal AR Devices
JP2025181315A (ja) 情報処理装置、情報処理方法、およびプログラム
VanWaardhuizen et al. Table top augmented reality system for conceptual design and prototyping

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20160510

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20171020

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20171020

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20181122

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20181210

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20190311

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20190819

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20190918

R150 Certificate of patent or registration of utility model

Ref document number: 6591411

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees