WO2013151947A1 - Touch sensitive user interface - Google Patents

Touch sensitive user interface Download PDF

Info

Publication number
WO2013151947A1
WO2013151947A1 PCT/US2013/034868 US2013034868W WO2013151947A1 WO 2013151947 A1 WO2013151947 A1 WO 2013151947A1 US 2013034868 W US2013034868 W US 2013034868W WO 2013151947 A1 WO2013151947 A1 WO 2013151947A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
display
touch region
hand
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2013/034868
Other languages
English (en)
French (fr)
Inventor
Anthony J. Ambrus
Abdulwajid N. Mohamed
Andrew D. Wilson
Brian J. MOUNT
Jordan D. Andersen
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to JP2015504664A priority Critical patent/JP2015515688A/ja
Priority to EP13771808.6A priority patent/EP2834723B1/en
Priority to KR1020147030415A priority patent/KR102107867B1/ko
Publication of WO2013151947A1 publication Critical patent/WO2013151947A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
    • G06F3/005—Input arrangements through a video camera
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/0416—Control or interface arrangements specially adapted for digitisers
    • G06F3/0418—Control or interface arrangements specially adapted for digitisers for error correction or compensation, e.g. based on parallax, calibration or alignment
    • G06F3/04186—Touch location disambiguation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • G06F3/0426—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking fingers with respect to a virtual keyboard projected or printed on the surface
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted

Definitions

  • FIGURE 6D shows a flowchart of one embodiment of a process of tracking fingertips.
  • Figure 9C is a top view of a third embodiment of a movable display optical system of a see-through, near-eye, mixed reality device including an arrangement of gaze detection elements.
  • processing unit 4 may be integrated in software and hardware components of the HMD 2.
  • the processing unit may also be referred to as "processing logic.”
  • Processing logic may include any combination of hardware and/or software.
  • processing logic could include an application specific integrated circuit (ASIC).
  • ASIC application specific integrated circuit
  • Processing logic could include a computer readable storage device having stored thereon processor executable instructions and a processor which executes the instructions.
  • step 306 user input is received for the electronic device based on the 3D position of the user's hand relative to the touch region 11. If the user is touching or nearly touching the surface having the touch region 11, the input might be a selection of an element on the display 96. If the user is hovering slightly further away from the surface, the input might be for a hover or focus event.
  • a hover event is to present additional information for an element on the display 96. For example, if the user is hovering over a telephone number on the display 96, the hover event could be to present a menu of options including dialing that number, performing a reverse lookup, sending an email to that person (if email address is known), etc.
  • each pixel in the depth image is associated with a unique (u, v) coordinate, in one embodiment.
  • Each depth pixel from the depth image may be processed by looking up the plane equation that was previously determined for that (u, v) coordinate. Based on that plane equation the distance from the depth pixel and the plane may be determined. Thus, in effect, each depth pixel is assigned a distance to the surface.
  • FIG 7 is a flowchart of one embodiment of a process 700 of rendering the user input element 18 and shadows 21.
  • a point cloud model 605 for the hand is accessed.
  • each pixel in the point cloud model 605 is processed based on its depth value such that further rendering steps can produce an image such as the example of Figure 1C.
  • the user input element 18 may have a transparent inner portion 13, and edges 15, 17. Some of the edges can be highlighted to show the user that these are touch points 15.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Optics & Photonics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)
PCT/US2013/034868 2012-04-02 2013-04-02 Touch sensitive user interface Ceased WO2013151947A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2015504664A JP2015515688A (ja) 2012-04-02 2013-04-02 タッチ感応ユーザー・インターフェース
EP13771808.6A EP2834723B1 (en) 2012-04-02 2013-04-02 Touch sensitive user interface
KR1020147030415A KR102107867B1 (ko) 2012-04-02 2013-04-02 터치 감지 사용자 인터페이스

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US13/437,461 US8933912B2 (en) 2012-04-02 2012-04-02 Touch sensitive user interface with three dimensional input sensor
US13/437,461 2012-04-02

Publications (1)

Publication Number Publication Date
WO2013151947A1 true WO2013151947A1 (en) 2013-10-10

Family

ID=49095260

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/034868 Ceased WO2013151947A1 (en) 2012-04-02 2013-04-02 Touch sensitive user interface

Country Status (6)

Country Link
US (1) US8933912B2 (https=)
EP (1) EP2834723B1 (https=)
JP (1) JP2015515688A (https=)
KR (1) KR102107867B1 (https=)
CN (1) CN103294260B (https=)
WO (1) WO2013151947A1 (https=)

Families Citing this family (100)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9477302B2 (en) * 2012-08-10 2016-10-25 Google Inc. System and method for programing devices within world space volumes
FR2976681B1 (fr) * 2011-06-17 2013-07-12 Inst Nat Rech Inf Automat Systeme de colocalisation d'un ecran tactile et d'un objet virtuel et dispostif pour la manipulation d'objets virtuels mettant en oeuvre un tel systeme
US8698901B2 (en) * 2012-04-19 2014-04-15 Hewlett-Packard Development Company, L.P. Automatic calibration
TWI436251B (zh) * 2012-04-30 2014-05-01 Univ Nat Taiwan 觸碰式控制裝置及控制方法
WO2013175389A2 (en) * 2012-05-20 2013-11-28 Extreme Reality Ltd. Methods circuits apparatuses systems and associated computer executable code for providing projection based human machine interfaces
US10114609B2 (en) 2012-05-31 2018-10-30 Opportunity Partners Inc. Computing interface for users with disabilities
KR20140019950A (ko) * 2012-08-07 2014-02-18 성균관대학교산학협력단 단말기의 모노 카메라에 입력된 손가락 영상을 이용한 3차원 좌표 생성 방법 및 모노 카메라에 입력된 손가락 영상을 이용하여 3차원 좌표를 생성하는 이동 단말기
US9978178B1 (en) * 2012-10-25 2018-05-22 Amazon Technologies, Inc. Hand-based interaction in virtually shared workspaces
US10241638B2 (en) * 2012-11-02 2019-03-26 Atheer, Inc. Method and apparatus for a three dimensional interface
US20140201685A1 (en) * 2013-01-14 2014-07-17 Darren Lim User input determination
US9041691B1 (en) * 2013-02-11 2015-05-26 Rawles Llc Projection surface with reflective elements for non-visible light
WO2014128749A1 (ja) * 2013-02-19 2014-08-28 株式会社ブリリアントサービス 形状認識装置、形状認識プログラム、および形状認識方法
IL308285B2 (en) 2013-03-11 2024-11-01 Magic Leap Inc System and method for augmentation and virtual reality
US9144905B1 (en) * 2013-03-13 2015-09-29 Hrl Laboratories, Llc Device and method to identify functional parts of tools for robotic manipulation
US10599328B2 (en) * 2013-03-14 2020-03-24 Valve Corporation Variable user tactile input device with display feedback system
US9417452B2 (en) 2013-03-15 2016-08-16 Magic Leap, Inc. Display system and method
US9218090B2 (en) * 2013-04-03 2015-12-22 Dell Products, Lp System and method for controlling a projector via a passive control strip
US9323338B2 (en) 2013-04-12 2016-04-26 Usens, Inc. Interactive input system and method
US20140354602A1 (en) * 2013-04-12 2014-12-04 Impression.Pi, Inc. Interactive input system and method
US10891003B2 (en) * 2013-05-09 2021-01-12 Omni Consumer Products, Llc System, method, and apparatus for an interactive container
US9465488B2 (en) * 2013-05-09 2016-10-11 Stephen Howard System and method for motion detection and interpretation
KR101571096B1 (ko) * 2013-11-11 2015-11-23 주식회사 시공미디어 사용자 적응형 인터페이스 제공 시스템 및 방법
CN106415439B (zh) * 2013-11-21 2020-04-10 惠普发展公司，有限责任合伙企业 用于镜面反射红外光的投影屏幕
US9875019B2 (en) * 2013-12-26 2018-01-23 Visteon Global Technologies, Inc. Indicating a transition from gesture based inputs to touch surfaces
US9390726B1 (en) 2013-12-30 2016-07-12 Google Inc. Supplementing speech commands with gestures
US9213413B2 (en) 2013-12-31 2015-12-15 Google Inc. Device interaction with spatially aware gestures
US20150205360A1 (en) * 2014-01-20 2015-07-23 Lenovo (Singapore) Pte. Ltd. Table top gestures for mimicking mouse control
US9233469B2 (en) * 2014-02-13 2016-01-12 GM Global Technology Operations LLC Robotic system with 3D box location functionality
CN104850330B (zh) * 2014-02-18 2018-12-14 联想(北京)有限公司 信息处理方法、系统及电子设备
CN104850215B (zh) * 2014-02-18 2018-11-09 联想(北京)有限公司 信息处理方法、系统及电子设备
US9971490B2 (en) 2014-02-26 2018-05-15 Microsoft Technology Licensing, Llc Device control
CN106255938B (zh) 2014-02-28 2019-12-17 惠普发展公司，有限责任合伙企业 传感器和投影仪的校准
WO2015142023A1 (en) 2014-03-21 2015-09-24 Samsung Electronics Co., Ltd. Method and wearable device for providing a virtual input interface
US9563956B2 (en) * 2014-03-26 2017-02-07 Intel Corporation Efficient free-space finger recognition
EP2975580B1 (en) * 2014-07-16 2019-06-26 Wipro Limited Method and system for providing visual feedback in a virtual reality environment
US10664090B2 (en) 2014-07-31 2020-05-26 Hewlett-Packard Development Company, L.P. Touch region projection onto touch-sensitive surface
US9952677B2 (en) 2014-09-08 2018-04-24 Atheer, Inc. Method and apparatus for distinguishing features in data
US10444894B2 (en) 2014-09-12 2019-10-15 Hewlett-Packard Development Company, L.P. Developing contextual information from an image
US10275092B2 (en) 2014-09-24 2019-04-30 Hewlett-Packard Development Company, L.P. Transforming received touch input
US10168838B2 (en) 2014-09-30 2019-01-01 Hewlett-Packard Development Company, L.P. Displaying an object indicator
US10281997B2 (en) * 2014-09-30 2019-05-07 Hewlett-Packard Development Company, L.P. Identification of an object on a touch-sensitive surface
KR102029756B1 (ko) * 2014-11-03 2019-10-08 삼성전자주식회사 웨어러블 디바이스 및 그 제어 방법
US9606697B1 (en) 2014-11-11 2017-03-28 Google Inc. Display cursor for motion controller
KR20160063812A (ko) * 2014-11-27 2016-06-07 삼성전자주식회사 화면 구성 방법, 전자 장치 및 저장 매체
US9519360B2 (en) * 2014-12-11 2016-12-13 Synaptics Incorporated Palm rejection visualization for passive stylus
CN112631438B (zh) 2014-12-30 2023-08-04 欧姆尼消费品有限责任公司 交互式投影的系统和方法
CN111522434B (zh) 2015-04-30 2024-12-10 谷歌有限责任公司 用于手势跟踪和识别的基于rf的微运动跟踪
EP3594787B1 (en) 2015-08-06 2025-04-09 InterDigital VC Holdings, Inc. Methods and systems for providing haptic feedback for virtual 3d objects
US10067569B2 (en) * 2015-08-14 2018-09-04 Fresenius Medical Care Holdings, Inc. Touchless interface for a medical treatment system
GB2541730B (en) * 2015-08-28 2020-05-13 Samsung Electronics Co Ltd Displaying graphical user interface elements on a touch screen
US10503265B2 (en) * 2015-09-08 2019-12-10 Microvision, Inc. Mixed-mode depth detection
CN106550206A (zh) * 2015-09-18 2017-03-29 三亚中兴软件有限责任公司 视频会议的视频显示方法及装置
US10817065B1 (en) 2015-10-06 2020-10-27 Google Llc Gesture recognition using multiple antenna
EP3392740A4 (en) * 2015-12-18 2018-12-19 Sony Corporation Information processing device, information processing method, and program
WO2017110257A1 (ja) * 2015-12-21 2017-06-29 ソニー株式会社 情報処理装置および情報処理方法
US20180239417A1 (en) * 2015-12-30 2018-08-23 Shenzhen Royole Technologies Co. Ltd. Head-mounted display device, head-mounted display system, and input method
US10025492B2 (en) * 2016-02-08 2018-07-17 Microsoft Technology Licensing, Llc Pointing detection
KR102568710B1 (ko) 2016-03-30 2023-08-21 삼성전자주식회사 전자장치 및 그 동작 방법
US20170336918A1 (en) * 2016-05-22 2017-11-23 Intel Corporation Touch-sensing devices using minimum depth-value surface characterizations and associated methods
CN105915418A (zh) * 2016-05-23 2016-08-31 珠海格力电器股份有限公司 一种家电控制方法及装置
KR102629254B1 (ko) * 2016-09-01 2024-01-24 주식회사 넥슨코리아 게임 키맵 제공 방법 및 장치
US10607069B2 (en) * 2016-09-12 2020-03-31 Intel Corporation Determining a pointing vector for gestures performed before a depth camera
KR101964192B1 (ko) * 2016-09-30 2019-04-02 주식회사 케이엠씨로보틱스 시뮬레이션용 스마트 테이블 장치
US10157308B2 (en) * 2016-11-30 2018-12-18 Whirlpool Corporation Interaction recognition and analysis system
US10762641B2 (en) 2016-11-30 2020-09-01 Whirlpool Corporation Interaction recognition and analysis system
CN106598390B (zh) * 2016-12-12 2021-01-15 联想(北京)有限公司 一种显示方法、电子设备及显示装置
US20180173300A1 (en) * 2016-12-19 2018-06-21 Microsoft Technology Licensing, Llc Interactive virtual objects in mixed reality environments
US10761188B2 (en) 2016-12-27 2020-09-01 Microvision, Inc. Transmitter/receiver disparity for occlusion-based height estimation
US10061441B2 (en) * 2016-12-27 2018-08-28 Microvision, Inc. Touch interactivity with occlusions in returned illumination data
US11002855B2 (en) 2016-12-27 2021-05-11 Microvision, Inc. Occlusion-based height estimation
US10291890B2 (en) * 2017-01-12 2019-05-14 Intel Corporation Touch interface for a projection device
US12194375B2 (en) 2017-01-25 2025-01-14 Kieran S. Lyden Game controller
US10507385B2 (en) 2017-01-25 2019-12-17 Kieran S. Lyden Game controller
EP4050403B1 (en) * 2017-01-30 2025-10-22 Alcon Inc. Systems and method for augmented reality ophthalmic surgical microscope projection
US10785850B2 (en) * 2017-02-16 2020-09-22 Signify Holding B.V. Controller for indicating a presence of a virtual object via a lighting device and a method thereof
AU2018355446A1 (en) 2017-10-26 2020-05-14 Magic Leap, Inc. Broadband adaptive lens assembly for augmented reality display
US10345895B2 (en) 2017-11-07 2019-07-09 International Business Machines Corporation Hand and finger line grid for hand based interactions
US11029756B2 (en) * 2018-01-12 2021-06-08 Intel Corporation Display with pixel level eye tracking
TWI710926B (zh) * 2018-03-23 2020-11-21 宏碁股份有限公司 鍵盤投影裝置及智慧選字方法
US10831030B2 (en) * 2018-04-20 2020-11-10 Microsoft Technology Licensing, Llc Systems and methods for visually guided gaze-based targeting
US10452947B1 (en) * 2018-06-08 2019-10-22 Microsoft Technology Licensing, Llc Object recognition using depth and multi-spectral camera
JP2021536592A (ja) 2018-08-31 2021-12-27 マジック リープ, インコーポレイテッドMagic Leap, Inc. 拡張現実デバイスのための空間的に分解された動的調光
US10970922B2 (en) 2018-09-28 2021-04-06 Apple Inc. Generating a 3D model of a fingertip for visual touch detection
US11386621B2 (en) 2018-12-31 2022-07-12 Whirlpool Corporation Augmented reality feedback of inventory for an appliance
CN113544570B (zh) 2019-01-11 2025-02-14 奇跃公司 各种深度处的虚拟内容的时间复用显示
US11245875B2 (en) 2019-01-15 2022-02-08 Microsoft Technology Licensing, Llc Monitoring activity with depth and multi-spectral camera
CN110060318B (zh) * 2019-04-26 2023-03-21 网易（杭州）网络有限公司 信息处理方法、信息处理装置、处理器和存储介质
CN110308817B (zh) * 2019-06-10 2023-04-07 青岛小鸟看看科技有限公司 一种触控动作识别方法及触控投影系统
US10510155B1 (en) * 2019-06-11 2019-12-17 Mujin, Inc. Method and processing system for updating a first image generated by a first camera based on a second image generated by a second camera
US11216065B2 (en) * 2019-09-26 2022-01-04 Lenovo (Singapore) Pte. Ltd. Input control display based on eye gaze
US11597078B2 (en) 2020-07-28 2023-03-07 Nvidia Corporation Machine learning control of object handovers
US11893206B2 (en) * 2020-09-02 2024-02-06 Autodesk, Inc. Transitions between states in a hybrid virtual reality desktop computing environment
CN114327109B (zh) * 2020-09-30 2023-12-19 明基智能科技（上海）有限公司 触控操作方法及触控操作系统
EP3985491A1 (en) * 2020-10-19 2022-04-20 ameria AG Control method for touchless gesture control
TWI763208B (zh) * 2020-12-25 2022-05-01 宏碁股份有限公司 聲音訊號處理方法及電子裝置
CN215873824U (zh) * 2021-08-19 2022-02-22 宁波沱沱河设计有限公司 一种可发光的桌面
US11853509B1 (en) 2022-05-09 2023-12-26 Microsoft Technology Licensing, Llc Using a camera to supplement touch sensing
CH722073A1 (de) * 2024-08-28 2026-03-13 Lusee Ag Verfahren und Vorrichtung zur Bestimmung der Höhe eines Zeigeobjekts über einer Oberfläche in einem Zielbereich
CN118762243B (zh) * 2024-09-06 2024-12-06 天津市工业和信息化研究院(天津市节能中心、天津市工业和信息化局教育中心) 基于特征标定的机器视觉检测方法及系统
US12326971B1 (en) 2024-10-03 2025-06-10 Bansen Labs, Llc System and method for facilitating adaptive recentering in virtual reality environments

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070098250A1 (en) * 2003-05-01 2007-05-03 Delta Dansk Elektronik, Lys Og Akustik Man-machine interface based on 3-D positions of the human body
US20070132721A1 (en) * 2005-12-09 2007-06-14 Edge 3 Technologies Llc Three-Dimensional Virtual-Touch Human-Machine Interface System and Method Therefor
WO2008132724A1 (en) * 2007-04-26 2008-11-06 Mantisvision Ltd. A method and apparatus for three dimensional interaction with autosteroscopic displays
US20090219253A1 (en) * 2008-02-29 2009-09-03 Microsoft Corporation Interactive Surface Computer with Switchable Diffuser
US20100066735A1 (en) * 2004-07-30 2010-03-18 Dor Givon Apparatus system and method for human-machine interface
US20100315413A1 (en) 2009-06-16 2010-12-16 Microsoft Corporation Surface Computer User Interaction
WO2011011009A1 (en) 2009-07-23 2011-01-27 Hewlett-Packard Development Company, L.P. Display with an optical sensor

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1991007826A1 (en) 1989-11-22 1991-05-30 Russell David C Computer control system
US6512838B1 (en) * 1999-09-22 2003-01-28 Canesta, Inc. Methods for enhancing performance and data acquired from three-dimensional image systems
TW480431B (en) 1999-11-18 2002-03-21 Tai-He Yang Desk-placed structure of hand-held electric control apparatus
US6847354B2 (en) * 2000-03-23 2005-01-25 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Three dimensional interactive display
US6747635B2 (en) 2000-12-16 2004-06-08 Kamran Ossia Multi-mode handheld computer
JP4383864B2 (ja) * 2001-09-05 2009-12-16 徹 大田 文字入力機能を有する装置
US7359003B1 (en) * 2001-11-09 2008-04-15 Synerdyne Corporation Display, input and form factor for portable instruments
US7646372B2 (en) * 2003-09-15 2010-01-12 Sony Computer Entertainment Inc. Methods and systems for enabling direction detection when interfacing with a computer program
IL151255A0 (en) * 2002-08-14 2003-04-10 Ariel Yedidya System and method for interacting with computer using a video-camera image on screen and appurtenances useful therewith
US20050197843A1 (en) 2004-03-07 2005-09-08 International Business Machines Corporation Multimodal aggregating unit
JP4351599B2 (ja) * 2004-09-03 2009-10-28 パナソニック株式会社 入力装置
US20060181519A1 (en) 2005-02-14 2006-08-17 Vernier Frederic D Method and system for manipulating graphical objects displayed on a touch-sensitive display surface using displaced pop-ups
US20060267946A1 (en) * 2005-05-25 2006-11-30 Microsoft Corporation Methods and systems for providing feedback corresponding to user input
CA2699628A1 (en) * 2007-09-14 2009-03-19 Matthew Bell Gesture-based user interactions with status indicators for acceptable inputs in volumetric zones
US8259163B2 (en) * 2008-03-07 2012-09-04 Intellectual Ventures Holding 67 Llc Display with built in 3D sensing
JP4318056B1 (ja) * 2008-06-03 2009-08-19 島根県 画像認識装置および操作判定方法
US20100053151A1 (en) 2008-09-02 2010-03-04 Samsung Electronics Co., Ltd In-line mediation for manipulating three-dimensional content on a display device
US20100079413A1 (en) * 2008-09-29 2010-04-01 Denso Corporation Control device
US8237666B2 (en) * 2008-10-10 2012-08-07 At&T Intellectual Property I, L.P. Augmented I/O for limited form factor user-interfaces
KR20100041006A (ko) * 2008-10-13 2010-04-22 엘지전자 주식회사 3차원 멀티 터치를 이용한 사용자 인터페이스 제어방법
US8135299B2 (en) * 2009-04-03 2012-03-13 Xerox Corporation Computer generated imagery (CGI) fault clearance instructions
US8760391B2 (en) * 2009-05-22 2014-06-24 Robert W. Hawkins Input cueing emersion system and method
US9044172B2 (en) 2009-10-01 2015-06-02 Intel Corporation Ergonomic detection, processing and alerting for computing devices
JP4783456B2 (ja) * 2009-12-22 2011-09-28 株式会社東芝 映像再生装置及び映像再生方法
US8749557B2 (en) * 2010-06-11 2014-06-10 Microsoft Corporation Interacting with user interface via avatar
US8649592B2 (en) * 2010-08-30 2014-02-11 University Of Illinois At Urbana-Champaign System for background subtraction with 3D camera
WO2012088443A1 (en) * 2010-12-24 2012-06-28 Kevadiya, Inc. System and method for automated capture and compaction of instructional performances
US8736583B2 (en) * 2011-03-29 2014-05-27 Intel Corporation Virtual links between different displays to present a single virtual object
US8897491B2 (en) * 2011-06-06 2014-11-25 Microsoft Corporation System for finger recognition and tracking
US8928735B2 (en) * 2011-06-14 2015-01-06 Microsoft Corporation Combined lighting, projection, and image capture without video feedback
US9560314B2 (en) * 2011-06-14 2017-01-31 Microsoft Technology Licensing, Llc Interactive and shared surfaces

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070098250A1 (en) * 2003-05-01 2007-05-03 Delta Dansk Elektronik, Lys Og Akustik Man-machine interface based on 3-D positions of the human body
US20100066735A1 (en) * 2004-07-30 2010-03-18 Dor Givon Apparatus system and method for human-machine interface
US20070132721A1 (en) * 2005-12-09 2007-06-14 Edge 3 Technologies Llc Three-Dimensional Virtual-Touch Human-Machine Interface System and Method Therefor
WO2008132724A1 (en) * 2007-04-26 2008-11-06 Mantisvision Ltd. A method and apparatus for three dimensional interaction with autosteroscopic displays
US20090219253A1 (en) * 2008-02-29 2009-09-03 Microsoft Corporation Interactive Surface Computer with Switchable Diffuser
US20100315413A1 (en) 2009-06-16 2010-12-16 Microsoft Corporation Surface Computer User Interaction
WO2011011009A1 (en) 2009-07-23 2011-01-27 Hewlett-Packard Development Company, L.P. Display with an optical sensor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2834723A4 *

Also Published As

Publication number Publication date
EP2834723A4 (en) 2015-04-29
KR20150002718A (ko) 2015-01-07
JP2015515688A (ja) 2015-05-28
CN103294260A (zh) 2013-09-11
CN103294260B (zh) 2016-11-16
KR102107867B1 (ko) 2020-05-07
EP2834723A1 (en) 2015-02-11
US8933912B2 (en) 2015-01-13
US20130257748A1 (en) 2013-10-03
EP2834723B1 (en) 2017-12-27

Similar Documents

Publication Publication Date Title
EP2834723B1 (en) Touch sensitive user interface
EP3008567B1 (en) User focus controlled graphical user interface using an head mounted device
US9727132B2 (en) Multi-visor: managing applications in augmented reality environments
EP2652543B1 (en) Optimized focal area for augmented reality displays
US9116666B2 (en) Gesture based region identification for holograms
EP3191921B1 (en) Stabilizing motion of an interaction ray
EP3011411B1 (en) Hybrid world/body locked hud on an hmd
US9288468B2 (en) Viewing windows for video streams
CA2815372C (en) Automatic variable virtual focus for augmented reality displays
US20130326364A1 (en) Position relative hologram interactions
US20130342572A1 (en) Control of displayed content in virtual environments
WO2016077508A1 (en) System for automatic eye tracking calibration of head mounted display device
HK1187427B (zh) 觸敏用戶界面
HK1187427A (en) Touch sensitive user interface

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13771808

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2015504664

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2013771808

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 20147030415

Country of ref document: KR

Kind code of ref document: A