WO2022014445A1 - 検出装置及び検出方法 - Google Patents

検出装置及び検出方法 Download PDF

Info

Publication number
WO2022014445A1
WO2022014445A1 PCT/JP2021/025658 JP2021025658W WO2022014445A1 WO 2022014445 A1 WO2022014445 A1 WO 2022014445A1 JP 2021025658 W JP2021025658 W JP 2021025658W WO 2022014445 A1 WO2022014445 A1 WO 2022014445A1
Authority
WO
WIPO (PCT)
Prior art keywords
finger
sensor
information
posture
hand
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/025658
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
邦仁 澤井
郁男 山野
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Group Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Group Corp filed Critical Sony Group Corp
Priority to JP2022536299A priority Critical patent/JP7667961B2/ja
Priority to US18/013,137 priority patent/US12045388B2/en
Priority to CN202180048229.0A priority patent/CN115777091A/zh
Publication of WO2022014445A1 publication Critical patent/WO2022014445A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/26—Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Definitions

  • This disclosure relates to a detection device and a detection method.
  • Patent Document 1 discloses that various controls are performed based on the shape of a user's finger (for example, index finger, middle finger, ring finger, little finger) detected by a non-contact sensor.
  • a user's finger for example, index finger, middle finger, ring finger, little finger
  • one finger may be hidden by another, and the shape of the finger may not be detected correctly.
  • the index finger is hidden by the thumb, and the shape of the index finger cannot be detected correctly.
  • it is necessary to provide a contact sensor for each finger which increases the number of sensors. Therefore, it is inconvenient for the user to move his / her fingers or grab an object.
  • FIG. 1A is a view of the controller 100 from the back side of the hand
  • FIG. 1B is a view of the controller 100 seen from the side surface of the thumb side
  • FIG. 2 is a view of the controller 100 as viewed from the palm side.
  • the sensor 1 detects information (first information) that senses at least one finger (first finger) included in the detection range of the sensor 1.
  • the sensor 1 detects depth information indicating a distance (depth) from a finger included in the detection range. More specifically, the sensor 1 detects the depth information of each joint portion included in the at least one finger.
  • the sensor 1 is a depth sensor, and the sensor 1 detects the depth information of each joint portion.
  • the information detected by the sensor 1 is not limited to the depth information, and may be any information as long as it is useful for specifying the finger (or the position and posture of each joint included in the finger). For example, an image that does not include a distance (RGB image, black-and-white image, infrared image, etc.
  • the measurement marker 14 may be a light emitting marker.
  • the measurement marker 14 may be a two-dimensional marker or a three-dimensional marker. In the case of a two-dimensional marker or a three-dimensional marker, non-light emission or light emission may be detected by an RGB camera or a black-and-white camera, and in the case of a light-emitting marker, detection is possible even in a dark place.
  • an infrared camera may be used to detect a marker that emits infrared rays.
  • a contact sensor 25 is provided on the back surface of the mounting portion 10 (the surface facing the palm) on the palm side.
  • the contact sensor 25 is a sensor that detects contact with the palm.
  • the contact sensor 25 is electrically connected to the main body 11 via wiring provided in the mounting portion 10 (for example, wiring embedded in a belt). By detecting the contact with the palm by the contact sensor 25, it is possible to detect that the user is wearing the controller 100.
  • the signal processing circuit 42 calculates information (posture information) that identifies the postures of five fingers or at least one finger based on the depth information of the sensor 1. More specifically, the position / posture information that specifies the position / posture of each joint of each finger is calculated. The position / posture information of each joint of each finger calculated from the depth information of the sensor 1 is referred to as position / posture information 1.
  • the position / attitude information 1 is the position / attitude information of the local coordinate system based on the position of the sensor 1.
  • the calculation unit 44 determines the position / posture information of each joint of each finger with respect to the main body 11 based on the position / posture information 1 of each joint of each finger and the position / posture information 2 of each joint of each finger. ..
  • the position of the main body 11 is detected by the head device (or IMU43).
  • the position of the main body 11 may be treated as the position of the sensor 1 as an example.
  • the sensor 1 is fixed to the housing 12, and the positional relationship with the housing 12 is constant.
  • the sensor 2 since the sensor 2 is connected to the sensor 1 (housing 12) via the mounting portion 10, the relative position and orientation of the coordinate system between the sensor 1 and the sensor 2 is undefined.
  • Equation (3) can be transformed into equation (4).
  • the capacitance detection circuit 45 When it is determined that the user has mounted the controller 100, the capacitance detection circuit 45 provides the mounting detection signal to the transmission / reception unit 46. When it is determined that the user does not wear the controller 100, the wearing detection signal is not provided to the transmission / reception unit 46, or the non-wearing detection signal is provided to the transmission / reception unit 46.
  • the infrared irradiation unit 301 is, for example, an LED (Light Emitting Diode) that emits infrared rays.
  • LED Light Emitting Diode
  • the computer 200 includes a transmission / reception unit 201, a transmission / reception unit 202, a gesture recognition unit 230, an instruction generation unit 240, and an application unit 250.
  • an input unit for inputting data or instructions by the user, an output unit for outputting information to the user, and the like may be provided.
  • the elements 201, 202, 230, 240, and 250 are configured by, for example, a hardware circuit, a storage device, and the like.
  • the hardware circuit is composed of a processing circuit such as a CPU, DSP, ASIC, FPGA, or a combination thereof.
  • [Item 10] The detection device according to any one of items 1 to 9, further comprising a contact sensor for detecting contact between the thumb and the index finger.
  • [Item 11] The detection device according to any one of items 7 to 9, further comprising a contact sensor for detecting contact with at least one of the palm and the back of the hand.
  • [Item 12] A calculation for calculating the relative position and posture of the first sensor and the second sensor based on the first information and the second information of the joint portion commonly included in the first finger and the second finger.
  • the detection device according to item 3 comprising a unit.
  • [Item 13] The detection device according to item 12, wherein the joint portion commonly included is the terminal segment of the index finger.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2021/025658 2020-07-14 2021-07-07 検出装置及び検出方法 Ceased WO2022014445A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2022536299A JP7667961B2 (ja) 2020-07-14 2021-07-07 検出装置及び検出方法
US18/013,137 US12045388B2 (en) 2020-07-14 2021-07-07 Finger detection device and method providing increased reliability detecting attitudes of fingers
CN202180048229.0A CN115777091A (zh) 2020-07-14 2021-07-07 检测装置及检测方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020120810 2020-07-14
JP2020-120810 2020-07-14

Publications (1)

Publication Number Publication Date
WO2022014445A1 true WO2022014445A1 (ja) 2022-01-20

Family

ID=79554830

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/025658 Ceased WO2022014445A1 (ja) 2020-07-14 2021-07-07 検出装置及び検出方法

Country Status (4)

Country Link
US (1) US12045388B2 (enExample)
JP (1) JP7667961B2 (enExample)
CN (1) CN115777091A (enExample)
WO (1) WO2022014445A1 (enExample)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11872499B2 (en) * 2019-02-08 2024-01-16 Sony Interactive Entertainment Inc. Wearing member
US11959997B2 (en) * 2019-11-22 2024-04-16 Magic Leap, Inc. System and method for tracking a wearable device
CN120002689A (zh) * 2025-02-13 2025-05-16 哈尔滨工业大学 基于imu的软体灵巧手姿态传感系统及其集成方法

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050179644A1 (en) * 2002-03-12 2005-08-18 Gunilla Alsio Data input device
JP2006276651A (ja) * 2005-03-30 2006-10-12 Nagasaki Prefecture 動作検出装置および手話動作検出システム
US20100156783A1 (en) * 2001-07-06 2010-06-24 Bajramovic Mark Wearable data input device
JP2012073830A (ja) * 2010-09-29 2012-04-12 Pioneer Electronic Corp インターフェース装置
US20140098018A1 (en) * 2012-10-04 2014-04-10 Microsoft Corporation Wearable sensor for tracking articulated body-parts
JP2017516185A (ja) * 2014-03-14 2017-06-15 株式会社ソニー・インタラクティブエンタテインメント 回転自在に設置されたカメラ付きのゲーミングデバイス
WO2017102922A1 (de) * 2015-12-18 2017-06-22 Schannath Christina Handschuh-artige vorrichtung mit ergonomisch vorteilhaft auslösbarer integrierter kamera
JP2017227687A (ja) * 2016-06-20 2017-12-28 聖 星野 カメラアセンブリ、そのカメラアセンブリを用いる手指形状検出システム、そのカメラアセンブリを用いる手指形状検出方法、その検出方法を実施するプログラム、及び、そのプログラムの記憶媒体
WO2018079384A1 (ja) * 2016-10-28 2018-05-03 株式会社ソニー・インタラクティブエンタテインメント 情報処理システム、情報処理装置、制御方法、及びプログラム
WO2018198272A1 (ja) * 2017-04-27 2018-11-01 株式会社ソニー・インタラクティブエンタテインメント 制御装置、情報処理システム、制御方法、及びプログラム

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7498956B2 (en) * 2006-01-04 2009-03-03 Iron Will Creations, Inc. Apparatus and method for inputting information
US20100023314A1 (en) * 2006-08-13 2010-01-28 Jose Hernandez-Rebollar ASL Glove with 3-Axis Accelerometers
US20150035743A1 (en) * 2013-07-31 2015-02-05 Plantronics, Inc. Wrist Worn Platform for Sensors
US9710060B2 (en) * 2014-06-09 2017-07-18 BeBop Senors, Inc. Sensor system integrated with a glove
US10055018B2 (en) * 2014-08-22 2018-08-21 Sony Interactive Entertainment Inc. Glove interface object with thumb-index controller
JP6669069B2 (ja) 2014-09-10 2020-03-18 ソニー株式会社 検出装置、検出方法、制御装置、および制御方法
WO2016074087A2 (en) * 2014-11-11 2016-05-19 Helio Technology Inc. A method of detecting user input in a 3d space and a 3d input system employing same
KR102550595B1 (ko) * 2014-12-08 2023-06-30 로힛 세스 착용가능 무선 hmi 디바이스
US9652038B2 (en) * 2015-02-20 2017-05-16 Sony Interactive Entertainment Inc. Magnetic tracking of glove fingertips
US10168776B2 (en) * 2016-08-23 2019-01-01 International Business Machines Corporation Remote control via proximity data
DE112018005530T5 (de) * 2017-09-29 2020-07-02 Apple Inc. IMU-basierter Handschuh
US11681369B2 (en) * 2019-09-16 2023-06-20 Iron Will Innovations Canada Inc. Control-point activation condition detection for generating corresponding control signals

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100156783A1 (en) * 2001-07-06 2010-06-24 Bajramovic Mark Wearable data input device
US20050179644A1 (en) * 2002-03-12 2005-08-18 Gunilla Alsio Data input device
JP2006276651A (ja) * 2005-03-30 2006-10-12 Nagasaki Prefecture 動作検出装置および手話動作検出システム
JP2012073830A (ja) * 2010-09-29 2012-04-12 Pioneer Electronic Corp インターフェース装置
US20140098018A1 (en) * 2012-10-04 2014-04-10 Microsoft Corporation Wearable sensor for tracking articulated body-parts
JP2017516185A (ja) * 2014-03-14 2017-06-15 株式会社ソニー・インタラクティブエンタテインメント 回転自在に設置されたカメラ付きのゲーミングデバイス
WO2017102922A1 (de) * 2015-12-18 2017-06-22 Schannath Christina Handschuh-artige vorrichtung mit ergonomisch vorteilhaft auslösbarer integrierter kamera
JP2017227687A (ja) * 2016-06-20 2017-12-28 聖 星野 カメラアセンブリ、そのカメラアセンブリを用いる手指形状検出システム、そのカメラアセンブリを用いる手指形状検出方法、その検出方法を実施するプログラム、及び、そのプログラムの記憶媒体
WO2018079384A1 (ja) * 2016-10-28 2018-05-03 株式会社ソニー・インタラクティブエンタテインメント 情報処理システム、情報処理装置、制御方法、及びプログラム
WO2018198272A1 (ja) * 2017-04-27 2018-11-01 株式会社ソニー・インタラクティブエンタテインメント 制御装置、情報処理システム、制御方法、及びプログラム

Also Published As

Publication number Publication date
JPWO2022014445A1 (enExample) 2022-01-20
CN115777091A (zh) 2023-03-10
US20230244312A1 (en) 2023-08-03
JP7667961B2 (ja) 2025-04-24
US12045388B2 (en) 2024-07-23

Similar Documents

Publication Publication Date Title
US11747901B1 (en) Fiducial rings in virtual reality
US11481031B1 (en) Devices, systems, and methods for controlling computing devices via neuromuscular signals of users
CN112424730B (zh) 具有手指设备的计算机系统
JP6669069B2 (ja) 検出装置、検出方法、制御装置、および制御方法
US9367136B2 (en) Holographic object feedback
KR101844390B1 (ko) 사용자 인터페이스 제어를 위한 시스템 및 기법
US20220155866A1 (en) Ring device having an antenna, a touch pad, and/or a charging pad to control a computing device based on user motions
US20140160055A1 (en) Wearable multi-modal input device for augmented reality
EP4116939A1 (en) Information processing device and information processing method, computer program, and augmented reality sensing system
JP7667961B2 (ja) 検出装置及び検出方法
EP3035164A1 (en) Wearable sensor for tracking articulated body-parts
JP7135444B2 (ja) 情報処理装置及びプログラム
JP7754170B2 (ja) オブジェクトの保持を判定する情報処理装置
WO2021207033A1 (en) Input device to control a computing device with a touch pad having a curved surface configured to sense touch input
WO2023042489A1 (ja) 触感生成装置、触感生成方法及びプログラム
JPWO2018074054A1 (ja) 表示制御装置、表示制御方法及びプログラム
KR20240115679A (ko) 햅틱 정보를 생성하기 위한 전자 장치 및 방법
KR20250002501A (ko) Xr 디바이스, xr 디바이스의 컨트롤러 장치, 이들을 이용한 xr 디바이스의 동작 방법
CN121713146A (zh) 用于显示接口的头戴式显示装置以及头戴式显示装置的操作方法
KR20170110442A (ko) 증강현실 기능을 구비한 제어용 컴퓨터 장치 및 그 제어방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21842587

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022536299

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21842587

Country of ref document: EP

Kind code of ref document: A1