WO2019083405A1 - Gant de réalité virtuelle - Google Patents

Gant de réalité virtuelle

Info

Publication number
WO2019083405A1
WO2019083405A1 PCT/RU2018/000630 RU2018000630W WO2019083405A1 WO 2019083405 A1 WO2019083405 A1 WO 2019083405A1 RU 2018000630 W RU2018000630 W RU 2018000630W WO 2019083405 A1 WO2019083405 A1 WO 2019083405A1
Authority
WO
WIPO (PCT)
Prior art keywords
sensors
glove
imu
fingers
module
Prior art date
Application number
PCT/RU2018/000630
Other languages
English (en)
Russian (ru)
Inventor
Андрей Сергеевич КАМОЦКИЙ
Original Assignee
Федоров, Александр Владимирович
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Федоров, Александр Владимирович filed Critical Федоров, Александр Владимирович
Publication of WO2019083405A1 publication Critical patent/WO2019083405A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Definitions

  • the device relates to manipulators, namely to virtual gloves,
  • the prior art discloses a solution KR100221335, which describes a system and method for a glove to transmit a sensor value in a virtual reality system, including a series of sensors measuring movements of the wrist and fingers and outputting an electrical signal; The multiplexer selects the number of sensor signals included in one of the sensor groups.
  • the purpose of the patent is fundamentally different from the stated solution.
  • the analogs describe exactly the high-speed data transfer between the sensors and the virtual reality system. In the claimed solution, the data transfer method is irrelevant.
  • CN 1480822 describes a data entry device for people with
  • the sensor contains a virtual key detector and a key decoder, both of which are individually calibrated.
  • the purpose of the glove and the detection method differ. In the analogue, it is proposed to interact by touching
  • the device includes a glove that includes sensors for detecting hand gestures, as well as hand position sensors connected to the glove and connected to a computer system for
  • gestures are determined by means of external sensors installed, for example, on a computer monitor.
  • the solution EP2624238 describes a virtual mock-up with a tactile hand, having simulated objects that are manipulated with the aid of hand wearable gloves during the operation of the simulator.
  • the fingers on the fingers are equipped with 3D general motion detection tools integrated with at least one sensor for interaction tasks and one tactile component.
  • the analog is not specified as it is precisely the motion sensors used.
  • a gesture-based data management system for processing real-time computer animation, such as virtual reality, and is controlled on a computer.
  • the system includes a digital glove for data management based on operator hand gestures. The decision is based on the use of materials that change their resistance when bent.
  • the difference between the declared utility model is the use of inertial sensors.
  • information devices such as a personal computer (PC), a pocket computer, and a cell phone, so that the positions of the corresponding fingertips are displayed on a virtual keyboard on the display of one of the information-processing devices.
  • PC personal computer
  • pocket computer a pocket computer
  • cell phone a cell phone
  • the analog is not specified as it is precisely the motion sensors used.
  • the closest analogue is a virtual glove according to patent US9060385, publ .: 06/16/2015.
  • the prototype describes a virtual reality glove containing sensors, gloves located on the fingers, and hand position sensors,
  • the sensors are connected to the microprocessor.
  • the adaptation time depends on the strength of the stimulus, for different parts of the body it varies from 2 to 20 seconds. Thanks to the adaptation, we do not feel the touch of clothes to the body. See [Ekzertseva Ekaterina Vadimovna, Topic 1.1. General issues of life safety, Russian State Technological University. K.E. Tsiolkovsky (MATI), Lectures,
  • the technical problem of the known solutions when using IMU sensors without additional external tracking is the problem of drift - the accumulation of gyro / inertial sensor errors, a significant deviation of the obtained data from the real ones and, as a result, the impossibility of precise absolute positioning.
  • the technical result is the ability to receive and transmit data on the position of fingers, hands, elbow and shoulder joints to the computer or other device, as well as to carry out tactile feedback by transmitting vibrations to the fingers.
  • the technical result is the ability to fix a virtual model of the hand not in the elbow joint, but in the shoulder and with high accuracy to recognize and transfer to the computer or other device all possible hand movements, including in the horizontal plane.
  • Another advantage over other known devices based only on external tracking is the ability to use outside stationary equipped rooms, which is convenient when using the solution with mobile devices (smartphones) or portable solutions of virtual and augmented reality.
  • the virtual reality glove which contains sensors, gloves located on the fingers, and the sensors are connected to the system board, characterized in that IMU sensors located inside the glove made of fabric are used as sensors, each IMU sensors contain an accelerometer, a gyroscope and a magnetometer, with four IMU sensors located on the penultimate phalanges little finger, ring, middle and index fingers, two IMU sensors located on the first and second phalanges of the thumb, one IMU sensor located on the system board; on the fingers of the gloves are mounted vibration motors, one on each finger, and on all fingers except the big one, the vibration motors are fixed on the penultimate phalanges of the fingers, and on the thumb a vibration motor is fixed on the last phalanx; the glove holds the motherboard where the computing module is located; IMU sensors, vibration motors, battery connected to the motherboard through wires; Combined module with IMU sensor and light sensor located on the wrist of the glove.
  • the motherboard and battery can be made in a single package of translucent plastic.
  • Photodiodes are installed on the motherboard, which are connected to a combined module containing a photodiode and an IMU sensor.
  • the photodiodes are mounted on the system board at its edges.
  • photodiodes are mounted on the system board.
  • An additional shoulder module can be installed on the shoulder, and connected to the system board using a wire through a combined module, which is made containing a connecting connector.
  • An IMU sensor can be installed inside the shoulder module.
  • the shoulder module can be made containing a wireless communication module and configured to connect to the system board over the air, and a battery is also located in the shoulder module.
  • the shoulder module alternatively, contains a microcontroller and strain gauges connected to it.
  • Figure 1 shows a diagram of the device gloves using on the wrist and IMU sensor and without the use of photodiodes.
  • Figure 2 shows a diagram of the device gloves using photodiodes and located on the wrist of the combined module containing the photodiode and the IMU sensor
  • Fig. 3 shows a diagram of the device of the glove without using photodiodes in the system board and located on the wrist of the combined module containing the photodiode and IMU sensor.
  • Figure 4 shows the connection diagram to the glove of the shoulder module.
  • 1 - glove 2 - inertial sensor (sensor) IMU, combining accelerometer, gyroscope and magnetometer, 3 - vibrating motor, designed to transmit vibration sensations, 4 - motherboard, 5 - battery, 6 - wires, 7 - the case that combines the motherboard and battery, 8 - photodiode, 9 - combined module, 10 - shoulder module, 11 - connecting wire between the combined module and the shoulder module.
  • the device ( Figure 2) consists of a glove (1) made of a fabric inside which there are: IMU sensors (2), each of which contains an accelerometer, a gyroscope and a magnetometer, with four IMU sensors (2) located on
  • IMU sensors (2) are located on the first and second phalanges of the thumb, one IMU sensor (2) is located on the system board.
  • the fingers of the gloves (1) also have vibration motors (3) fixed, one on each finger, and on all fingers except the big one, the vibration motors (3) are fixed on the penultimate phalanges of the fingers, and on the thumb a vibration motor (3) is fixed on the last phalanx.
  • the motherboard (4) is fixed on the glove (1), where the computing module is located.
  • the Bluetooth wireless communication module for communication with a computer or other device via radio, is installed on the motherboard (4).
  • the motherboard (4) is mounted on top of the battery (5), which is mounted on the glove (1).
  • IMU sensors (2), vibration motors (3), battery (5) are connected to the system board (4) via wires (b).
  • the motherboard (4) and battery (5) can be made in single body (7) of translucent plastic.
  • the photodiodes (8) can be mounted on the system board (4) at its edges equidistant. There may be four.
  • a module (9) with an IMU sensor and a light sensor is placed on the wrist of the glove (1).
  • An additional advantage of the virtual glove (1) is the use of the shoulder module (10), which is placed on the shoulder (biceps), and connected to the system board with a wire (11) through the module (9), which is performed with a connecting connector (not shown in the drawings ).
  • An IMU sensor can be installed inside the shoulder module (10).
  • the shoulder module (10) can also be implemented in the wireless version, and connect to the computing module via radio (Bluetooth or Wifi).
  • radio Bluetooth or Wifi
  • a battery and a radio module (not shown in the drawings) and a wire (11) are not used in a single module with a sensor (10).
  • the shoulder module (10) can be optionally implemented not with the help of an IMU sensor, but with the help of one or several strain gauges (bend sensors).
  • a microcontroller is installed in the shoulder module (10) to which the resistance strain gages are connected, and through them aggregates the information and transfers it to the main computing module of the motherboard (4).
  • the device works as follows.
  • the claimed solution allows you to receive and transfer data on the position of fingers, hands, elbow and shoulder joints to a computer or other device, and also implement tactile feedback by transmitting vibrations to the fingers. This is achieved as follows.
  • Gyroscopic / inertial sensors are mounted on fingers: on the index, middle, ring fingers and little finger one sensor is installed in the penultimate phalanx of the finger, on the outside of the palm; two sensors are installed on the thumb - on the first and second phalanx, from the outside. Additionally, similar sensors are mounted on the hand, from the outside, and on the wrist, from the outside or inside of the palm.
  • the sensors can be fixed in these positions with a rag glove, sewn into the required positions, or each sensor can be attached independently with a rag (or other material) ring (fastening) worn on the finger / hand / wrist or any other in the way
  • an IMU-sensor can be used, for example:
  • the control module contains microcontroller-based control electronics, a wireless radio module, and a battery.
  • Sensors constantly collect motion data and transmit it to the microcontroller, which transmits this data over the air to a computer or other device.
  • a software driver is installed on the computer (or other device) that uses the data from the sensors (angular velocities and
  • index, middle, ring fingers and little finger in the vertical plane relative to the palm; rotation of the index, middle, ring fingers and little finger in the horizontal plane (palm plane); twist / fold of the thumb relative to the palm in space; rotate / fold the hand (palm) relatively elbow joint; rotation of the elbow joint relative to the shoulder joint.
  • the obtained data is then provided for access as a software application interface (API) and can be used by third-party
  • API software application interface
  • a vibration motor is placed, which is mounted in the same way as using IMU sensors, and also connected to the motherboard and microcontroller using a cable.
  • the application programming interface allows the vibration motor to be launched independently on each of the fingers, setting the required parameters of rotary modulation (vibration frequency / intensity).
  • the command from the software interface is processed by the device driver and, via the radio channel, is transmitted to the microcontroller, which directly controls the vibration motor.
  • the vibration can be controlled by third-party software manufacturers using the application programming interface (API).
  • API application programming interface
  • a vibration motor can be used, for example:
  • one more gyroscopic / inertial sensor can be additionally attached to the shoulder joint, for example, inside the shoulder module (10) ( Figure 4) and connected to the system board (4) using a cable (11) through socket module (9).
  • the data from the sensor is transmitted to the microcontroller, and then through the radio channel to the driver software, where they are used to calculate the rotation of the elbow joint relative to the shoulder joint, as well as the rotation of the elbow joint in a plane perpendicular to it. This allows you to more accurately calculate the position of the brush in space and give additional degrees of freedom to the hand.
  • control module To completely solve the problem of accumulated error in the calculated absolute position of the sensors, support for external "tracking" is used.
  • two or more photodiodes reacting to flashes of light are placed on the control module.
  • the housing for the control module is made of
  • Another photodiode is located next to the IMU-sensor on the wrist.
  • another photodiode is also located along with the IMU-sensor on the shoulder joint.
  • Photodiodes register light signals sent by a pair of special external light emitters installed stationary, stationary and separate from
  • controller motherboard and a set of sensors in the form of gloves or other form, on opposite sides of it.
  • Emitters are programmed in a special way and generate light pulses of a certain length, shape and direction, at certain time intervals. Light can be emitted both in the visible and in the invisible (infrared, ultraviolet) ranges.
  • the emitter can generate a set of short pulses of light, and then generate a narrow strip with a laser mounted on a rotating mechanism, rotating in a horizontal plane.
  • Another emitter can do the same in parallel (but at different time intervals), but in the vertical plane.
  • the time intervals, the sequence of light pulses, the shape and nature of the pulses are known in advance, these data are compared by the software driver with the photodiode registration time of various light pulses, thereby calculating their position relative to the emitters, and so image - the absolute position of the photodiodes in space. This data is then used to
  • photodiode As a photodiode can be used, for example:
  • an external radiator can be used, for example:
  • the claimed solution can be used to animate 3D models of a human (or other) hand in computer programs, as a way to interact with
  • the claimed solution can also be used in medicine, for patients with impaired motility of the hands to track the movements of the hands and fingers and stimulate their activity.
  • Feedback using vibration motors can be used to simulate the sensations of touching objects in virtual reality, when position data Hands and fingers are used to identify collisions with virtual objects and, if they are detected, apply vibration fingers to the corresponding contact point.
  • the degree of vibration may depend on the shape of the contact - the size of the intersection area, or on the characteristics of the virtual object.
  • Another possible scenario of using vibration is to confirm an event with interactions with virtual interfaces — for example, pressing a virtual button.
  • the claimed solution offers a new principle of placement of IMU-sensors and, as a result, high accuracy of the results: the relative position of the sensors allows you to get the necessary angles between the fingers, hand, elbow and shoulder joints; secondly, the location of the sensors and
  • Vibration motors allows you to release the last phalanx of fingers, which is convenient in practical application - the user can use touch screens, perform actions requiring fine motor skills of fingers, etc.
  • strain gauges fold sensors
  • solutions based only on IMU sensors unlike methods based on external optical or other external tracking, this approach completely lacks "blind spots", even if there is no direct view between the LED (photodiode) and the camera (light emitter), the stated solution will still allow quite accurately calculate the position of the arm in space.
  • Another advantage over other known devices based only on external tracking is the ability to use outside stationary equipped rooms, which is convenient when using the solution with mobile devices (smartphones) or portable solutions of virtual and augmented reality.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne les manipulateurs et notamment des gants virtuels destinés à travailler avec des dispositifs interactifs. Le dispositif de l'invention comprend des capteurs disposés sur les doigts du gant, lesdits capteurs étant branchés sur la carte mère. On peut utiliser en tant que capteurs des capteurs IMU disposés à l'intérieur du gant réalisé à partir d'un tissu, chacun des capteurs IMU comprenant un accéléromètre, un gyroscope et un magnétomètre. Les quatre capteurs IMU sont disposés sur les avant-dernières phalanges de l'auriculaire, de l'annulaire, du médian et de l'index, deux capteurs IMU sont disposés sur les première et deuxième phalanges du pouce, un capteur IMU est disposé sur la carte mère. Des moteurs vibratoires sont fixés sur chaque doigt, un pour chaque doigt, et sur tous les doigts, à part le pouce, les moteurs vibratoires sont fixés sur les avant-dernières phalanges des doigts, et sur le pouce le moteur vibratoire est fixé sur la dernière phalange. On a monté une carte mère sur le gant, ladite carte mère comprenant le module de calcul. Les capteurs IMU, les moteurs vibratoires, l'accumulateur sont branchés sur la carte mère au moyen de fils, le module combiné avec un capteur IMU et un capteur lumineux disposé sur la poignée du gant.
PCT/RU2018/000630 2017-10-27 2018-09-28 Gant de réalité virtuelle WO2019083405A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
RU2017137682 2017-10-27
RU2017137682 2017-10-27

Publications (1)

Publication Number Publication Date
WO2019083405A1 true WO2019083405A1 (fr) 2019-05-02

Family

ID=66247568

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/RU2018/000630 WO2019083405A1 (fr) 2017-10-27 2018-09-28 Gant de réalité virtuelle

Country Status (1)

Country Link
WO (1) WO2019083405A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030048312A1 (en) * 1987-03-17 2003-03-13 Zimmerman Thomas G. Computer data entry and manipulation apparatus and method
US20100231505A1 (en) * 2006-05-05 2010-09-16 Haruyuki Iwata Input device using sensors mounted on finger tips
US9060385B1 (en) * 2007-04-20 2015-06-16 Lloyd Douglas Manning Universal bluetooth/wireless glove
RU156547U1 (ru) * 2015-02-17 2015-11-10 Сергей Владимирович Тюлюсов Устройство ввода информации

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030048312A1 (en) * 1987-03-17 2003-03-13 Zimmerman Thomas G. Computer data entry and manipulation apparatus and method
US20100231505A1 (en) * 2006-05-05 2010-09-16 Haruyuki Iwata Input device using sensors mounted on finger tips
US9060385B1 (en) * 2007-04-20 2015-06-16 Lloyd Douglas Manning Universal bluetooth/wireless glove
RU156547U1 (ru) * 2015-02-17 2015-11-10 Сергей Владимирович Тюлюсов Устройство ввода информации

Similar Documents

Publication Publication Date Title
KR101666096B1 (ko) 강화된 제스처 기반 상호작용 시스템 및 방법
RU179301U1 (ru) Перчатка виртуальной реальности
RU187548U1 (ru) Перчатка виртуальной реальности
US10534431B2 (en) Tracking finger movements to generate inputs for computer systems
CN109313493B (zh) 用于基于手的运动和位置控制计算机的装置
CN111638801A (zh) 用于手势辨识的控制器及其手势辨识的方法
CN206162394U (zh) 一种基于触感反馈的用于虚拟现实环境的动作捕捉系统
US20100103104A1 (en) Apparatus for user interface based on wearable computing environment and method thereof
US20200201460A1 (en) Universal Handheld Controller of a Computer System
JP2010108500A (ja) 着用型コンピューティング環境基盤のユーザインターフェース装置およびその方法
RU2670649C1 (ru) Способ изготовления перчатки виртуальной реальности (варианты)
US20220155866A1 (en) Ring device having an antenna, a touch pad, and/or a charging pad to control a computing device based on user motions
US11054923B2 (en) Automatic switching between different modes of tracking user motions to control computer applications
US20230142242A1 (en) Device for Intuitive Dexterous Touch and Feel Interaction in Virtual Worlds
Shigapov et al. Design of digital gloves with feedback for VR
Kao et al. Novel digital glove design for virtual reality applications
KR102162922B1 (ko) 촉각 피드백이 가능한 가상현실 기반의 손 재활 시스템
RU176318U1 (ru) Перчатка виртуальной реальности
RU2673406C1 (ru) Способ изготовления перчатки виртуальной реальности
JP6932267B2 (ja) コントローラ装置
US10409375B1 (en) Apparatus, system, and method for providing localized tactile feedback
WO2019083405A1 (fr) Gant de réalité virtuelle
KR101686585B1 (ko) 비행 가상 시뮬레이터 환경에서 회전형 꼭지 손잡이를 구동하기 위한 손 움직임 추적 시스템
RU186397U1 (ru) Перчатка виртуальной реальности
CN202230440U (zh) 操控计算机空间绘图类软件的控制系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18871345

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18871345

Country of ref document: EP

Kind code of ref document: A1