WO2020067909A2 - A controller worn on a human body and a method of controlling devices with the use of such controller - Google Patents

A controller worn on a human body and a method of controlling devices with the use of such controller Download PDF

Info

Publication number
WO2020067909A2
WO2020067909A2 PCT/PL2019/000074 PL2019000074W WO2020067909A2 WO 2020067909 A2 WO2020067909 A2 WO 2020067909A2 PL 2019000074 W PL2019000074 W PL 2019000074W WO 2020067909 A2 WO2020067909 A2 WO 2020067909A2
Authority
WO
WIPO (PCT)
Prior art keywords
controller
holder
harness
hand
computer device
Prior art date
Application number
PCT/PL2019/000074
Other languages
French (fr)
Other versions
WO2020067909A3 (en
Inventor
Krzysztof Wróblewski
Original Assignee
Matrix Dynamix Sp. Z O. O.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from PL426941A external-priority patent/PL239579B1/en
Priority claimed from PL430730A external-priority patent/PL245298B3/en
Application filed by Matrix Dynamix Sp. Z O. O. filed Critical Matrix Dynamix Sp. Z O. O.
Publication of WO2020067909A2 publication Critical patent/WO2020067909A2/en
Publication of WO2020067909A3 publication Critical patent/WO2020067909A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03547Touch pads, in which fingers can move on a surface
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/211Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20Input arrangements for video game devices
    • A63F13/21Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/214Input arrangements for video game devices characterised by their sensors, purposes or types for locating contacts on a surface, e.g. floor mats or touch pads
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/90Constructional details or arrangements of video game devices not provided for in groups A63F13/20 or A63F13/25, e.g. housing, wiring, connections or cabinets
    • A63F13/98Accessories, i.e. detachable arrangements optional for the use of the video game device, e.g. grip supports of game controllers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1626Constructional details or arrangements for portable computers with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/163Wearable computers, e.g. on a belt
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1632External expansion units, e.g. docking stations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1637Details related to the display arrangement, including those related to the mounting of the display in the housing
    • G06F1/1643Details related to the display arrangement, including those related to the mounting of the display in the housing the display being associated to a digitizer, e.g. laptops that can be used as penpads
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16Constructional details or arrangements
    • G06F1/1613Constructional details or arrangements for portable computers
    • G06F1/1633Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1694Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01Indexing scheme relating to G06F3/01
    • G06F2203/013Force feedback applied to a game

Definitions

  • the object of the invention is to provide a controller, to be worn on a human body, used to control devices equipped with a radio receiver, especially in the systems of virtual, mixed or augmented reality, as well as drones and robots.
  • a game controller consisting of two control units is known from the American patent description US 8267786 B2.
  • the units are equipped with an electronic gyroscope and accelerometer and generate three groups of operational data.
  • the controller is equipped with physical keys and joystick.
  • the controller units are connected to each other by a cable and wirelessly to a computer.
  • a way of controlling virtual objects using a controller where input data is collected from the controller is known from the American patent description US 9041622 B2.
  • Input data including a request for specific actions to be performed by a virtual object is implemented in augmented reality depending on the user perspective, just as actions performed by the controlled object are displayed with respect to the user perspective.
  • a virtual object controlled by a physical controller can be identified from the input data collected from the natural user interface.
  • the controller is equipped with a force feedback mechanism.
  • a system for controlling content displayed in virtual environments is known from the American invention description US 20130342572 Al.
  • a user may interact with the displayed content using predefined gestures.
  • the system includes a display device, such as virtual reality goggles for example, and an operationally connected computer.
  • An input device for use in computer systems to specify a position on a display using radial coordinates, cylindrical coordinates or spherical coordinates is known from the American patent description US 7683883 B2.
  • the device incorporates an ergonomic housing allowing it to be held in a hand, buttons and an accelerometer.
  • the device allows to interact with the 3D environment displayed on a computer display screen.
  • a hand-worn control device dedicated for use with an augmented reality eyepiece is known from the American invention description US 20110221672 Al.
  • the control device contains a control element that generates a control command when activated by the user's finger.
  • Control moment gyroscopes used in satellites to provide a reference point and to counterbalance manoeuvring in all directions, are known from the state of the art. Such gyroscopes have rotors of relatively high weight. It is also known that similar gyroscopes are used in an exercise apparatus such as, for example, a gyroscopic exercise apparatus, comprising: at least one control-moment gyroscope; at least one motion sensor for sensing movement of the apparatus; at least one spindle motor for providing rotation to a rotor of the gyroscope; known from the American patent description US 10252151 B2.
  • a handheld controller for interacting with physical moving objects and graphical user interfaces equipped with a gyroscope unit comprising a minimum of one mechanical gyroscope, the rotor of which is coupled to an electric motor is known from the International Application Form WO 2016195503.
  • a gyroscope unit comprising a minimum of one mechanical gyroscope, the rotor of which is coupled to an electric motor.
  • This enables to simulate motion according to activity and moving objects shown graphical on a screen, which gives the user a three- dimensional physical feedback resulting from the activity of controlled objects.
  • Using the device requires the use of two upper limbs, and due to its size, longer use may be difficult.
  • a system for control of a drone is known from the American patent description US 20140008496 Al.
  • the commands are executed using a handheld device which includes a motion sensor module with a gyro-sensor and an g-sensor for controlling roll, yaw and pitch under a coordinate system.
  • the drone camera image is displayed on the device screen.
  • a handheld system for controlling devices consisting of an audio-video unit and a control unit for controlling the movement of a device is known from the Polish patent description PL 223180 Bl.
  • the AV receiver is fitted to the wrist or forearm, while the control unit is held in a hand and secured with a wrist strap.
  • controllers known from the state of the art do not allow comfortable controlling with only one upper limb, even if all fingers, in particular the thumb, are not functioning properly, which is necessary when using most of known handheld controllers.
  • Devices that allow to imitate the user motions in applications, in particular applications of virtual, augmented or mixed reality, as well as when controlling robots or drones are often characterized by a complex design and, consequently, high production costs.
  • a controller to be worn on a human body comprising a holder for a computer device used in that controller and equipped with at least an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter and receiver, according to the invention, is characterised by the fact that it has a harness for wearing around the hand, where this harness is joined, in its part corresponding to the palm of the hand, to a computer device holder, and this holder has an open part corresponding at least to the part of the touchscreen of the computer device, where this open part is on the side corresponding to the palm of the hand in the area of fingers.
  • the controller comprises a gyroscope unit attached to the harness and comprising at least one mechanical gyroscope.
  • its gyroscope unit comprises four mechanical gyroscopes.
  • each mechanical gyroscope of its gyroscope unit has a rotor connected to a separate electric motor.
  • the holder is connected to the harness by means of an arm, one end of which is connected to the harness at the palm of the hand and the other end is connected to the holder, the arm is connected to the harness and the holder by articulated joints, the gyroscope unit of the controller is fitted between the harness and the arm.
  • the articulated joints between the controller arm and the harness at a position corresponding to the palm area of the hand and between the arm and the holder have at least two axes of rotation each and its arm has an articulated joint at its centre, with the first articulated joint at the centre of the arm and each articulated joint has a pressure screw to lock the arm's position at the ends of the arm.
  • controller's harness is fitted with a spacer in its palm part where the spacer has a rigid base on the outside and an elastic cushion on the inside for supporting on the palm of the hand.
  • controller harness has at least one mounting strap to attach it to the circumference of the hand.
  • controller harness is in the form of a glove with openings exposing fingers.
  • a method of controlling devices equipped with a radio receiver by means of a controller is, according to the invention, characterised by the fact that, in the first stage, the controller's harness is attached to a hand and the computer device, equipped with at least the following sensors: an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter, is mounted in the controller holder with the touchscreen facing the fingers and connected wirelessly to the radio receiver of the controlled device, after which, in the second stage, the controller is moved and the touchscreen is touched with fingers, and with the use of computer device sensors, a point in the space, in which the controller is located, is established, and then information about the controller's position and touches of the computer device's touchscreen is transmitted wirelessly to the receiver of the controlled device and, on the basis of this information, the device is controlled.
  • the controller's harness is attached to a hand and the computer device, equipped with at least the following sensors: an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter, is
  • a smartphone is mounted in the holder, and when the smartphone is wirelessly connected to the radio receiver of the controlled device, buttons are displayed on the display of the smartphone mounted in the holder, and by touching these buttons, the function assigned to them is activated.
  • buttons and points corresponding to where the user touches the touchscreen of the computer device fitted in the controller's holder, on the display of the smartphone fitted in the virtual reality goggles after wireless connection of the computer device fitted in the holder.
  • the application of the invention allows to control a number of devices in a convenient way.
  • the controller is compact and cheap in design and use, especially if the computer device used in it is a smartphone or tablet already in the possession of the user.
  • the solution gives the possibility of precise positioning of the hand in space with easy access to the touch screen. Attaching the holder to the controller's harness by means of an arm allows to position the smartphone used in the controller in an ergonomic position in relation to the hand, which prevents the occurrence of discomfort during long-term use.
  • the controller's component is a computer program installed on a computer device used in this controller, it is possible to optimize it in order to perform specific tasks. Controller enables the application of feedback in applications of virtual reality consisting in starting the vibration engine of the smartphone in certain situations, such as a virtual obstacle.
  • the vibration engine starts when the ball is bounced back.
  • the mechanism of biological feedback also allows the device to be used in the rehabilitation of people with unilateral paresis because the patient does not need to have a fully functional hand to perform exercises in virtual reality.
  • the use of a spacer pad and stiffener also affects the comfort of use, increasing the possibility of setting the smartphone in different positions relative to the hand.
  • the spacer pad with its cushion contributes to a more stable fixing of the controller on the hand and prevents it from slipping, which is important, especially when used in games requiring vigorous movements.
  • the construction of the holder and its connection with the harness by means of an articulated arm ensures precise control and prevents the occurrence of unwanted movements occurring in controllers held directly in hands.
  • controller is used to control a drone by hand movement and a specific function is activated using the touch screen, it is possible to conveniently control it in six degrees of freedom by setting the position of a point in space.
  • buttons adjusted for controlling a specific device which gives the possibility of its wide adaptation to many applications.
  • virtual reality it is possible to display on the display of a smartphone mounted in goggles the preview of buttons and places where the user touches the display mounted in the controller holder. Finger tracking makes it easy to use the controller in virtual reality applications.
  • the application used in the controller allows you to extend its functions and personalize the way you control it, for example by changing the layout of buttons and their size, which allows you to make additional facilitations for people with dysfunction of the upper limbs.
  • the controller allows for comfortable control of devices not only with one upper limb, but also in case of lack of all fingers being efficient, including a thumb.
  • the controller contains mechanical gyroscopes
  • they allow not only to stabilize the controller during controlling, but also to obtain physical biological feedback or force feedback.
  • Rotating rotors of mechanical gyroscopes cause the creation of a centrifugal force depending on the given rotational speed controlled by the computer device used in the controller. This allows to change the value of the resulting centrifugal force, the vector of which has its sense opposite to the hand movement performed by the user with the controller attached, which allows to simulate the weight, resistance and speed of objects, which is particularly beneficial in applications of virtual reality.
  • Feedback will allow precise and safe drone control, allowing not only more stable control, but also more accurate information about the speed of the controlled object and the manoeuvres being performed. Force feedback will provide you with additional stimuli that will positively impact your experience of using the controller in games and applications in virtual reality. This solution also allows, after the development of dedicated sets of exercises, for wide application in rehabilitation. This applies, among others, to patients with upper limb dysfunctions, as well as patients with unilateral paresis. Thanks to the controlled centrifugal force generated by the system of rotating rotors of mechanical gyros, the patient will be able to perform exercises that would not be possible without this system. Additionally, thanks to stimuli informing the patient about the correctness of exercises performed, the effectiveness of therapy is improved.
  • the controller's insert attached peripherally from the inner side of the harness allows to attach it to other parts of the body, such as for example shank or forearm.
  • the location of the smartphone allows you to use its camera to recognize hand gestures and use them to control devices.
  • Fig. 1 shows a perspective view of the controller in the first embodiment
  • Fig. 2 - a side view of the same controller
  • Fig. 3 - a top view with a broken-out section of the same controller with the pad turned eccentrically
  • Fig. 4 - a top view of the same controller attached to a hand
  • Fig. 5 - a top view with a broken-out section of the controller in the second embodiment
  • Fig. 7 - a side view with a broken-out section of the same controller
  • Fig. 1 shows a perspective view of the controller in the first embodiment
  • Fig. 2 - a side view of the same controller
  • Fig. 3 - a top view with a broken-out section of the same controller with the pad turned eccentrically
  • Fig. 4 - a top view of the same controller attached to a hand
  • Fig. 5 - a top view with
  • a human body worn controller with a computer device comprises a hand worn harness 1, which is attached to a computer device holder 2, in its part corresponding to the palm of the hand.
  • the computer device used in the controller is a smartphone 3 comprising an accelerometer, an electronic gyroscope, a capacitive touchscreen and a camera.
  • the connection of the holder 2 with the harness 1 is made by means of an arm 4.
  • the first end of the arm 4 is connected to the harness 1 and the second end is connected to the holder 2.
  • the connections of the arm 4 with the harness 1 and the holder 2 are made as articulated joints with two axes of rotation.
  • the arm 4 comprises the first cylinder articulated joint 5 in its central part.
  • the harness 1 of the controller comprises in its part, corresponding to the palm of the hand, a spacer pad 6, which comprises, from the inside, a cushion 7 of resilient polyurethane foam and, from the outside, a plastic rigid base 8.
  • the connection between the arm 4 and the harness 1 is placed on the rigid base 8.
  • the cushion 7 comprises a recess 9.
  • the cushion 7 is detachably connected by means of Velcro fasteners 14 with a base 8' seated eccentrically on the rigid base 8. This positioning of the cushion 7 ensures a better fit of the harness 1 to various hand shapes and increases the ergonomics of the controller.
  • the holder 2 comprises a spring clip 10 for clamping the smartphone 3.
  • the harness 1 comprises two straps 11 and 12 for fastening at the circumference of the hand, the first of which 11 is fastened at the circumference of the hand between the thumb and other fingers using a buckle 13 and the second strap 12 is fastened at the circumference of the hand near the wrist using a Velcro fastener 14.
  • the harness 1 has a stiffener 15 on the first strap 11, in the part corresponding to the back of the hand, which is lined with elastic foam on the side intended for the back of the hand.
  • the stiffener 15 acts as a slider for adjusting the circumference of the first strap 11 which is locked in the buckle 13.
  • the straps 11 and 12 are connected to each other by means of a connector 16.
  • the use of the stiffener 15 in combination with the spacer pad 6 allows stable positioning of the controller on the hand, preventing it from moving, when making vigorous movements and increasing comfort during long-term use.
  • Articulated joints between the ends of the arm 4 and the harness 1 and the holder 2 allow to place the smartphone 3 in a position allowing for comfortable control with four or five fingers depending on its position in relation to the hand.
  • the first articulated joint 5 and the articulated joints of the arm 4 are fitted with clamping screws 17 that allow the arm 4 and the holder 2 to be locked in a fixed position to prevent the smartphone 3 mounted in the holder 2 from sliding against the hand during use.
  • the harness 1 has a strap 18, between the first strap 11 and the second strap 12, which connects the straps 11 and 12 on the side of the spacer pad 6, additionally stabilizing the attachment of the harness 1 on the hand.
  • the spacer pad 6 of the controller comprises a gyroscope unit 19 inside its base 8, in which one mechanical gyroscope 20 is mounted, comprising a rotor 21 supported by a fixed suspension 22.
  • the rotor 21 is driven by an electric motor 23.
  • the speed of the rotor 21 is controlled by the smartphone 3 mounted in the holder 2 of the controller.
  • the design is the same as in the first embodiment.
  • the spacer pad 6 of the controller comprises a gyroscope unit 19 inside its base 8, in which four mechanical gyroscopes 20 are mounted, each of which comprises a rotor 21 supported by a suspension 22 comprising a second articulated joint 24 with angle of inclination electromechanically adjustable in two axes.
  • the rotor 21 of each gyroscope is connected to a separate electric motor 23.
  • the design is the same as in the first embodiment.
  • the harness 1 is made in the form of a glove 25.
  • the glove 25 is connected, in its part corresponding to the palm of the hand, to the holder 2 of a computer device.
  • the smartphone 3 is used as a computer device.
  • the glove 25 has a spacer pad 6 in its part corresponding to the palm of the hand.
  • the connection between the glove 25 and the holder 2 is made by means of the arm 4, one end of which is connected to the spacer pad 6 of the glove 25 and the other end to the holder 2.
  • the design of the holder 2, the arm 4 and the spacer pad 6 is the same as in the first embodiment.
  • the glove 25 has a first strap 11 and a second strap 12 to stabilize the glove on the hand and adjust its size.
  • the harness 1 of the controller is additionally equipped with an elastic foam insert 26 placed peripherally from the inside of the straps 11 and 12 of the harness 1.
  • the insert 26 is used after disconnecting the cushion 7 and in addition, to attaching the harness 1 to the hand it also allows its attachment to the forearm.
  • the design is the same as in the first embodiment.
  • the harness 1 has one strap 11.
  • the design of the harness 1 is the same as in the first embodiment.
  • the controller worn on the human body is in the form of a glove 25 detachably connected, in its part corresponding to the palm of the hand, with a silicone holder 2.
  • the smartphone 3 as in the first embodiment is placed in the holder 2.
  • the glove 25 has two holes 27, one for the thumb and one for the other fingers.
  • the silicone holder 2 has a pocket 28 for the smartphone 3 and a wing 29 fastened to this pocket 28 on the circumference of the smartphone 3 with Velcro fasteners 14.
  • the glove 25 has three straps 11, 12 and 30 fastened with Velcro fasteners 14 for attaching to the hand.
  • the straps 11, 12 and 30 are fastened with Velcro fasteners 14.
  • the first strap 11 is fastened around the circumference of the hand in the area between the thumb and the other fingers
  • the second strap 12 is fastened around the circumference of the wrist
  • the third strap 30 is fastened around the forearm.
  • the circumference of the glove 25 may be adjusted with the straps 11, 12 and 30.
  • the adjustable circumference of the glove 25, as well as the hole 27 for four fingers, also allows it to be used as a strap for fixing for example, on the shank or shoulder.
  • the controller is equipped with a wrist stabilizer 31 in the form of a rigid tape running from the glove cuff 25, from the side corresponding to the palm of the hand, to the midhand.
  • the holder 2 has an elastic pressure straps 32, which are attached with Velcro fasteners 14 to the back of the glove 25 to ensure a more stable attachment of the smartphone 3 to the holder 2.
  • the glove 25 has a battery compartment 33, which makes it possible to extend the working time of the controller.
  • the glove 25 is connected to the holder 2 by means of a snap fastener.
  • the way of controlling devices equipped with a radio receiver is done using the controller described in the third embodiment, and the smartphone placed in the virtual reality goggles is controlled.
  • the harness 1 of the controller is attached to the hand, and the smartphone 3 is mounted in the holder 2 so that its touchscreen is facing the fingers.
  • the smartphone 3 is equipped with the following sensors: a camera, electronic gyroscope, accelerometer, magnetometer, capacitive touch screen, as well as a radio receiver and transmitter.
  • the smartphone 3 connects wirelessly with the second smartphone mounted in the virtual reality goggles.
  • the display of the smartphone 3 mounted in the holder 2 shows the shapes acting as the buttons 34 of the controller.
  • the smartphone mounted in the virtual reality goggles displays a preview of the buttons 34 displayed on the smartphone 3 mounted in the holder 2 of the controller.
  • the preview of the buttons 34 is reduced in size and also shows the points 35 of touching the smartphone 3 mounted in the holder 2 of the controller, so that the user can see the position of his/her fingers and there is no problem with finding the buttons 34 on the smartphone 3 of the controller.
  • the hand and the entire upper limb move with the controller attached and the touchscreen is touched with fingers in the area of the displayed buttons 34 as well as predefined gestures are made.
  • the controller's position in the space and the touches and gestures made using the touchscreen are then transmitted wirelessly to a smartphone mounted in the virtual reality goggles.
  • virtual reality objects are controlled, for example, a character in a game. If an object encounters virtual obstacles, the vibration engine starts.
  • gestures are made above the touchscreen of the smartphone 3 and its front camera. These gestures are recorded and recognized by the front camera of the smartphone 3 mounted in the controller holder, and the information about the gestures made is sent to the smartphone mounted in the virtual reality goggles.
  • the rotors 21 of the mechanical gyroscopes 20 are rotated and the position of the controller is stabilised. By means of the second articulated joints 24, electromechanically controlled, of the suspensions 22 of the mechanical gyroscopes 20, the angle of inclination of the rotors 21 is adjusted.
  • the rotors 21 When being rotated, the rotors 21 also allow to obtain a force feedback informing the user about the speed of movement of the controlled object, or about being faced with a virtual obstacle, and also allow to simulate the weight, resistance and speed of objects in virtual reality.
  • the rotational speed of rotors 21 and their inclination is controlled by the smartphone 3 mounted in the holder 2 of the controller.
  • the method is implemented using dedicated applications installed on the smartphone 3 mounted in the holder 2 of the controller and in the virtual reality goggles.
  • a drone is controlled.
  • the controller is used according to the first embodiment.
  • the harness 1 of the controller is attached to the hand, and the smartphone 3 is mounted in its holder 2 as in the first embodiment, and it is wirelessly connected to the drone and the display of the smartphone 3 shows the buttons 34.
  • a hand movement is made with the controller attached and the sensors of the smartphone 3 determines the point in the space in which the controller is located.
  • the touchscreen of smartphone 3 is touched with fingers and the predefined gestures are made or the buttons 34 displayed on the display are touched, which are assigned to specific drone functions.
  • Information about the controller's position in the space and the buttons 34 or gestures touched on the touchscreen are sent wirelessly to the drone.
  • the drone moves in a manner corresponding to the movements made by the user's hand with the controller attached.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Heart & Thoracic Surgery (AREA)
  • User Interface Of Digital Computer (AREA)
  • Position Input By Displaying (AREA)

Abstract

A controller according to the invention, characterised by the fact that, it comprises a harness (1) for attaching on the circumference of the hand, whereby this harness (1) is connected in its part corresponding to the palm of the hand to a holder (2) for a computer device. The holder (2) comprises an exposed part corresponding at least to a part of a touchscreen of the computer device, where that exposed part is on the side corresponding to the palm of the hand in the area of fingers. A method of controlling according to the invention, characterised by the fact that, in the first stage, the harness (1) of the controller is attached to a hand, and the computer device equipped with at least the following sensors: an accelerometer, an electronic gyroscope, a camera, a touch screen, and a radio transmitter, is mounted in the holder (2) of the controller with the touch screen facing the fingers and it is connected wirelessly to the radio receiver of the controlled device. In the second stage, the controller is being moved and the touchpad is finger touched, and a point in the space where the controller is located is determined with the use of computer device sensors, and then information about the controller position and touches on the touchscreen of the computer device is wirelessly transmitted to the receiver of the controlled device and, on the basis of this information, the device is controlled.

Description

A controller worn on a human body and a method of controlling devices with the use of such controller
The object of the invention is to provide a controller, to be worn on a human body, used to control devices equipped with a radio receiver, especially in the systems of virtual, mixed or augmented reality, as well as drones and robots.
A game controller consisting of two control units is known from the American patent description US 8267786 B2. The units are equipped with an electronic gyroscope and accelerometer and generate three groups of operational data. The controller is equipped with physical keys and joystick. The controller units are connected to each other by a cable and wirelessly to a computer.
A way of controlling virtual objects using a controller where input data is collected from the controller is known from the American patent description US 9041622 B2. Input data including a request for specific actions to be performed by a virtual object is implemented in augmented reality depending on the user perspective, just as actions performed by the controlled object are displayed with respect to the user perspective. A virtual object controlled by a physical controller can be identified from the input data collected from the natural user interface. The controller is equipped with a force feedback mechanism.
A system for controlling content displayed in virtual environments is known from the American invention description US 20130342572 Al. A user may interact with the displayed content using predefined gestures. The system includes a display device, such as virtual reality goggles for example, and an operationally connected computer.
An input device for use in computer systems to specify a position on a display using radial coordinates, cylindrical coordinates or spherical coordinates is known from the American patent description US 7683883 B2. The device incorporates an ergonomic housing allowing it to be held in a hand, buttons and an accelerometer. The device allows to interact with the 3D environment displayed on a computer display screen. A hand-worn control device dedicated for use with an augmented reality eyepiece is known from the American invention description US 20110221672 Al. The control device contains a control element that generates a control command when activated by the user's finger.
Control moment gyroscopes, used in satellites to provide a reference point and to counterbalance manoeuvring in all directions, are known from the state of the art. Such gyroscopes have rotors of relatively high weight. It is also known that similar gyroscopes are used in an exercise apparatus such as, for example, a gyroscopic exercise apparatus, comprising: at least one control-moment gyroscope; at least one motion sensor for sensing movement of the apparatus; at least one spindle motor for providing rotation to a rotor of the gyroscope; known from the American patent description US 10252151 B2.
In addition, a handheld controller for interacting with physical moving objects and graphical user interfaces, equipped with a gyroscope unit comprising a minimum of one mechanical gyroscope, the rotor of which is coupled to an electric motor is known from the International Application Form WO 2016195503. This enables to simulate motion according to activity and moving objects shown graphical on a screen, which gives the user a three- dimensional physical feedback resulting from the activity of controlled objects. Using the device requires the use of two upper limbs, and due to its size, longer use may be difficult.
A system for control of a drone is known from the American patent description US 20140008496 Al. The commands are executed using a handheld device which includes a motion sensor module with a gyro-sensor and an g-sensor for controlling roll, yaw and pitch under a coordinate system. The drone camera image is displayed on the device screen.
A handheld system for controlling devices, consisting of an audio-video unit and a control unit for controlling the movement of a device is known from the Polish patent description PL 223180 Bl. The AV receiver is fitted to the wrist or forearm, while the control unit is held in a hand and secured with a wrist strap.
The controllers known from the state of the art do not allow comfortable controlling with only one upper limb, even if all fingers, in particular the thumb, are not functioning properly, which is necessary when using most of known handheld controllers. Devices that allow to imitate the user motions in applications, in particular applications of virtual, augmented or mixed reality, as well as when controlling robots or drones, are often characterized by a complex design and, consequently, high production costs. In addition, nowadays, for full control without any significant limitations, it is necessary to use dedicated controllers for specific applications.
A controller to be worn on a human body, comprising a holder for a computer device used in that controller and equipped with at least an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter and receiver, according to the invention, is characterised by the fact that it has a harness for wearing around the hand, where this harness is joined, in its part corresponding to the palm of the hand, to a computer device holder, and this holder has an open part corresponding at least to the part of the touchscreen of the computer device, where this open part is on the side corresponding to the palm of the hand in the area of fingers.
Preferably, the controller comprises a gyroscope unit attached to the harness and comprising at least one mechanical gyroscope.
Further advantages are obtained if its gyroscope unit comprises four mechanical gyroscopes.
Further advantages are obtained if each mechanical gyroscope of its gyroscope unit has a rotor connected to a separate electric motor.
Further advantages are obtained if the holder is connected to the harness by means of an arm, one end of which is connected to the harness at the palm of the hand and the other end is connected to the holder, the arm is connected to the harness and the holder by articulated joints, the gyroscope unit of the controller is fitted between the harness and the arm.
Further advantages are obtained if the articulated joints between the controller arm and the harness at a position corresponding to the palm area of the hand and between the arm and the holder, have at least two axes of rotation each and its arm has an articulated joint at its centre, with the first articulated joint at the centre of the arm and each articulated joint has a pressure screw to lock the arm's position at the ends of the arm.
Further advantages are obtained when the controller's harness is fitted with a spacer in its palm part where the spacer has a rigid base on the outside and an elastic cushion on the inside for supporting on the palm of the hand.
Further advantages are obtained if the controller harness has at least one mounting strap to attach it to the circumference of the hand.
Further benefits are obtained if the controller harness is in the form of a glove with openings exposing fingers.
A method of controlling devices equipped with a radio receiver by means of a controller is, according to the invention, characterised by the fact that, in the first stage, the controller's harness is attached to a hand and the computer device, equipped with at least the following sensors: an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter, is mounted in the controller holder with the touchscreen facing the fingers and connected wirelessly to the radio receiver of the controlled device, after which, in the second stage, the controller is moved and the touchscreen is touched with fingers, and with the use of computer device sensors, a point in the space, in which the controller is located, is established, and then information about the controller's position and touches of the computer device's touchscreen is transmitted wirelessly to the receiver of the controlled device and, on the basis of this information, the device is controlled.
Preferably, in the first stage, a smartphone is mounted in the holder, and when the smartphone is wirelessly connected to the radio receiver of the controlled device, buttons are displayed on the display of the smartphone mounted in the holder, and by touching these buttons, the function assigned to them is activated.
Further advantages are obtained when a virtual, mixed or augmented reality device is controlled.
Further advantages are obtained when, after installing a computer device in the holder, it is connected wirelessly with a smartphone mounted in a virtual reality goggle, then in the second stage a hand, on which the controller is worn, is moved and its touchscreen is touched with fingers, and its sensors are used to determine the point in the space where the controller is located, and then the information about the location of the controller and the touches of the touchscreen is wirelessly send to the receiver of the smartphone mounted in the virtual reality goggle and on the basis of this information a virtual object is controlled.
Further advantages are obtained by displaying a reduced image of the buttons and points, corresponding to where the user touches the touchscreen of the computer device fitted in the controller's holder, on the display of the smartphone fitted in the virtual reality goggles after wireless connection of the computer device fitted in the holder.
Further advantages are obtained if in the second stage the rotor of at least one mechanical gyroscope is rotated.
The application of the invention allows to control a number of devices in a convenient way. The controller is compact and cheap in design and use, especially if the computer device used in it is a smartphone or tablet already in the possession of the user. The solution gives the possibility of precise positioning of the hand in space with easy access to the touch screen. Attaching the holder to the controller's harness by means of an arm allows to position the smartphone used in the controller in an ergonomic position in relation to the hand, which prevents the occurrence of discomfort during long-term use. Because the controller's component is a computer program installed on a computer device used in this controller, it is possible to optimize it in order to perform specific tasks. Controller enables the application of feedback in applications of virtual reality consisting in starting the vibration engine of the smartphone in certain situations, such as a virtual obstacle. For example, when playing tennis in virtual reality, the vibration engine starts when the ball is bounced back. The mechanism of biological feedback also allows the device to be used in the rehabilitation of people with unilateral paresis because the patient does not need to have a fully functional hand to perform exercises in virtual reality. The use of a spacer pad and stiffener also affects the comfort of use, increasing the possibility of setting the smartphone in different positions relative to the hand. In addition to the positive effect on the comfort of use, the spacer pad with its cushion contributes to a more stable fixing of the controller on the hand and prevents it from slipping, which is important, especially when used in games requiring vigorous movements. The construction of the holder and its connection with the harness by means of an articulated arm ensures precise control and prevents the occurrence of unwanted movements occurring in controllers held directly in hands.
If the controller is used to control a drone by hand movement and a specific function is activated using the touch screen, it is possible to conveniently control it in six degrees of freedom by setting the position of a point in space.
On the display of the smartphone mounted in the holder it is possible to display buttons adjusted for controlling a specific device, which gives the possibility of its wide adaptation to many applications. When used in virtual reality, it is possible to display on the display of a smartphone mounted in goggles the preview of buttons and places where the user touches the display mounted in the controller holder. Finger tracking makes it easy to use the controller in virtual reality applications. The application used in the controller allows you to extend its functions and personalize the way you control it, for example by changing the layout of buttons and their size, which allows you to make additional facilitations for people with dysfunction of the upper limbs.
The controller allows for comfortable control of devices not only with one upper limb, but also in case of lack of all fingers being efficient, including a thumb.
In the version in which the controller contains mechanical gyroscopes, they allow not only to stabilize the controller during controlling, but also to obtain physical biological feedback or force feedback. Rotating rotors of mechanical gyroscopes cause the creation of a centrifugal force depending on the given rotational speed controlled by the computer device used in the controller. This allows to change the value of the resulting centrifugal force, the vector of which has its sense opposite to the hand movement performed by the user with the controller attached, which allows to simulate the weight, resistance and speed of objects, which is particularly beneficial in applications of virtual reality.
Feedback will allow precise and safe drone control, allowing not only more stable control, but also more accurate information about the speed of the controlled object and the manoeuvres being performed. Force feedback will provide you with additional stimuli that will positively impact your experience of using the controller in games and applications in virtual reality. This solution also allows, after the development of dedicated sets of exercises, for wide application in rehabilitation. This applies, among others, to patients with upper limb dysfunctions, as well as patients with unilateral paresis. Thanks to the controlled centrifugal force generated by the system of rotating rotors of mechanical gyros, the patient will be able to perform exercises that would not be possible without this system. Additionally, thanks to stimuli informing the patient about the correctness of exercises performed, the effectiveness of therapy is improved.
The controller's insert attached peripherally from the inner side of the harness allows to attach it to other parts of the body, such as for example shank or forearm. The location of the smartphone allows you to use its camera to recognize hand gestures and use them to control devices.
The controller attached to the human body according to the invention in an exemplary embodiment is shown in the drawing, in which Fig. 1 shows a perspective view of the controller in the first embodiment, Fig. 2 - a side view of the same controller, Fig. 3 - a top view with a broken-out section of the same controller with the pad turned eccentrically, Fig. 4 - a top view of the same controller attached to a hand, Fig. 5 - a top view with a broken-out section of the controller in the second embodiment, Fig. 6 - a bottom view with a broken-out section of the controller in the third embodiment, Fig. 7 - a side view with a broken-out section of the same controller, Fig. 8 - a side view of the controller in the fourth embodiment, Fig. 9 - a side view with a broken-out section of the controller in the fifth embodiment, Fig. 10 - a front view the same controller, Fig. 11 - the same controller attached on a forearm, Fig. 12 - a side view of the controller in the sixth embodiment, Fig. 13 - a top view of the controller in the sixth embodiment, Fig. 14 - a bottom view of the same controller, Fig. 15 - preview of buttons displayed on the display of the smartphone mounted in the controller, Fig. 16 - preview of buttons displayed stereo scopically on the display of the smartphone mounted in virtual reality goggles, Fig. 17 - preview of buttons from Fig. 15 shown magnified.
In the first embodiment shown in Figures 1 to 6, a human body worn controller with a computer device comprises a hand worn harness 1, which is attached to a computer device holder 2, in its part corresponding to the palm of the hand. The computer device used in the controller is a smartphone 3 comprising an accelerometer, an electronic gyroscope, a capacitive touchscreen and a camera. The connection of the holder 2 with the harness 1 is made by means of an arm 4. The first end of the arm 4 is connected to the harness 1 and the second end is connected to the holder 2. The connections of the arm 4 with the harness 1 and the holder 2 are made as articulated joints with two axes of rotation. The arm 4 comprises the first cylinder articulated joint 5 in its central part. The harness 1 of the controller comprises in its part, corresponding to the palm of the hand, a spacer pad 6, which comprises, from the inside, a cushion 7 of resilient polyurethane foam and, from the outside, a plastic rigid base 8. The connection between the arm 4 and the harness 1 is placed on the rigid base 8. In its central part, the cushion 7 comprises a recess 9. The cushion 7 is detachably connected by means of Velcro fasteners 14 with a base 8' seated eccentrically on the rigid base 8. This positioning of the cushion 7 ensures a better fit of the harness 1 to various hand shapes and increases the ergonomics of the controller. The holder 2 comprises a spring clip 10 for clamping the smartphone 3. The harness 1 comprises two straps 11 and 12 for fastening at the circumference of the hand, the first of which 11 is fastened at the circumference of the hand between the thumb and other fingers using a buckle 13 and the second strap 12 is fastened at the circumference of the hand near the wrist using a Velcro fastener 14. The harness 1 has a stiffener 15 on the first strap 11, in the part corresponding to the back of the hand, which is lined with elastic foam on the side intended for the back of the hand. The stiffener 15 acts as a slider for adjusting the circumference of the first strap 11 which is locked in the buckle 13. The straps 11 and 12 are connected to each other by means of a connector 16. The use of the stiffener 15 in combination with the spacer pad 6 allows stable positioning of the controller on the hand, preventing it from moving, when making vigorous movements and increasing comfort during long-term use. Articulated joints between the ends of the arm 4 and the harness 1 and the holder 2 allow to place the smartphone 3 in a position allowing for comfortable control with four or five fingers depending on its position in relation to the hand. The first articulated joint 5 and the articulated joints of the arm 4 are fitted with clamping screws 17 that allow the arm 4 and the holder 2 to be locked in a fixed position to prevent the smartphone 3 mounted in the holder 2 from sliding against the hand during use. The harness 1 has a strap 18, between the first strap 11 and the second strap 12, which connects the straps 11 and 12 on the side of the spacer pad 6, additionally stabilizing the attachment of the harness 1 on the hand.
In the second embodiment shown in Figure 5, the spacer pad 6 of the controller comprises a gyroscope unit 19 inside its base 8, in which one mechanical gyroscope 20 is mounted, comprising a rotor 21 supported by a fixed suspension 22. The rotor 21 is driven by an electric motor 23. The speed of the rotor 21 is controlled by the smartphone 3 mounted in the holder 2 of the controller. For the remaining part, the design is the same as in the first embodiment.
In the third embodiment, shown in Figures 6 to 7, the spacer pad 6 of the controller comprises a gyroscope unit 19 inside its base 8, in which four mechanical gyroscopes 20 are mounted, each of which comprises a rotor 21 supported by a suspension 22 comprising a second articulated joint 24 with angle of inclination electromechanically adjustable in two axes. The rotor 21 of each gyroscope is connected to a separate electric motor 23. For the remaining part, the design is the same as in the first embodiment.
In the fourth embodiment, shown in Figure 8, the harness 1 is made in the form of a glove 25. The glove 25 is connected, in its part corresponding to the palm of the hand, to the holder 2 of a computer device. The smartphone 3 is used as a computer device. The glove 25 has a spacer pad 6 in its part corresponding to the palm of the hand. The connection between the glove 25 and the holder 2 is made by means of the arm 4, one end of which is connected to the spacer pad 6 of the glove 25 and the other end to the holder 2. The design of the holder 2, the arm 4 and the spacer pad 6 is the same as in the first embodiment. The glove 25 has a first strap 11 and a second strap 12 to stabilize the glove on the hand and adjust its size.
In the fifth embodiment, shown in Figures 9 to 11, the harness 1 of the controller is additionally equipped with an elastic foam insert 26 placed peripherally from the inside of the straps 11 and 12 of the harness 1. The insert 26 is used after disconnecting the cushion 7 and in addition, to attaching the harness 1 to the hand it also allows its attachment to the forearm. For the remaining part, the design is the same as in the first embodiment.
In the sixth embodiment, shown in Figure 12, the harness 1 has one strap 11. For the remaining part, the design of the harness 1 is the same as in the first embodiment.
In the seventh embodiment, shown in Figures 13 to 14, according to the invention the controller worn on the human body is in the form of a glove 25 detachably connected, in its part corresponding to the palm of the hand, with a silicone holder 2. The smartphone 3 as in the first embodiment is placed in the holder 2. Once attached to the hand, the glove 25 has two holes 27, one for the thumb and one for the other fingers. The silicone holder 2 has a pocket 28 for the smartphone 3 and a wing 29 fastened to this pocket 28 on the circumference of the smartphone 3 with Velcro fasteners 14. The glove 25 has three straps 11, 12 and 30 fastened with Velcro fasteners 14 for attaching to the hand. The straps 11, 12 and 30 are fastened with Velcro fasteners 14. The first strap 11 is fastened around the circumference of the hand in the area between the thumb and the other fingers, the second strap 12 is fastened around the circumference of the wrist and the third strap 30 is fastened around the forearm. The circumference of the glove 25 may be adjusted with the straps 11, 12 and 30. The adjustable circumference of the glove 25, as well as the hole 27 for four fingers, also allows it to be used as a strap for fixing for example, on the shank or shoulder. The controller is equipped with a wrist stabilizer 31 in the form of a rigid tape running from the glove cuff 25, from the side corresponding to the palm of the hand, to the midhand. In addition, the holder 2 has an elastic pressure straps 32, which are attached with Velcro fasteners 14 to the back of the glove 25 to ensure a more stable attachment of the smartphone 3 to the holder 2. In its lower part, the glove 25 has a battery compartment 33, which makes it possible to extend the working time of the controller. The glove 25 is connected to the holder 2 by means of a snap fastener.
In the first embodiment, the way of controlling devices equipped with a radio receiver is done using the controller described in the third embodiment, and the smartphone placed in the virtual reality goggles is controlled. The harness 1 of the controller is attached to the hand, and the smartphone 3 is mounted in the holder 2 so that its touchscreen is facing the fingers. The smartphone 3 is equipped with the following sensors: a camera, electronic gyroscope, accelerometer, magnetometer, capacitive touch screen, as well as a radio receiver and transmitter. Then, the smartphone 3 connects wirelessly with the second smartphone mounted in the virtual reality goggles. The display of the smartphone 3 mounted in the holder 2 shows the shapes acting as the buttons 34 of the controller. The smartphone mounted in the virtual reality goggles displays a preview of the buttons 34 displayed on the smartphone 3 mounted in the holder 2 of the controller. The preview of the buttons 34 is reduced in size and also shows the points 35 of touching the smartphone 3 mounted in the holder 2 of the controller, so that the user can see the position of his/her fingers and there is no problem with finding the buttons 34 on the smartphone 3 of the controller. Then, in the second stage, the hand and the entire upper limb move with the controller attached and the touchscreen is touched with fingers in the area of the displayed buttons 34 as well as predefined gestures are made. The controller's position in the space and the touches and gestures made using the touchscreen are then transmitted wirelessly to a smartphone mounted in the virtual reality goggles. Based on the information sent, virtual reality objects are controlled, for example, a character in a game. If an object encounters virtual obstacles, the vibration engine starts. In addition, in the second stage, gestures are made above the touchscreen of the smartphone 3 and its front camera. These gestures are recorded and recognized by the front camera of the smartphone 3 mounted in the controller holder, and the information about the gestures made is sent to the smartphone mounted in the virtual reality goggles. In addition, in the second stage, the rotors 21 of the mechanical gyroscopes 20 are rotated and the position of the controller is stabilised. By means of the second articulated joints 24, electromechanically controlled, of the suspensions 22 of the mechanical gyroscopes 20, the angle of inclination of the rotors 21 is adjusted. When being rotated, the rotors 21 also allow to obtain a force feedback informing the user about the speed of movement of the controlled object, or about being faced with a virtual obstacle, and also allow to simulate the weight, resistance and speed of objects in virtual reality. The rotational speed of rotors 21 and their inclination is controlled by the smartphone 3 mounted in the holder 2 of the controller. The method is implemented using dedicated applications installed on the smartphone 3 mounted in the holder 2 of the controller and in the virtual reality goggles.
In the second embodiment, according to the invention, a drone is controlled. The controller is used according to the first embodiment. In the first stage, the harness 1 of the controller is attached to the hand, and the smartphone 3 is mounted in its holder 2 as in the first embodiment, and it is wirelessly connected to the drone and the display of the smartphone 3 shows the buttons 34. Then in the second stage, a hand movement is made with the controller attached and the sensors of the smartphone 3 determines the point in the space in which the controller is located. At the same time the touchscreen of smartphone 3 is touched with fingers and the predefined gestures are made or the buttons 34 displayed on the display are touched, which are assigned to specific drone functions. Information about the controller's position in the space and the buttons 34 or gestures touched on the touchscreen are sent wirelessly to the drone. The drone moves in a manner corresponding to the movements made by the user's hand with the controller attached.
List of designations
1 - Harness 18 - Securing strap
2 - Holder 19 - Gyroscope unit
3 - Smartphone 20 - Mechanical gyroscope
4 - Arm 21 - Rotor
5 - First articulated joint 22 - Suspension
6 - Pad 23 - Electric motor Cushion 24 - Second articulated joint
Base 25 - Glove
Base 26 - Insert
Recess 27 - Hole
Clip 28 - Pocket
First strap 29 - Wing
Second strap 30 - Third strap
Buckle 31 - Stabiliser
Velcro fastener 32 - Pressure strap
Stiffener 33 - Compartment
Connector 34 - Button
Screw 35 - Points

Claims

Patent claims
1. A human body worn controller comprising a holder for a computer device used by that controller and equipped with at least an accelerometer, an electronic gyroscope, a camera, a touchscreen, and a radio transmitter and receiver, characterised in that, it has a harness (1) for attaching on the circumference of the hand, this harness (1) is connected in its part corresponding to the palm of the hand with a holder (2) for a computer device, and this holder (2) has an exposed part corresponding at least to the touchscreeen part of the computer device, this exposed part being on the side corresponding to the palm of the hand in the finger area.
2. A controller as claimed in claim 1, characterised in that, it comprises a gyroscope unit (19) attached to the harness (1), comprising at least one mechanical gyroscope (20).
3. A controller as claimed in claim 2, characterised in that, its gyroscope unit (19) comprises four mechanical gyroscopes (20).
4. A controller as claimed in claim 2 or 3, characterised in that, each of the mechanical gyroscopes (20) of its gyroscope unit (19) has a rotor (21) connected to a separate electric motor (23).
5. A controller as claimed in one of claims from 1 to 4, characterised in that, the connection of the holder (2) with the harness (1) is provided by an arm (4) of which one end is connected to the harness (1) at the position corresponding to the palm of the hand and the other end is connected to the holder (2), the connections of the arm (4) with the harness (1) and with the holder (2) are made as articulated joints.
6. A controller as claimed in claim 5, characterised in that, its gyroscope unit (19) is attached between the harness (1) and the arm (4).
7. A controller as claimed in one of claims from 1 to 6, characterised in that, The articulated joints between its arm (4) and the harness (1) in a position corresponding to the palm area of the hand and between the arm (4) and the holder (2) have at least two axes of rotation, and its arm (4) has the first articulated joint (5) in its centre, with the first articulated joint (5) in the centre of the arm (4) and each of the articulated joints at the ends of the arm (4) having a pressure bolt (17) to lock the arm position (4).
8. A controller as claimed in one of claims from 1 to 7, characterised in that, its harness (1) in its part corresponding to the palm of the hand has a spacer pad (6) attached.
9. A controller as claimed in claim 8, characterised in that, its spacer pad (6) has a rigid base (8) on the outside and an elastic cushion (7) on the inside to support the palm of the hand.
10. A controller as claimed in any of claims from 1 to 9, characterised in that, its harness (1) has at least one strap (11) for attaching it to the circumference of the hand.
11. A controller as claimed in any of claims from 1 to 10, characterised in that, its harness (1) is shaped like a glove (25) with holes (26) for fingers.
12. A method of controlling devices equipped with a radio receiver by means of a controller as claimed in claim 1, characterised in that, in the first stage, the harness (1) is attached to a hand and the computer device, equipped with at least the following sensors: an accelerometer, an electronic gyroscope, a camera, a touchscreen and a radio transmitter, is mounted in the controller holder (2) with the touchscreen facing the fingers and connected wirelessly to the radio receiver of the controlled device, after which, in the second stage, the controller is moved and the touchscreen is touched with fingers, and with the use of computer device sensors, a point in the space, in which the controller is located, is established, and then information about the controller's position and touches of the computer device's touchscreen is transmitted wirelessly to the receiver of the controlled device and, on the basis of this information, the device is controlled.
13. A method as claimed in claim 12, characterised in that, in the first stage, a smartphone (3) is mounted in the holder (2), and when the smartphone (3) is wirelessly connected to the radio receiver of the controlled device, buttons (34) are displayed on the display of the smartphone (3) mounted in the holder (2), and by touching these buttons (34), the function assigned to them is activated.
14. A method as claimed in claims 12 or 13, characterised in that, a virtual, mixed or augmented reality device is controlled.
15. A method as claimed in one of claims from 12 to 14, characterised in that, after installing a computer device in the holder (2), it is connected wirelessly with a smartphone mounted in a virtual reality goggle, then in the second stage a hand, on which the controller is worn, is moved and its touchscreen is touched with fingers, and its sensors are used to determine the point in the space where the controller is located, and then the information about the location of the controller and the touches of the touchscreen is wirelessly send to the receiver of the smartphone mounted in the virtual reality goggle and on the basis of this information a virtual object is controlled.
16. A method as claimed in claim 15, characterised in that, a reduced image of the buttons (34) and points (35), corresponding to where the user touches the touchscreen of the computer device fitted in the controller's holder (2) is displayed on the display of the smartphone fitted in the virtual reality goggles, after wireless connection of the computer device fitted in the holder (2).
17. A method as claimed in any of claims from 12 to 16, characterised in that, in the second stage the rotor (21) of at least one mechanical gyroscope (20) is rotated.
PCT/PL2019/000074 2018-09-07 2019-09-05 A controller worn on a human body and a method of controlling devices with the use of such controller WO2020067909A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
PLP.426941 2018-09-07
PL426941A PL239579B1 (en) 2018-09-07 2018-09-07 Controller mounted on human body and method of controlling devices using such controller
PL430730A PL245298B3 (en) 2019-07-26 2019-07-26 Human body mounted controller and method of controlling devices using such a controller
PLP.430730 2019-07-26

Publications (2)

Publication Number Publication Date
WO2020067909A2 true WO2020067909A2 (en) 2020-04-02
WO2020067909A3 WO2020067909A3 (en) 2020-09-03

Family

ID=69724026

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/PL2019/000074 WO2020067909A2 (en) 2018-09-07 2019-09-05 A controller worn on a human body and a method of controlling devices with the use of such controller

Country Status (1)

Country Link
WO (1) WO2020067909A2 (en)

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090321483A1 (en) * 2008-06-30 2009-12-31 Walt Froloff Universal wrist-forearm docking station for mobile electronic devices
US20110303710A1 (en) * 2010-06-15 2011-12-15 Yu David C Method and Apparatus for a Wrist Attachment for a Personal Electronic Device
US8845461B2 (en) * 2010-09-01 2014-09-30 Jason Duke Hoop tracker
US10429888B2 (en) * 2014-02-25 2019-10-01 Medibotics Llc Wearable computer display devices for the forearm, wrist, and/or hand
US20150318885A1 (en) * 2014-05-02 2015-11-05 Justis Earle Mobile electronic device holder
US10250288B2 (en) * 2014-05-23 2019-04-02 Tara Chand Singhal Apparatus and method for a hand wearable electronic device
US10352498B2 (en) * 2015-07-16 2019-07-16 Elephino, LLC Tool for multi-positioning and holding a device

Also Published As

Publication number Publication date
WO2020067909A3 (en) 2020-09-03

Similar Documents

Publication Publication Date Title
US10877557B2 (en) IMU-based glove
US10534431B2 (en) Tracking finger movements to generate inputs for computer systems
US20190209918A1 (en) Gyroscopic apparatuses and methods of using same
KR101541082B1 (en) System and method for rehabilitation exercise of the hands
KR100948095B1 (en) Motion-input device for a computing terminal and method of its operation
US10860014B2 (en) Jacket for embodied interaction with virtual or distal robotic device
US9317108B2 (en) Hand-held wireless electronic device with accelerometer for interacting with a display
EP2613223A1 (en) System and method for enhanced gesture-based interaction
US20150290494A1 (en) Arm exercise device and system
WO2008073801A2 (en) Inertial sensor input device
KR102162922B1 (en) Virtual reality-based hand rehabilitation system with haptic feedback
CA2625748A1 (en) Interface device
US20200341550A1 (en) System, Method, and Apparatus for Virtual Reality Feedback
RU2662399C1 (en) System and method for capturing movements and positions of human body and parts of human body
US9454236B2 (en) Three-dimensional mouse device and marionette control system using the same
WO2020067909A2 (en) A controller worn on a human body and a method of controlling devices with the use of such controller
EP3247986B1 (en) A force measurement mechanism
PL245298B3 (en) Human body mounted controller and method of controlling devices using such a controller
US20210072820A1 (en) Sticky device to track arm movements in generating inputs for computer systems
US11366521B2 (en) Device for intuitive dexterous touch and feel interaction in virtual worlds
PL239579B1 (en) Controller mounted on human body and method of controlling devices using such controller
RU198065U1 (en) REHABILITATION DEVICE
JP2001125728A (en) Operation input device
US11847267B2 (en) Input device tracking systems
Costa A biofeedback strategy for human postural sway control in a smart walker

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 08/09/2021)

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19856485

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 19856485

Country of ref document: EP

Kind code of ref document: A2