WO2019073490A1

WO2019073490A1 - 3d mouse and ultrafast keyboard

Info

Publication number: WO2019073490A1
Application number: PCT/IN2018/050656
Authority: WO
Inventors: Glenn Fernandes; Alwyn D'SOUZA; Apoorva Agwan
Original assignee: Glenn Fernandes; Dsouza Alwyn; Apoorva Agwan
Priority date: 2017-10-12
Filing date: 2018-10-12
Publication date: 2019-04-18

Abstract

The present disclosure provides an electronic data input device comprising: a pressure pad (16) comprising one or more sensors (10,11, 12) capable of user in a plurality of axes when the finger is inserted into the pressure pad (16), without requiring lifting or moving of the finger (14) or the electronic data input device, wherein the pressure change data along the plurality of axes is interpreted by a pre-programmed software in real time based on a preselected input mode, the preselected input mode being one of a three dimensional (3D) mouse, a keyboard, a combined 3D mouse and a keyboard, a chord keyboard, an electronic music inputting device, a simulator to learn music and sports, computer games, or for remote manipulation of a robot, a machine or a vehicle.

Description

3D MOUSE AND ULTRAFAST KEYBOARD

FIELD OF INVENTION:

This invention relates to an electronic device data input and control device and method, more specifically one that can work as a 3d mouse and keyboard which can also be used as an ultrafast data inputting/control device for computers, laptops, mobile phones, other electronic devices, robots, machines, vehicles or even musical instruments.

BACKGROUND OF INVENTION: Electronic Data inputting device commonly used today are keyboard and mouse. The commonest used keyboard is QWERTY. The QWERTY keyboard requires a lot of finger and hand movement and is thus very slow to type. Other keyboard layouts like DVORAK have reduced the amount of finger and hand movement by placing the keys in a suitable manner. Yet, this does not allow the ability to type real-time spoken speech, and has a learning curve to learn the placement of the different characters, which is not easy to learn. There is a need for faster data inputting methods that could match or even surpass the speaking speed and work closer to thinking speed. The mouse has made a computer interface much easier in the 2D environment, compared to just using the keyboard alone. Even though, the mouse moves in a plane completely different to the monitor, it's use is intuitive and has a low learning curve. Yet when a person has to use both keyboard and mouse, it is rather inconvenient, as one hand has to keep moving from keyboard to mouse, reducing speed and focus. Further, virtual reality is becoming more and more easily available due to faster and affordable computing. Virtual reality completely changes the way the user interacts with the computer, and opens up a whole new realm of possibilities in education, product design, engineering, architecture, medicine, entertainment etc. The standard keyboard and mouse do not allow for a truly immersive 3D experience. Even the 3D joystick allows for only limited movement and doesn't allow the user the feeling of directly interacting with the 3D world with his own hands. Hence, 3D mice are being developed to allow for the user's hands to show in the 3D virtual space, interacting with the objects there. However, these 3D mice are very expensive and require the person to hold the arms up in midair, without support, performing the gestures that they want to replicate in 3D space. This may lead to "Gorilla Arm Syndrome", fatiguing the arms due to lack of support, reducing the amount of time a user can use the 3D mouse at one stretch. Also, the regular keyboard cannot be used with the 3D mouse.

Thus, there is a need for a combined 3D mouse and keyboard that can be used comfortably for many hours at a stretch, allowing the user to have a direct experience of interacting with the objects in the virtual 3D environment, that can also allow for ultra-fast data inputting. Thus the present invention, which discloses a novel data inputting means that can function both as 3D mouse to interact with objects in virtual 3D world and also function as an ultra-fast keyboard.

SUMMARY OF THE INVENTION:

The present disclosure seeks to provide an electronic data input device comprising: a pressure pad that comprises one or more sensors that are capable of detecting miniscule analogue pressure changes by a finger of a user in a plurality of axes when the finger is pressed onto the pressure pad, without requiring lifting or moving of the finger or the electronic data input device, wherein the pressure change data along the plurality of axes is interpreted by a pre-programmed software in real time based on a preselected input mode, the preselected input mode being one of a three dimensional (3D) mouse, a keyboard, a combined 3D mouse and a keyboard, a chord keyboard, an electronic music inputting device, a simulator to learn music and sports, computer games, or for remote manipulation of a robot, a machine or a vehicle.

In an embodiment, the pressure pad is placed under other body parts and includes one dimensional (1 D) sensors to simulate other body movements selected from at least one of running, jumping or any other activity, by detecting the miniscule analogue pressure changes caused by varying pressure by those body parts.

In another embodiment, the pressure pad is supported on any surface comfortable for long use by the user, where pressure changes by the user's fingers are detected and outputted as one of:

natural movements of the user's hand interacting with objects in a 3D virtual environment, or

movements that cannot be performed in real life by the user such as rotation of the hand in 360 degrees, spinning of hand by continuous pressure in one direction, with a haptic feedback to the user to increase the sensation of actual interaction with the objects that are being manipulated through the electronic data input device.

In yet another embodiment, varying pressure of one or more fingers or palm emulates natural gestures to control software, the robot, the machine and the vehicle to input a new functionality in real-time with a haptic feedback to the user to increase sensation of actual interaction with the objects that are being manipulated through the electronic data input device.

In yet another embodiment, to detect upward movement accurately in a vertical axis, the one or more fingers can be placed in one of: thimbles, vacuum or suction cups, or sticky pads.

In yet another embodiment, the electronic data input device comprises additionally at least one of a microphone, a galvanic skin resistance, an electroencephalogram, an electromyogram and an eye-tracking sensors to additionally aid the electronic data input device to better understand the user intent.

In yet another embodiment, the pressure pads are attachable and conformal to either one of a body part, a mobile phone, chair handles, or any object. In yet another embodiment, the pressure pads with sensors are conformal to the fingertips and wherein the pressure pads comprise a support pad that is conformal to the fingers against which the pressure pads are pressed, both the sensors and the support pad being fixed to the hand, allowing the user to use the electronic data input device even when mobile.

In yet another embodiment, when a particular pressure movement activates a typing mode as a chorded keyboard, different pressure movements of the one or more fingers simultaneously enables the user to directly type in real-time, using any pre-programmed phonetic script, which is then be converted into the desired output script by the preprogrammed software, wherein the fingers are used in several axes and varying levels of pressure are used such that

one finger is used in the several axes for vowels with varying levels of pressure to allow for long and short vowels,

consonants are specified in a grid shape, so only two fingers are used to denote a specific consonant, one finger specifies the x-axis of the grid, another finger specifies y-axis of the grid,

conjunct characters are specified with a separate finger,

additional functions are operated by another finger and modes are changed by the index finger, which has an additional pressure pad for selecting the mode.

BRIEF DESCRIPTION OF DRAWINGS:

FIG. 1 and 2 illustrate an electronic data input device in accordance with an embodiment herein; and

FIG. 3 illustrates an embodiment of the electronic data input device where it is set to use as a keyboard in the Devanagari script which is a phonetic script.

DETAILED DESCRIPTION OF EMBODIMENTS: The present invention discloses a novel 3D mouse and method of using it which can be used to interact intimately with objects in the 3D virtual reality world, useful in gaming, entertainment, education and 3D CAD design and which can also be used as an ultrafast data inputting method for computers, laptops, mobile phones and musical instruments. Different aspects of various embodiments are briefly described below.

Working principle:

The present disclosure seeks to provide an electronic data input device comprising: a pressure pad that comprises one or more sensors that are capable of detecting miniscule analogue pressure changes by a finger of a user in a plurality of axes when the finger is pressed onto the pressure pad, without requiring lifting or moving of the finger or the electronic data input device, wherein the pressure change data along the plurality of axes is interpreted by a pre-programmed software in real time based on a preselected input mode, the preselected input mode being one of a three dimensional (3D) mouse, a keyboard, a combined 3D mouse and a keyboard, a chord keyboard, an electronic music inputting device, a simulator to learn music and sports, computer games, or for remote manipulation of a robot, a machine or a vehicle There are pressure detecting sensors below one / many fingers and/or other body parts, detecting miniscule analogue pressure changes by a finger of a user in X, Y, Z axes. Differential movement or pressure detecting sensors are preferred and within this 10 - 100 levels of accuracy are preferred, with a minimum of 10 levels. There is ideally zero movement of the hand, and only pressure variation-based sensing by the pressure detecting sensors.

This zeroing of hand movement, eliminates the risk of carpal tunnel syndrome or wrist strain that occurs in people who do extensive data inputting. In fact, the software learns the user's nuances and with higher speeds the pressure needed drops, while accuracy keeps increasing. If another user uses the same 3D mouse, it learns the new user's nuances and requires less and fewer corrections over some time. Sensitivity can be set for extreme sensitivity, so the slightest pressure will do, allowing for least effort. The more the pressure resolution, the smoother can be the output.

The pressure detecting sensors below each finger can be reprogrammed to perform any function e.g. scroll, move left, move right, rotate etc. Sensor movement detected in up to 8 directions for certain modes like keyboards. Maximum pressure levels sensed in thumb and index finger, as more precise control is possible with these. Additionally, other areas e.g. base of palm/elbow etc. could be used with similar pressure sensors. The electronic data input device doesn't need to be used mid-air, but can be supported on any surface and thus usable comfortably for hours at a stretch. Further the user gets haptic feedback from the electronic data input device, giving the user a direct hands-on experience of the 3D virtual world, as the user feels he is interacting with the world with his own hands, comfortably for hours at a stretch. Both of these are enormous advantages over other 3D Virtual Realtiy interface tools.

The electronic data input device allows for a very natural and realistic interaction with the 3D world. This is especially advantageous in the field of 3D CAD design, as the user can quickly design 3D products and assemblies, at speeds several times faster than regular 3D CAD design using a 2D mouse. To touch an object, the user applies less pressure. Now to hold the same object, the user merely increases the pressure. This 3D CAD design becomes incredibly easy and extremely intuitive because the user actually presses the fingers in the same way as the action he/she wants the 3D object to perform - one doesn't need to learn complex 3D CAD design software e.g. If the user wants to rotate a sphere, he/she can just apply pressure on the thumb and index finger as if the sphere was held by them and by applying more pressure on one finger and reducing the pressure on the other finger, one simulates a clockwise and counterclockwise movement of both fingers, which together would rotate any object held by them. It is so natural, that soon by merely imagining the movement, the fingers automatically apply / decrease pressure in the correct direction to cause that movement. Zero gravity setting can be incorporated, so that objects moved by you stay where you have moved them. The user can bring two objects together with both hands, such that the surfaces touch each other, they could snap-lock in place, if in assembly mode. Using a physics engine, the user can assemble objects in the 3D world. The settings can be so done that the hands can even be visible in the 3D virtual world, giving the user a very realistic experience of manipulating the 3D world. The user's hands can also be seen on the screen as transparent, to enable better visualization of the object being manipulated by the hands.

The electronic data input device could additionally be coupled with voice commands, to further aid design process - example "give me a cylinder" - a cylinder is dropped into one hand. Menu items are preferably activated using voice commands, wherever one is choosing an item among several, whereas actions are performed using the 3D hands. When the finger touches an object, it will stop on screen, giving the sensation of the touching the object to the user.

In yet another embodiment, varying pressure of one or more fingers or palm emulates natural gestures to control software, the robot, and the machine and to input a new functionality in real-time with a haptic feedback to the user to increase sensation of actual interaction with the objects that are being manipulated through the electronic data input device.

Haptic and other feedback means can be given back to the fingers to increase the sensation of actual interaction with the 3D objects, for a very real experience with sensations of heat/cold by using Peltier modules etc. vibrating elements for roughness, cloth feel, glass feel etc. Each finger can move objects independently - for e.g. one finger can go and touch a wall that is at a distance, just by increasing pressure in the forward direction sensor below that finger. The other fingers are on the screen but it appears as if this finger is elastic.

The action is cumulative. The user applies more pressure, the further it goes. He/she applies still more pressure; it goes even further. It also time integrates the values - the faster you go, the further it moves, in the same time. Because there is only minimal movement of the fingers, without actually moving or lifting any finger, this electronic data input device allows for hours of work without tiring.

The electronic data input device can be placed anywhere where it is most convenient for the user - on his knees, thighs, on the table, chair handles, or held up in mid-air.

In yet another embodiment, the pressure pads are attachable and conformal to either one of a body part, a mobile phone, chair handles, or any object. In another embodiment, the pressure pad is supported on any surface comfortable for long use by the user, where pressure changes by the user's fingers are detected and outputted as one of: natural movements of the user's hand interacting with objects in a 3D virtual environment, or

movements that cannot be performed in real life by the user such as rotation of the hand in 360 degrees, spinning of hand by continuous pressure in one direction, with a haptic feedback to the user to increase the sensation of actual interaction with the objects that are being manipulated through the electronic data input device. When it is to be used with a mobile device, it can be a small pad, operated with even just one, three/four fingers, suitable for gestures and even for typing. They could be on the corner of a mobile phone, button less, long-lasting. The user is discreetly typing, or using the mobile phone without any seeming movement.

Pressure pads under each hand can be used together as two keyboards, two 3D mice, or separately as keyboard and mouse. When used as two keyboards, it can take syllables alternately from each hand, or the user can favour one hand. The user has complete freedom to program the device to process pressure sensor output of any finger for whatever action/character he chooses. The learning curve is very low in this electronic data input device, but the improvement can keep on for a long time, till the electronic data input device is very intuitive to use. As both hands are mirrors of each other, when one hand learns how to use this electronic data input device, the other hand automatically learns it.

Instead of, or in addition to pressure sensors, other sensors requiring minimal movement by the fingers like magnetic, infrared, resistive, capacitive, inductive sensors can be used. For example, in one embodiment of this invention, an infrared sensor (mouse sensor) is used for detecting movement in X, Y axes and resistive pressure sensors are used for further levels along the Z axis.

Settings: The software could be pre-programmed to suit the use of this electronic data input device in various modes.

1 . Gaming Mode: If it is to be used as a 3D mouse in a gaming environment, the software could be pre-programmed to interpret various outputs of the pressure sensors as the movements required by that particular game.

2. 3D CAD Mode: If it is to be used as a 3D mouse in a 3D CAD environment, then the software could be pre-programmed to interpret these same outputs of the pressure sensors to perform the function required in a 3D CAD environment e.g. - extrude, align, rotate etc. . Both hands can have their own electronic input device and can work together. As a keyboard-type fast data input device: If this device is to be used to type out words in any language, the software can be set to interpret the outputs of the pressure sensors as the charactres of that language. Ideally, the setting can be in a phonetic script, allowing the typist to transcribe audios from any language real-time, without any language barrier.

As a musical instrument: The device can also be programmed to play music - with different fingers and pressure positions to denote different notes to the software, with the ability to play musical chords and melody, even allowing one to use various pre-set rhythms, setting the beat to various tempos and even outputting different instrument tones like an electronic keyboard, or even as a wind instrument like a saxophone with an additional blowing sensor.

As an aid for the speech impaired: The device software can be programmed to type the script based on pressure sensor input and simultaneously a text to speech engine on the device or on a computer/mobile phone could speak out the text, allowing a person with speech impairment to have a normal conversation with anyone, because the typing speed here is equal or more than that of regular speech.

To train, control and operate robots, machines and vehicles: The device software can be configured such that pressures along different axes on each pressure sensor would be outputted as specific movements like move in the forward direction, rotate, move backward, etc on the output device and can be used to train robots to perform actions, and also to remotely operate and control machines, vehicles and robots.

In yet another embodiment, the pressure pads with sensors are conformal to the fingertips and wherein the pressure pads comprise a support pad that is conformal to the fingers against which the pressure pads are pressed, both the sensors and the support pad are attached to the hand with any suitable means like being enclosed in a glove, or being strapped onto the hand and wrist, allowing the user to use the electronic data input device even when mobile.

In yet another embodiment, when a particular pressure movement activates a typing mode as a chorded keyboard, different pressure movements of the one or more fingers simultaneously enables the user to directly type in spoken words in real-time as he speaks, using any preprogrammed phonetic script, which is then be converted into the desired output script by the preprogrammed software, wherein the fingers are used in several axes and varying levels of pressure are used such that one finger is used in the several axes for vowels with varying levels of pressure to allow for long and short vowels,

conjunct characters are specified with a separate finger,

DETAILED DESCRIPTION OF DRAWINGS:

One embodiment of this invention is illustrated in FIG 1 and 2. It consists of 3 pressure sensors 10, 1 1 and 12 below each finger 14 of both hands. The pressure sensors 10, 1 1 and 12 act in differential mode, are encapsulated in a thimble-like cap (e.g. pressure pad) 16 on each finger and detect movement in X, Y, and Z axes respectively. Once the user places his fingers in the desired position, the electronic data input device can preferably be locked, to keep this spacing between fingers - the hand can be withdrawn from the thimble-like caps 16 after use, and just slipped in again for use - if the user desires to change the relative spacing between the fingers at any point in time, he can at any time unlock the electronic data input device, reposition the fingers, and lock the electronic data input device. There could even be additional thimble-like caps with sensors 10, 1 1 , 12 for the index finger, middle finger and/or thumb, in that order, for additional characters, as is seen as extra thimble-like capl 8 in FIG 1 .

FIG 3 illustrates an embodiment of this electronic data input device where it is set to use as a keyboard in the Devanagari script which is a phonetic script. This type of keyboard will allow for even more than 150 words/minute. The Devanagari script is the preferred script as it has several advantages - 1 /6^th of the world knows it - so they don't have to learn the placement of the alphabet, and its phonetic, so the user can type out any foreign language based on the phonetics, which can then be outputted in the original language for the speaker to read. This script is also well organized based on the grouping of labials, gutturals etc. in separate rows and columns for the consonants, arranged in a grid form. The vowels are also separately assigned to another finger. Other phonetic scripts can also be programmed into this electronic data input device. The device can even electronic data input have more than 10 thimble-like caps with sensors, to accommodate more characters. Movement in X + Z or in Y + Z can indicate harsher sounds than movement in X/Y direction alone. As seen in the diagram, the circle 20 represents the sensor programming on the little finger. Circle 21 represents the sensor programming on the ring finger, which is capable of the least directional movement. Circle 22 represents the sensor programming on the middle finger. While Circles 23 and 24 both represent sensor programming on two sets of sensors for the index finger. Circle 25 represents the sensor programming for the thumb. Each finger is capable of moving in up to 8 direction, by utilizing X+, X-, Y+, Y- directions. Further, Z gives additional pressure, for even more characters. Several fingers can be pressed simultaneously in the required directions at any point of time, allowing for ultra-high speed typing of one or several words simultaneously. Further, the phonetic script grid being memorized by the native users in childhood, there in no learning curve as the same grid is utilized while typing. While the above embodiment is configured to work with the Devanagiri script, it could be suitably configured to work with any phonetic/regular typing script. The user could hear a foreign language being spoken, type it out in his own native script phonetically, and the software will then output the typed matter in the original foreign language being spoken, allowing for ultra-fast live transcripts of speech by persons typing in any part of the world.

In the foregoing specification, specific embodiments of the present invention have been described. However, one of ordinary skill in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present invention as set forth in the various embodiments discussed above. Accordingly, the specification is to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present inventions. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements as described herein.

Claims

We claim: 1 . An electronic data input device, characterized in that, comprising: a pressure pad (16) that comprises one or more sensors (10, 1 1 , 12) that are capable of detecting miniscule analogue pressure changes by a finger (14) of a user in a plurality of axes when the finger (14) is pressed onto the pressure pad (16), without requiring lifting or moving of the finger (14) or the electronic data input device, wherein the pressure change data along the plurality of axes is interpreted by a pre-programmed software in real time based on a preselected input mode, the said preselected input mode being one of a three dimensional (3D) mouse, a keyboard, a combined 3D mouse and a keyboard, a chord keyboard, an electronic music inputting device, a simulator to learn music and sports, computer games, or for remote manipulation of a robot, a machine or a vehicle.

2. The electronic data input device as claimed in claim 1 , wherein said pressure pad (16) is placed under other body parts and includes one dimensional (1 D) sensors (10, 1 1 , 12) to simulate other body movements selected from at least one of running, jumping or any other activity, by detecting the miniscule analogue pressure changes caused by varying pressure by those body parts.

3. The electronic data input device as claimed in claim 1 , wherein the pressure pad (16) is supported on any surface comfortable for long use by the user, where pressure changes by the user's fingers (14) are detected and outputted as one of natural movements of the user's hand interacting with objects in a 3D virtual environment, or movements that cannot be performed in real life by the user such as rotation of the hand in 360 degrees, spinning of hand by continuous pressure in one direction, with a haptic feedback to the user to increase sensation of actual interaction with the objects that are being manipulated through the electronic data input device.

4. The electronic data input device as claimed in claim 1 , wherein varying pressure of one or more fingers (14) or palm emulates natural gestures to control software, the robot the machine and the vehicle and to input a new functionality in real-time with a haptic feedback to the user to increase sensation of actual interaction with the objects that are being manipulated through the electronic data input device.

5. The electronic data input device as claimed in claim 1 , wherein to detect upward movement accurately in a vertical axis, the one or more fingers (14) can be placed in one of: thimbles, vacuum or suction cups, or sticky pads

6. The electronic data input device as claimed in claim 1 , wherein said electronic data input device comprises additionally at least one of a microphone, a galvanic skin resistance, an electroencephalogram, an electromyogram and eye-tracking sensors to additionally aid the electronic data input device to better understand the user intent.

7. The electronic data input device as claimed in claim 1 , wherein the pressure pads (16) is attachable and conformal to either one of a body part, a mobile phone, chair handles, or any object.

8. The electronic data input device as claimed in claim 1 , wherein the pressure pads (16) with sensors are conformal to the fingertips (14) and wherein the pressure pads (16) comprise a support pad that is conformal to the fingers against which the pressure pads (16) are pressed, both the sensors (10, 1 1 , 12) and the support pad fixed to the hand, allowing the user to use the electronic data input device even when mobile.

9. The electronic data input device as claimed in claim 1 , wherein when a particular pressure movement activates a typing mode as a chorded keyboard, different pressure movements of the one or more fingers simultaneously enables the user to directly type in real-time, using any pre- programmed phonetic script, which is then be converted into the desired output script by the preprogrammed software, wherein the fingers (14) are used in several axes and varying levels of pressure are used such that one finger is used in the several axes for vowels with varying levels of pressure to allow for long and short vowels,

consonants are specified in a grid shape, so only two fingers (14) are used to denote a specific consonant, one finger specifies the x-axis of the grid, another finger specifies y-axis of the grid,

conjunct characters are specified with a separate finger,

10. The electronic data input device as claimed in claim 1 , wherein, instead of, or in addition to pressure sensors (10,1 1 ,12), other sensors requiring minimal movement by the fingers like magnetic, infrared, resistive, capacitive, inductive sensors can be used.