WO2016195503A1

WO2016195503A1 - Handheld controller with haptic feedback unit

Info

Publication number: WO2016195503A1
Application number: PCT/NO2016/000018
Authority: WO
Inventors: Ziad Badarneh
Original assignee: Univcoda As
Priority date: 2015-06-02
Filing date: 2016-05-31
Publication date: 2016-12-08

Abstract

A gyroscope unit comprise of a minimum of one weight loaded disc or wheel coupled with and on the same axis, to an electric motor for actuating rotary motion, and coupled with at least one actuator or servo motor for force of disc and said rotary axis in all directions, 360 degrees, (xyz - pitch, roll, yaw.), and as a result of the generated gyroscopic forces which acts as reference in space relative to the said portable means whereas motion feedback is produced to the casing of the controller of which the user can feel. This enables the unit and controller to interact and simulate motion according to activity and moving objects shown graphical on a screen and thereof give the user three dimensional physical feedback as result of the graphical activity and to interact and simulate motion according to the motion of an external moving apparatus and thereof give the user physical feedback of the objects movements.

Description

Handheld controller with haptic feedback unit.

F i e l d o f t h e i n v e n t i o n

The present application will describe embodiments of handheld controller for interaction with physical moving object and interaction with graphical interfaces, the controller comprising means for physical feedback motion.

B a c k g r o u n d o f t h e i n v e n t i o n

The world is now being overflowed with handheld devises for wireless remote control of apparatus and software (as games) shown on a screen. These have none or limited physical feedback to the user's hands, thus resulting no or in limited "feeling" of the action going on and displayed through a screen. So called "force feedback" and "haptic feedback", as developed by Immersion Corporation, rattles and shake a game controller during a game, but does not give the user any real physical motion or force resistance. Injuries to people is now beginning to become a problem as for example players using the Nintendo Wii handheld units performing physical force are "hitting or throwing" their hand and arm in to the air with too great a force, the result being over stretched muscles and worse, dislocated joints as the result of no feedback from the handheld units. Real physical feedback to a handheld unit during use will prevent such injuries as the user will receive physical feedback in order to control body movements. Such feedback will also give the user a new dimension of experience when playing a game or operating any software towards a graphical interface. The present invention also provides for remote control units with physical motion feedback for operating apparatus such as radio controlled model airplanes, helicopters, cars, boats etc., and for remote control of robots within diverse industries. Operation of mentioned apparatus today has to be done visually directly or through a camera with no or limited physical feedback to the user from the handheld remote control unit.

The present invention will utilize mechanical gyroscopes to make the remote unit simulate motion of the apparatus of which is operated and such give feedback to the user of motion direction and orientation of the moving apparatus. The invention when implemented for a radio controlled apparatus, will provide better operation and control of the apparatus even when the apparatus is out of sight. The invention will therefore provide a huge step towards full control, for example, when operating radio controlled flying apparatus, mobile robots especially those at remote locations as in space (on the moon or mars) and unmanned submarine vessels deep down in the ocean.

Gyroscopes are today used in space crafts and satellites as reference and counterweight for maneuvering in all directions, say turning in x-y-z by utilizing the gyro effect torque. These type of gyros are called Control Moment Gyros (CMG), are however very large with substantial weight of the gyro flywheels. CMG are also known to be used in boats for stabilizing the hull in rough sea.

Use of 3 -axis accelerometers and digital or piezoelectric gyros, reflects the precise position and orientation of the handheld in space (as on earth), the physical spinning flywheel of the gyro represent a physical reference point and resistance in space when motors or actuators for feedback of motion is activated.

The means for physical motion feedback enables units as remote controls to simulate motion according to the motion of the dedicated moving apparatus and thereof give the user physical feedback of the apparatus movements. The means for physical motion feedback also enables handheld units dedicated to a computer to interact and simulate motion according to activity and moving objects shown graphical on a screen and thereof give the user physical feedback as result of the graphical activity. This motion feedback is activated interactively according to a software program run on the computer and on the handheld itself.

B r i e f s u m m a r y o f t h e i n v e n t i o n

The invention comprise of solutions for handheld 3D motion feedback units for remote control and interaction with moving apparatus and with visual computer generated graphical interfaces, the units comprise of means for physical motion of the units for feedback to the user. The handheld 3D motion feedback units has a casing shaped to be held with at least one human hand and contain means for; computing data, receiving and sending wireless data to and from dedicated stationary or moving apparatus, input and control, optional visual and audio, physical motion, utilising motional feedback from physically spinning gyroscopes, optionally including 3 -axis accelerometers and electronic/piezoelectric 3- axis gyros for full detection of 3D motion in space (x-y-z), also known as pitch, roll, yaw/rotation, in space. The present invention operates towards a graphical user interface, such as a computer game and (or) being units for remote control of moving apparatus such as a radio controlled airplane.

Having 3 -axis accelerometers and 3 axis electronic gyroscopes the disclosed embodiments also contain mechanical means to motion the unit casing for physical motion feedback to the user of which is the core of this invention. The use of accelerometers for interaction between a handheld and a graphic user interface is amongst other known from Nintendo Wii and Apple I-phone. Electronic gyroscopes are used in radio controlled model aircrafts, especially helicopters and are able to provide direct measurement of the rate and degree of rotation. These gyroscopes have sensors which are very responsive, and when they are used with accelerometers provide a system of very accurate 3 dimensional detection of object in space. The gyroscope responds to the slightest degrees of rotation and rejects linear movements and hand jitters, while accelerometers recognize the linear motions. When 3-axes of gyroscopes are combined with 3-axes of accelerometers, the sensors will recognize how far, fast, and in which direction the hand held unit, remote control, game controller, etc., has moved 3 dimensional in space, thus creating the ability for accurate representation of real life movements replicated to a graphic interface or control of moving apparatus. The gyroscopes of the invention have rotary discs and means for angular motion. These type of gyros combined with linear motion feedback motors and 3-axis digital gyroscopes and 3-axis accelerometers, make a new category handheld units for superior user friendliness comprising full motion control and motion feedback. The different embodiments of gyroscopes vvdthin the invention hereof disclosed, represents a point of reference when spinning, which resist the motion from actuators and motors, forcing the casing of the handheld unit itself in motion providing feedback to the user. These and other advantages of the present invention will be apparent from the attached drawings and description as follows.

B r i e f d e s c r i p t i o n o f t h e d r a w i n g s

The foregoing aspects and many of the advantages of the present invention will be more appreciated and better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:

Fig. 1 shows a schematic illustration for the inventive handheld 3D motion feedback unit and environment.

Figs. 2 and 3 illustrate a screenshot of interactive activity.

Figs. 4 and 5 show handheld controller and physical moving apparatus.

Fig. 6 shows schematically interaction between handheld and physically moving apparatus.

Fig. 7-1 lc shows an embodiment of a gyroscope unit according to the invention, whereas figs. 7 and 8 show ISO and exploded view.

Figs. 9a-9b, 10a- 10b and 1 la-1 lc show views from the top and sides, whereas figs. 9b, 10b, l ib and 11c show an altered positioning of the gyro discs compared with a neutral position which is shown in figs. 9a, 10a and 11a.

Figs. 12-17 show a second embodiment of the invention comprising a handheld control unit fitted with a gyroscope unit, whereas figs.12 and 13 show ISO views.

Figs. 14a-14c show side views of said embodiment whereas figs. 15 and 16 show ISO views in exploded version.

Figs. 17a- 17c and 18a- 18c show movement of gyro unit relative the handheld controller.

Fig. 19 shows a third embodiment of a handheld controller which has a gyro unit with three wheels. D e t a i l e d d e s c r i p t i o n o f t h e i n v e n t i o n

Fig. 1 shows a schematic comprising the main elements of the invention. Box 1 represents a handheld controller unit comprising a CPU (computer processer unit) 2, of which all following components are connected; one or up to three axis accelerometers 4, one or up to three axis digital/piezoelectric gyros 6, mechanical rotary motion feedback gyros 7 comprising one to three discs, with motion actuators/servomotors 9, (as shown in figs. 7-16), radio/wireless (for example Bluetooth) receiver and transmitter 11, display 10, input and control means 12, this being one or more of; push buttons, button cluster, rotary multifunctional switches, keypad, touchpad, touch screen, joystick and or mouse ball. There also is audio means 13 and power distributor 14 connected with adaptor for mains electricity/ AC power or with a battery source. The unit will either interact with an ordinary stationary computer, game console 20 connected to a screen for graphical display 22, touch pad or smartphone, the unit will be used with and connected with a moving apparatus 24, both through wireless/radio receiver and transmitter 21 , 28. This moving apparatus can be of the following type; remote/radio controlled model airplane, helicopter, car, boat, submarine etc; mobile robots for earth and planetary science and for police, military (bomb detectors/destructors, spy planes and bombers - drones), etc. Power, motor, direction control and other basic elements within a moving apparatus 24 as mentioned above is of prior art and is numbered 25, camera numeral 29. The moving apparatus also has a processor 26 and a digital 3 -axis gyro 27, preferable also an accelerometer and of course radio/wireless receiver and transmitter 28.

The inventive handheld may also be used for interaction with stationary or semi stationary apparatus and equipment here numbered 23, as ladders/lifts, cranes and other construction machinery, robots within the manufacturing industry and robots used for medical purposes.

The handheld with the inventive system of motion feedback can therefore represent a type of apparatus as; mobile phone (smartphone), minicomputer, touch pad, computer mouse, game consol/handheld game unit, joystick, remote control, radio control unit, etc. Introducing three dimensional physical motion feedback within a personal device as a mobile phone unit opens for a wide range of applications. In addition to physical motion feedback when playing a game on your mobile phone unit, the unit can have a security system for geographical guidance of the user when in smoke/fog/dark by physical motion feedback from the mechanical system within the invention working with GPS and existing maps. This system would be very useful for the blind.

Joystick, remote control and radio control units with the inventive three dimensional physical motion feedback system may also be applied in the offshore industry (oil and other underwater activity) controlling underwater machinery, within the medicine industry, especially for remote operations within the human body, and as mentioned in outer space for instance controlling mobile units on planet surfaces.

The following will describe the functionality of a handheld controller unit utilising the present invention.

Fig. 2 illustrates a screenshot of an interactive aeroplane game or simulator. The controller or handheld unit 30 as shown in fig. 5 is held by the user, which actively uses it for controlling the game. In this case the user sees on screen part of the cockpit 40 and out of the aeroplane window 41. The controller 30 is used for manoeuvring the aeroplane along a runway 42 for eventually to take off and do a flight. The movements of the aeroplane (movements in x, y, z direction) is simulated by the gyroscope unit within the controller which communicate with the ongoing programme as shown on the screen. The same situation applies for any interactive games, also as illustrated in fig. 3, for instance when playing a game driving a car. Any movement of the car is simulated by the gyroscope unit within the handheld, such as acceleration, sudden stop, sharp turning, uneven road or terrain surface, obstacles and crashes.

All motions made by the handheld is transmitted to a computer unit of for instance a game console and processed according to the running software interactively shown graphical on a screen. Processed physical motion as created within the program and shown graphically is simultaneously processed by the handheld where the gyro unit give the user a sensation of realism when communicating with the graphical interface of for instance operating a game console.

The handheld unit according to the invention is made for interaction with a graphical interface and also to interact with a physically moving apparatus, with reference to fig. 1 and the example embodiment in fig. 4. Fig. 5 suggests a handheld controller with a gyroscope unit inside the handheld controller made to control any moving apparatus, here an aeroplane 45. The operation of the aeroplane can be done using control sticks or trackballs 46, 46' as suggested by fig. 5 or by tilting of the unit, of which

accelerometers and digital gyros read and transfer to the processor of the plane of which activates motors and balance servos. However, motion of the aeroplane 45, as suddenly sideway twist movement caused by wind, is transferred to the handheld unit 30, activating the gyro unit fitted inside (or under as shown in figs. 7-19) as to twist the unit accordingly in the hand of the user. The unit here is shown with a screen 47 of which can show footage from the moving apparatus if fitted with a camera.

Unit 30 can be used for controlling any moving apparatus as remote/radio controlled model aeroplane, helicopter, car, boat, submarine, mobile robots for earth and planetary science and for police and military (bomb detectors/destructors, spy planes and bombers), etc.

Unit 30 may also be used for controlling parts of machinery of which distance is required or comfortable. This including lifts on lorries, cranes and other construction machinery, robots within the manufacturing industry, medical robots, etc. Fig. 6 shows schematically the interaction between the handheld unit and a moving unit, either a physical apparatus or a moving object within a graphical environment as within a computer game. Information and motion data is communicated between the handheld unit and moving apparatus/objects by any wireless technology. Whereas the handheld is the control unit and sends commands to the apparatus/object, the apparatus/object communicates to the handheld its position in space and any movement is communicated to the handheld as motion feedback generated by the motion feedback gyros unit within the scope of this invention. The means of motion feedback are gyroscope units, which in this description are shown in several embodiments including one wheel/disc solutions and three wheel/disc solutions.

The invention utilises the stability created by a spinning wheel or disc. A disc with a certain mass and spinning at high revolutions will create a moment of inertia/velocity of which keep the wheel stable.

Applying torque (input torque) on the gyroscope wheel, perpendicular to its axis of rotation (change of orientation) and perpendicular to its angular momentum, generates torque (output torque) which wants to rotate gyro wheel about an axis perpendicular to both the torque and the angular momentum. Value of this output torque is dependent of the circumference and mass of gyro wheel, its rpm and the speed and angular level of input torque.

The output force created makes the chassis of the handheld unit motion of which then is utilised in the invention for physical force feedback to the user according to an interactive game or remote control of a moving device. Figs. 7- 1 lc show a first embodiment of a gyroscope unit according to the invention.

Figs. 7 and 8 show an ISO view and an exploded view of the said gyroscope unit, disclosing a frame unit 50, supporting actuators 51, 52, 53. This frame unit is for connecting the gyroscope unit to any handheld control device chassis as disclosed anywhere else in this document. The end portions of the actuator arms 55, 56, 57, are fixed with ball joint connections 61, 62, 63 of which connects with snap and socket joints 61 ', 62' 63' on a triangular frame piece 66. To the frame 66 are fixed brackets 67, 68, 69 to which electric motors 71, 72, 73 are connected. Each of the gyro wheels 75, 76, 77 then are fixed to each of the said motor's rotary axles.

As the gyro wheels are fixed with the frame piece 66, any movement of this will force movement upon of all the three gyro wheels simultaneously. Figs. 9a, 10a and 11a, show gyro wheels in a neutral position. Figs. 9b, 10b and 1 lb show an example of a position of the gyro wheel whereas one actuator arm is extended, the two other arms are retracted. This tilts the frame piece 66 and positions the gyro wheels at different angles relative to the frame 50 and to any chassis as reference point.

Any motion of the actuator arms are controlled independently so to enable motion of the gyro wheels in all directions possible. As the gyro wheels are fixed positioned in a triangular manner to frame 66, they all will change position when any of the actuator is activated.

As known from prior art of gyro technology and the present document, the gyro forces created by the spinning gyro wheels will resist the motion from the actuators, thus forcing the actuators and frame to motion, as illustrated by fig 1 lc.

The following embodiment of the invention, utilises only one gyro wheel. Figs. 12 - 13 shows a controller unit for use with both hands of which comprise of the elements disclosed above with reference to fig. 1. The handles 80, 81 have joysticks 82, 83 as control elements. The middle body 84 of the unit has a screen 85. The invention however as an alternative embodiment, whereas it is prepared for connecting to and using a smart phone 86 with the unit, as illustrated in fig. 13.

The gyro unit 90 is fixed to underside of the middle body 84, figs 12-14 showing an outer casing 96 covering the wheel 92 of the gyro unit.

As further disclosed in figs. 15 and 16 showing using an exploded view of the embodiment, the gyro wheel 92 is fixed to an electric motor 94 again fixed to frame pieces 96'- 96". Three linkage arms 97, 98, 99, (97', 97", 98', 98", 99', 99") are connected to frame piece 96' via joints 97", 98", 99", and with arms 106, 107, 108 which are fixed to axles of servo motors 109, 110, 111, the motors fixed to a frame 100 on the underside of body 84". The motors and arms are arranged in a triangular fashion as apparent from fig. 16, and holds the frame and gyro casing 96 (96', 96", 96"'). The servo arms are individually controlled so to make motion to the gyro unit, thus creating motion feedback to the controller. Additional electro motors with unbalanced weight for vibration are also shown, numerals 102, 103. Within the casing 84', 84" is shown circuit boards and electronics for the handheld control unit, with reference to fig. 1. Fig. 16 also shows buttons 104, 105 on handheld frame parts.

Figs. 17a- 17c and 18a- 18c show two different orientation of the gyro wheel unit 120. As shown in figs. 12- 14c the servo arms 106, 107, 108, and connected linkages are at a neutral position, pointing downwards. Figs. 17a- 17c shows servo arm 106 and arm 97 pulled up. In this positions the gyro wheel unit 90 is at an angle, approx. 60 degrees to the control unit main body. Figs 18a- 18c shows the two servo arms 106, 107 and arms 97, 98 pulled up, positioning the gyro wheel unit 90 in another direction.

To generate any output torque to a handheld as disclosed above, the gyro wheel itself, need to have certain mass and rotational speed. Tests has shown that a gyro wheel of 61 grams with a circumference will need to rotate at speed from 15000 rpm and upwards to generate sufficient inertia and gyro forces, taken in to account the weight of handheld controller. Speed and torque of the actuators and servomotors is also critical to create usable feedback torque from the gyro wheel. The servo must react very fast, interacting with a game or moving object, so to gain quick responsive feedback torque from the gyro unit.

If downsizing the controller the gyro unit can be made smaller, with smaller wheel, motors, servos and actuators.

Fig. 19 shows an ISO view of a handheld controller with a gyro unit similar to what is disclosed with reference to figs 7-11. The gyro unit comprise of three wheels/discs 121, 122, 123 and the design and construction of gyro unit is according to said figs. 7-11. The actuators 124, 125 and 126(not visible here) as shown can as a variation of and according to the invention, be servo motors arms and links as disclosed in figs. 12-18. The means of the physical feedback within the handheld unit of the invention should be understood within the inventive scope to apply to use in any remote control unit, mobile phones, GPS units, game controls, and handheld multifunctional communicators and computers.

Claims

C l a i m s: 1.

A gyroscope unit and handheld controller, the unit comprising of a minimum of one weight loaded disc or wheel coupled with and on the same axis, to an electric motor for actuating rotary motion, and coupled with at least one actuator or servo motor for force of disc and said rotary axis in all directions, 360 degrees, (xyz - pitch, roll, yaw.), and as a result of the generated gyroscopic forces which acts as reference in space relative to the said portable means whereas motion feedback is produced to the casing of the portable means of which the user can feel, the gyroscope unit is characterized by having;

- one weight loaded disc coupled on the same axis with an electric motor fixed within a frame and connected to three actuators or servo motors, or

- three weight loaded discs, fixed to a frame and arranged in a triangular

configuration each coupled on the same axis as an electric motor, the frame connected to three linear actuators or servo motors,

the controller characterized by having;

- Means of computing - a CPU

- Digital accelerometers

- Digital gyros

- Means for radio and wireless

- Means for electric power

- Means for audio

- Means for display

- Physical input and control means as buttons or joystick,

whereas the gyroscope unit enables the controller to;

- interact and simulate motion according to activity and moving objects shown graphical on a screen and thereof give the user physical feedback as result of the graphical activity, - and to interact and simulate motion according to the motion of external moving apparatus and thereof give the user physical feedback as result of the apparatus movements.

2.

A gyroscope unit and handheld controller according to claim 1, wherein the speed of revolution and motional speed of the gyroscope units and actuator and servo motors are adjusted by the computing means of the controller according to the activity within the computer generated graphical interface and or to an independent moving apparatus, so to represent the correct motion, thrust, weight, pull - push, etc. to the object or environment which is simulated.

3.

A gyroscope unit and handheld controller according to claim 1, wherein any actuator or servo motors of the gyroscope unit are supplied with encoders.

4.

A gyroscope unit and handheld controller according to claim 1 , wherein it can be connected, by wire or wireless, to;

- A computer

- A smart phone

- A tablet

- Radio communication of a remote object or apparatus.

5.

A gyroscope unit and handheld controller according to claim 1, wherein the means for control and input comprise of one, several of or a combination of the following:

- Touchscreen

- Touch pad

- Keyboard

- Push buttons, button cluster - Joystick

- Mouseball/Trackball

- Rotary multifunctional switch

A gyroscope unit and handheld controller according to claim 1, wherein the controller made to interact and control the following:

Moving apparatus/objects as:

Remote/Radio controlled model airplane, helicopter, car, boat, submarine, mobile robots for earth and planetary science and for police, military (bomb detectors/destructors, spy planes and bombers), etc.,

and (semi)stationary apparatus:

Lift, crane and other construction machinery, robots within the manufacturing industry, medical robots, etc.

7.

A gyroscope unit and handheld controller unit according to claim 1 , wherein the technical content is built in and resembles the following:

Game controller (compact console or interacting with a remote screen)

Mobile phone/Smartphone

Remote control

PC-Mouse