WO2016116182A1

WO2016116182A1 - Flexible device for guiding a user

Info

Publication number: WO2016116182A1
Application number: PCT/EP2015/075944
Authority: WO
Inventors: José ARAÚJO; Guoqiang Zhang; Lars Andersson
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2015-11-06
Filing date: 2015-11-06
Publication date: 2016-07-28

Abstract

A device (10) is provided being configured to guide a user holding the device. The device (10) comprises a body (11) being at least partly flexible, and at least one actuator (14) for causing the flexible body (11) to flex in at least one direction relative to a reference orientation of the device (10). Advantageously, the user is provided with intuitive and easy-to-interpret directional feedback. The device (10), which may be bar-shaped for adequate fit in a user's hand, will flex in a direction to be indicated to the user by means of having one or more actuators (14) which cause the flexible body (11), or the part of the body (11) being flexible, to flex in a direction in which the user is to be directed by the device (10).

Description

FLEXIBLE DEVICE FOR GUIDING A USER

TECHNICAL FIELD

The invention relates to a device configured to guide a user holding the device. BACKGROUND

Haptic feedback is a powerful mechanism to supply information to people. Particularly, it can be used to effectively transmit navigation instructions for users in hazardous situations such as flooding, fire, etc., to visually impaired persons, to facilitate haptic expressions between users, and it can also be employed to provide an immersive experience in virtual or augmented reality.

The majority of haptic-feedback devices currently proposed for navigation utilize vibration motors, which are rather non-intuitive since the user must interpret the vibration pattern in order to know the orientation command. Recent examples of haptic-feedback devices aim to provide a more intuitive feedback in the form of, for instance, a wristband that vibrates or applies pressure to the user's wrist in order to provide the user with navigation aid. In another example, a ball-type device is proposed in order to provide haptic- feedback navigation to visually impaired persons. The ball is hand-held by the user and mechanical pins inside the ball move in order to generate vibrations in all directions.

A new soft-actuator technology has been proposed, which is based on pouch motors. This technology can enable a material to quickly move in various directions and stretch or shrink. Other materials that can change their shape include electroactive polymers, shape memory alloys and polymers, thermoplastics and piezoelectric elements.

Several haptic devices have been proposed for usage in virtual reality or teleoperation, such as haptic gloves using motor-based actuation. With such gloves, a user is able to receive haptic feedback when, e.g., handling objects in teleoperation or in a virtual environment. Additionally, any actions performed with the user's hand can be measured by the glove and translated into the teleoperation or virtual environment. Hence, this device is able to translate information into a haptic feedback but also a haptic movement into haptic feedback in a remote or virtual environment.

Again, a problem is that navigation using these feedback devices is non- intuitive and it is oftentimes difficult for a user to interpret the haptic feedback being provided.

SUMMARY

An object of the present invention is to solve, or at least mitigate, this problem in the art and to provide a device configured to indicate direction to a user holding the device in an improved manner.

This object is attained in an aspect of the invention by a device configured to guide a user holding the device. The device comprises a body being at least partly flexible, and at least one actuator for causing the flexible body to flex in at least one direction relative to a reference orientation of the device.

Advantageously, a user holding a device according to an embodiment of the invention is provided with intuitive and easy-to-interpret directional feedback. The device, which may be bar-shaped for adequate fit in a user's hand, will flex in a direction to be indicated to the user by means of having one or more actuators cause the flexible body, or the part of the body being flexible, to flex in a direction in which the user is to be directed by the device.

In an embodiment, three degrees of freedom (3DOF) of flexing the body are provided for in that the actuator(s) are configured to cause the flexible body to flex either in a forward and backward direction, a left or right direction, or an up or down direction, relative to the reference orientation of the device, thereby indicating transitional movement. Advantageously, such a device could be used to provide sufficient directional feedback to a user in most situations, such as in a virtual reality environment. In a further embodiment, 6DOF of flexing the body are provided for in that the actuator(s) are configured to cause the flexible body to flex in another three directions defined as roll, pitch and yaw movements, thereby indicating rotational movement, where roll typically indicates rotation around an x axis, pitch indicates rotation around an y axis, and yaw indicates rotation around a z axis. Advantageously, with 6DOF, the device is capable of directing the user to move in any direction.

In an embodiment, the device further comprises an orientation sensor arranged to detect an orientation of the device with respect to the reference orientation. The orientation sensor may be embodied in the form of an inertial measurement unit (IMU), such as an accelerometer, a gyroscope, a magnetometer or the like. This is advantageous, since the device can account for its own orientation when the actuators cause the flexible body to flex in a direction to be indicated to the user. Hence, no matter how the device is orientated, it is still advantageously capable of directing the user correctly in respect to the reference orientation. For instance, even if the device would be held by the user in tilted or upside-down orientation with respect to the reference orientation, the device is still capable of flexing in a correct direction. Hence, the reference orientation is fixed, and the detected orientation of the device is defined in relation to the fixed reference orientation.

In a further embodiment, the body of the device comprises a marker configured to indicate to the user how the body, or at least a part of the body, should be orientated in order to correctly direct the user with respect to a reference orientation. Hence, the body may for instance be equipped with a transversal line extending around the body which should be substantially aligned with (an imaginary) horizon for the directions to correct with respect to a reference orientation.

In still another embodiment of the invention, the device further comprises a position sensor arranged to detect a position of the device. This embodiment is advantageous since not only can directions be given to the user with respect to the orientation of the device, but further with respect to its positon in a given positional system.

This embodiment is particularly advantageous if the device further is complemented with an interface, preferably wireless, via which positional information of objects or obstacles in the vicinity of the device is

communicated to the device from, e.g., a Global Positioning System (GPS) or any other appropriate means of positioning, or positional information pertaining to a target position to which the user is to be guided.

In such an embodiment, the actuators can be configured to cause the flexible body to flex in a direction which steers the user of the device away from an object with which the user otherwise would be at risk of colliding. This would be greatly advantageous in, e.g., a VR environment comprising a plurality of users or objects, or for a visually impaired user.

In a further embodiment, positional data of the device (and other objects) is communicated via the interface from, for instance, a coordinating server using GPS, a smart watch, a smartphone, etc. Advantageously, in such an embodiment, the device can be remote controlled. Further, the device may not necessarily be equipped with a position sensor (even though it still could be), but may receive its positional data from a remote location or device. In still a further embodiment of the invention, the device comprises sensors configured to sense haptic feedback applied by the user to the device.

Advantageously, operation of the device may be controlled by the user applying a force to the device. For instance, a quick press on the body may turn it on or off. In a further embodiment, the sensed haptic feedback, for instance, the detected orientation of the device, may advantageously be utilized to remote control a further device, such as a robot or a sword or pen in a VR game, another device of the same type as the device of the invention, a video game console to, e.g., control movement of video game characters, or even a vehicle such as a car or an excavator, etc. In an embodiment, the device is advantageously equipped with fastening means for fastening the device to a user, via, e.g., a wrist band of a

smartwatch or to a haptic glove worn by the user.

In still a further embodiment, the actuators are advantageously configured to control a degree of flexibility of the body of the device depending on the particular application in which the device is used. For instance, if the device is used as a sword in a video game, the actuators will cause the body to be stiff, whereas if the device is used, e.g., to control another similar device to flex in different directions it should be flexible such that a user can flex the device. The particular application may, e.g., be communicated to the device, such that the actuators can activate different modes of flexibility of the body.

In still another embodiment, the device is capable of guiding the user to a target position supplied to the device, by having the at least one actuator causing the flexible body to flex in directions toward the target position. In a further embodiment, the device comprises a processing unit arranged to read sensor data of, e.g., an orientation sensor or position sensor and control the at least one actuator to flex the body in the appropriate direction based on the sensor data.

Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined otherwise herein. All references to "a/an/the element, apparatus, component, means, step, etc." are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is now described, by way of example, with reference to the accompanying drawings, in which: Figure la illustrates a hand-held device according to an embodiment of the invention, configured to guide a user holding the device;

Figure lb illustrates a 6DOF coordinate system of the device;

Figure 2a illustrates flexing of the device body in a forward direction and backward direction;

Figure 2b illustrates flexing of the device body in an upward direction and downward direction;

Figure 2c illustrates flexing of the device body in a left direction and a right direction; Figure 2d illustrates flexing of an upper section of the device body back and forth to indicate a roll movement;

Figure 2e illustrates flexing the device body in a twisting motion to indicate a yaw movement;

Figure 2f illustrates flexing an upper section of the device body from side to side to indicate a pitch movement;

Figure 3 shows a device according to an embodiment the invention;

Figure 4 illustrates an embodiment of the invention where a target position is communicated to the device for guiding a user;

Figure 5 illustrates a further embodiment, where real-time position data of obstacles is communicated to the device;

Figure 6 illustrates an embodiment of the invention where haptic feedback applied to a first device is utilized to control a second device;

Figure 7 illustrates a further embodiment of the invention where the device is arranged with fastening means to be engaged with corresponding fastening means of a haptic glove; and Figure 8 illustrates yet a further embodiment of the invention where the device is arranged with fastening means to be engaged with corresponding fastening means of a wristband.

DETAILED DESCRIPTION

Figure la illustrates a device 10 according to an embodiment of the invention, being held in a hand 20 of a user, configured to indicate direction to the user holding the device. The device 10 comprises a body 11 being at least partly flexible. That is, parts of the body may be flexible, but it is also envisaged that the complete body is capable of flexing. Materials providing for such behaviour include, for instance, electroactive polymers, shape memory alloys and polymers, thermoplastics and piezoelectric elements. The device 10 is illustrated as being bar-shaped, but other shapes may be envisaged. The device 10 may further comprise an orientation sensor 12 arranged to detect an orientation of the device 10 with respect to a reference orientation. Hence, if the device is to flex in a forward direction, it is able to conclude which direction is "forward". An advantage of equipping the device 10 with an orientation sensor 12 is that the user can hold the device in any orientation, and the device 10 will still be capable of orientating itself.

Alternatively, the body 11 may be arranged with a marker 13 for instructing the user how to hold the device 10 such that it is aligned with its reference orientation. For instance, the user may have to hold the device such that the marker is aligned with a substantially horizontal direction. In yet an alternative, the marker instructing the user how to hold the device 10 may be a shape or form of the body 12 indicating to the user how the device 10 should be held in order to be aligned with its reference orientation.

However, in the following, the orientation means will be embodied by the orientation sensor in the form of an inertial measurement unit (IMU), such as an accelerometer, a gyroscope, a magnetometer or the like.

The device further comprises one or more actuators 14 for causing the flexible body 12 to flex in at least one direction relative to the reference orientation of the device 10, thereby indicating direction to the user holding the device.

Typically, the device 10 comprises a processing unit 15 arranged to read sensor data of the IMU 12 and control the actuators 14 to flex the body 12 in accordance with the IMU readings acquired by the processing unit 15.

The processing unit 15 may be embodied in the form of one or more microprocessors arranged to execute a computer program downloaded to a suitable storage medium associated with the microprocessor, such as a Random Access Memory (RAM), a Flash memory or a hard disk drive. The processing unit 15 may alternatively be embodied in the form of a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), etc. Now, as will be described in the following, it is possible in embodiments of the invention to cause the device 10 to flex in 6DOF.

Figure lb illustrates a 6DOF orientation system of the device 10, where movement along the x axis indicates movement back and forth, movement along the y axis indicates movement to the left and right, and movement along the z axis indicates movement up and down. Theses are commonly known as transitional movements.

Further indicated is rotation around the x axis, i.e., tilting from side to side. This is known as a roll movement. Rotation around the y axis, i.e., tilting back and forth, is known as pitch movement, while rotation around the z axis, i.e., turning left and right, is known as yaw movement. These three types of movement are known as rotational movements.

The reference orientation is fixed relative to the coordinate system illustrated in Figure lb, i.e., any appropriate coordinate is set as the fixed reference orientation in the illustrated system, and the detected orientation of the device is defined in relation to the fixed reference orientation.

Figure 2a illustrates flexing of the body n in a forward direction (FD), and further with dotted lines that the body can be caused to flex in a backward direction (BD). Figure 2b illustrates flexing of the body n in an upward direction (UD), and further in a downward direction (DD) by having the device grow and shrink alternately in a longitudinal direction.

Figure 2c illustrates flexing of the body n, from the perspective of the user, in a left direction (LD), and further with dotted lines that the body can be caused to flex in a right direction (RD).

The directions illustrated in Figures 2a-c will provide for movement of the device 10 in 3D0F.

By adding directions indicating yaw, pitch and roll movements, a 6D0F device is provided. Figure 2d illustrates flexing an upper section of the body 11 back and forth to indicate a roll movement.

Figure 2e illustrates causing the body 11 to flex in a twisting motion around its longitudinal axis to indicate a yaw movement.

Figure 2f illustrates flexing an upper section of body 11 from side to side to indicate a pitch movement.

Advantageously, a user holding the device 10 according to an embodiment of the invention is provided with intuitive and easy-to-interpret directional feedback. The device 10, which may be bar-shaped for adequate fit in a user's hand 20, will flex in a direction to be indicated to the user by means of having one or more actuators cause the flexible body, or the part of the body being flexible, to flex in a direction in which the user is to be directed by the device. Figure 3 shows a device 10 according to an embodiment the invention comprising at least a partly flexible body 11, one or more actuators 14 for causing the flexible body 11 to flex in one or more directions relative to a fixed reference orientation as measured by the IMU 12, and the processing unit 15 arranged to read sensor data of the IMU 12 and control the actuators 14 to flex the body 12 on the basis of the IMU readings acquired by the processing unit 15.

In this embodiment, the device 10 further comprises a position sensor 16 arranged to detect a position of the device 10. Advantageously, with the position sensor 16, not only orientation of the device 10 can be sensed, but further its position. In this particular embodiment, the device 10 further comprises an interface 17, such as a wireless interface in the form of a radio interface, via which the device 10 can receive - and possibly also transmit - data from/to a remote location.

As is illustrated in Figure 3, the processing unit is communicatively connected to both the position sensor 16 and the interface 17 for reading position sensor data and receiving/transmitting information data via the interface 17.

In an embodiment of the invention, it is desirable to submit control data to the device 10 via its interface 17. Assuming for instance that a person having a visual impairment is to visit premises of a public authority or a hospital; the visual impaired person may not easily find a particular room on a particular floor in a large building.

With the device 10 of the invention, the person using the device may advantageously be guided to the particular room in the building. To this end, control data in the form of a target position is communicated to the device 10 via the wireless interface 17 from, e.g., a remotely located server, or a smartphone of the person.

Figure 4 illustrates a scenario where positional data has been communicated to the device 10 via the interface 17, which indicates a target position TP to which the user 30 is to be directed inside the building 40. The device 10 will thus, when the user 30 is at position A, initially flex in a forward direction (cf. Figure 2a), and when approaching a door 50 of the building, the device will combine the forward flexing with a slight left-hand pitch (cf . Figure 2f) in order to guide the user 30 to the door 50.

At point B, if the user 30 has not turned to face the door 50, the device 10 will flex to indicate a left-hand direction (cf. Figure 2c). However, if the user 30 is already facing the door 50, the device 10 will again flex in a forward direction and direct the user 30 towards a stairway 60. Now, upon arriving at the stairway 60 at point C, the device will flex in an upward direction (cf. Figure 2b) to indicate to the user 30 that movement up the stairs is required. In an embodiment, it is envisaged that the device 10 further comprises a loudspeaker (not shown) via which the user 30 is provided with audible directional instructions, which in this particular case could be "go up the stairs". Alternatively, an instruction to go up the stairs may be provided by haptic feedback.

The user will finally climb the stairway 60 and reach the target position TP, after appropriate guidance of the device 10 upon reaching the second floor.

Thus, the problem of having a visually impaired person 30 reach a target position TP in a building 40 is advantageously resolved by an embodiment of the device 10.

With reference to Figure 5, in still a further embodiment, real-time position data as regards objects in the vicinity of the user 30 is communicated to the device 10 via its interface 17. For instance, an obstacle in the form of another person 70 may unexpectedly arise along an expected path of movement of the user 30. The processing unit 15 will account for such obstacle position data and control the actuators 14 to flex the body 11 of the device 10 accordingly in the appropriate direction, such that the user may steer clear from any obstacle 70 arising in her path.

In a further embodiment, the device comprises sensors configured to sense haptic feedback in the form of force applied by the user to the device in order to perform one or more actions associated with a particular feedback. For instance, a quick press on the body of the device may turn it on or off. In another example the sensed haptic feedback is utilized to remote control a further device, for instance a further device of the same type as the device of the invention, or a video game console to, e.g., control movement of video game characters, or even a vehicle such as a car or an excavator, a robot, a drone, etc. Figure 6 illustrates an embodiment of the invention where haptic feedback applied to a first device 10 is utilized to control a second device 110 being of the same type as that of the invention. In this particular embodiment, haptic feedback is applied to the device 10 in the form of changes in orientation sensed by the IMU 12.

Imagine a scenario where a father and his child both have a device according to the invention, the father holding the first device 10 and the child holding the second device 110. The father may be aware of a current location of the child from a GPS located in a smartphone of the child. The location of the child is displayed on a smartphone of the father, whose goal is to help his child move from the current location to the location where the father is. This can advantageously be performed using the device 10 of the invention, where the father uses the first device 10 as a haptic input device, and the child uses the second device 110 as a haptic output device. By handling the first device 10 as a manipulator, the father can input navigation commands to the second device 110 held by the child. In Figure 6, haptic navigation commands are supplied to the first device 10 by the father by pivoting the first device to the right into position A. This navigation command is encoded into an appropriate control signal submitted wirelessly via the interface 17. Now, upon receiving the control signal, the second device 110 decodes the control signal and translates the decoded signal into a correct movement of the second device 110. In this particular example, the processing unit of the second device 110 will control actuators to flex the body 111 of the second device in a right-hand direction to a corresponding position A, thereby indicating to the child that she should move to the right.

Correspondingly, if the father pivots the first device 10 to the left into position B, a corresponding navigation command is encoded into a control signal submitted wirelessly via the interface 17. Again, upon receiving the control signal, the second device 110 decodes the control signal and translates the decoded signal into a correct movement of the second device 110. In this particular example, the processing unit of the second device 110 will control actuators to flex the body 111 of the second device in a left-hand direction to a corresponding position B, thereby indicating to the child that she should move to the left. As can be concluded from Figure 6, the device is highly advantageous, also for remotely guiding persons.

Figure 7 illustrates a further embodiment of the invention where the device is arranged with fastening means 18, in this particular example in the form of hook-like fasteners to be engaged with corresponding fastening loops 80 of a haptic glove 90 worn by a user. The hooks 18 are thus made to engage with the corresponding loops 80 of the glove 90 thereby tightly fastening and securing the device 10 to the hand of the user. It should be noted that many different fastening means can be envisaged such as button fasteners or Velcro fasteners. Figure 8 illustrates yet a further embodiment where the device is arranged with fastening means (not shown) as discussed with reference to Figure 7. In this particular example, the device 10 is fastened to a wristband 100 of the user. Advantageously, by fastening the device 10 to a wristband 100 (e.g., of a smartwatch), the user is still free to use her both hands when temporarily not using the device 10. By attaching the device 10 to the wristband 100, the device will further advantageously be capable of providing a higher torque in performing its movements to guide the user in the appropriate direction.

The disclosure has mainly been described above with reference to a few embodiments. However, as is readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the disclosure, as defined by the appended patent claims.

Claims

1. A device (10) for hand-held operation configured to guide a user holding the device (10), the device (10) comprising:

a body (n) being at least partly flexible;

at least one actuator (14) for causing the flexible body (11) to flex in at least one direction relative to a reference orientation of the device (10).

2. The device (10) of claim 1, wherein:

said at least one actuator (14) is configured to cause the flexible body (11) to flex in at least six directions pertaining to translational movement.

3. The device (10) of claim 2, wherein:

the translational movement comprises movement up and down, movement back and forth, and movement to the left and right, relative to the reference orientation.

4. The device (10) of any one of claims 1-3, wherein:

said at least one actuator (14) is configured to cause the flexible body (11) to flex in at least six degrees of freedom pertaining to translational and rotational movement.

5. The device (10) of claim 4, wherein:

the rotational movement comprises pitch movement, roll movement and yaw movement.

6. The device (10) of any one of claims 1-5, the body (11) comprising:

a marker (13) configured to indicate to the user how the body (11) should be orientated in order for the device (10) to correctly guide the user with respect to the reference orientation.

7. The device (10) of any one of claims 1-5, the body (11) being configured to have a shape indicating to the user how the body (11) should be orientated in order for the device (10) to correctly guide the user with respect to the reference orientation.

8. The device (10) of any one of claims 1-5, further comprising:

an orientation sensor (12) arranged to detect an orientation of the device (10) with respect to the reference orientation.

9. The device (10) of any one of the preceding claims, further comprising: at least one sensor configured to sense haptic feedback applied by the user to the device.

10. The device (10) of claims 8 or 9, being further configured to control, using the detected orientation of the device and/or haptic feedback of the user, a further device (110) with which the device (10) is capable of communicating.

11. The device (10) of any one of the preceding claims, further comprising: a position sensor (16) arranged to detect a position of the device (10).

12. The device (10) of any one of the preceding claims, further comprising: an interface (17) via which information is communicated to/from the device (10).

13. The device (11) of claim 12, the interface (17) being configured to receive positional information of the device and/or objects in the vicinity of the device (10) and/or a target position to which the user is to be guided.

14. The device (10) of any one of the preceding claims, further comprising: fastening means (18) for attaching the device (10) to a hand or wrist of the user.

15. The device (10) of any one of the preceding claims, wherein the at least one actuator (14) is capable of causing the flexible body (11) to flex in at least one direction guiding the user to a target position supplied to the device.

16. The device (10) of any one of the preceding claims, further comprising: a loudspeaker via which the user is provided with audible directional instructions.

17. The device (10) of any one of the preceding claims, further comprising: a processing unit (15) configured to control the at least one actuator (14) to cause the flexible body (11) to flex in said at least one direction based on control signals supplied to the processing unit.

18. The device (10) of any one of the preceding claims, the at least one actuator (14) being configured to control a degree of flexibility of the body (11) of the device (10) depending on a type application in which the device is used.