WO2019076459A1

WO2019076459A1 - Haptic device and method thereof

Info

Publication number: WO2019076459A1
Application number: PCT/EP2017/076715
Authority: WO
Inventors: Zeid AL-HUSSEINY; Soma TAYAMON; José ARAÚJO
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2017-10-19
Filing date: 2017-10-19
Publication date: 2019-04-25

Abstract

A haptic device comprising a haptic actuator (101) configured to render haptic content for sensing by a user, such as a haptic glove (100) or a haptic smartwatch (150), is provided. The haptic device (100, 150) is operative to acquire the haptic content, generate modified haptic content based on the acquired haptic content and a model representing anomalies in haptic perception of the user, and render the modified haptic content using the haptic actuator (101) for sensing by the user. Preferably, the haptic device is further operative to detect a reaction of the user in response to sensing rendered haptic content, using at least one sensor (104), and in response thereto adapt the model representing the anomalies in haptic perception of the user based on the detected reaction. Advantageously, an improved user experience is achieved for users which anomalies in haptic perception, such as degraded touch sensitivity, degraded motoric skills, or the like.

Description

HAPTIC DEVICE AND METHOD THEREOF

Technical field The invention relates to a haptic device, a method performed by a haptic device, a corresponding computer program, and a corresponding computer program product.

Background

Haptic devices, i.e., devices with haptic interfaces for recording haptic and/or rendering haptic content, are seen as the next step in mobile networking. Users are currently able to efficiently communicate using voice and video, but it is envisioned that people in the future networked society will also communicate through the sense of touch, using haptic devices.

Haptic perception consists of kinesthetic and tactile sense, and relates to the sensation of the size, shape, mass, texture, and stiffness, of physical objects, surfaces, etc. Kinesthetic sense refers to the information perceived by a person when moving joints, muscles, and tendons, while tactile sense refers to information retrieved via a person's skin.

In recent years, a variety of haptic devices has been developed, including haptic smartphones, haptic smartwatches, haptic garments (e.g., pieces of clothing such as gloves, vests, bodysuits, and the like), and exoskeleton-type of devices with the capability to record and/or render haptic content.

Haptic content, which can be rendered as a haptic sensation to a person using a haptic device by means of haptic actuators, may be received from a remote source. The remote source may be another haptic device used by the same or another user, or a database/server providing haptic content, e.g., as part of an Augmented Reality (AR)A/irtually Reality (VR) environment or an e-commerce service.

Typically, the haptic content is rendered in such a way that a user would not be able to distinguish between a sensation perceived as a result of contact with, e.g., a physical object, compared to the sensation perceived as a result of rendered haptic content representing the physical object.

Summary It is an object of the invention to provide an improved alternative to the above techniques and prior art.

More specifically, it is an object of the invention to provide improved solutions for rendering haptic content for sensing by a user. In particular, it is an object of the invention to provide solutions for rendering haptic content for sensing by a user which take anomalies in haptic perception of the user into consideration.

These and other objects of the invention are achieved by means of different aspects of the invention, as defined by the independent claims.

Embodiments of the invention are characterized by the dependent claims.

According to a first aspect of the invention, a haptic device is provided. The haptic device comprises at least one haptic actuator

configured to render haptic content for sensing by a user of the haptic device. The haptic device is operative to acquire the haptic content, generate modified haptic content based on the acquired haptic content and a model representing anomalies in haptic perception of the user, and render the modified haptic content using the at least one haptic actuator for sensing by the user.

According to a second aspect of the invention, a method performed by a haptic device is provided. The method comprises acquiring haptic content, generating modified haptic content based on the acquired haptic content and a model representing anomalies in haptic perception of the user, and rendering the modified haptic content. The modified haptic content is rendered using at least one haptic actuator comprised in the haptic device and configured to render haptic content for sensing by a user of the haptic device.

According to a third aspect of the invention, a computer program is provided. The computer program comprises computer-executable

instructions for causing a device to perform the method according to an embodiment of the second aspect of the invention, when the computer- executable instructions are executed on a processing unit comprised in the device.

According to a fourth aspect of the invention, a computer program product is provided. The computer program product comprises a computer- readable storage medium which has the computer program according to the third aspect of the invention embodied therein.

The invention makes use of an understanding that rendition of haptic content by a haptic device, such as a device comprising a haptic

touchscreen, or a haptic wearable, such as a haptic garment, a haptic glove, or an exoskeleton-type of haptic device, can be improved by taking anomalies in haptic perception of the user into account when rendering haptic content. In the present context, haptic content is understood to be a representation of one or more haptic signals, i.e., haptic properties which typically vary over space and time.

Anomalies in haptic perception may, e.g., be caused by injuries, skin or neurological diseases, thicker skin, degraded motoric skills,

permanent/temporary injuries, degraded/sensitive touch sensation, or other medical conditions of the user. Information about anomalies in haptic perception of a user are maintained in a model, in the following also referred to as "user model". The user model is preferably user specific, but may alternatively be shared among a group of users. For instance, this may be the case for a model which is suitable for a group of users suffering from a certain medical condition which is associated with certain, common

anomalies in haptic perception. Embodiments of the invention may refine such a generic model during use of the haptic device, as is described herein.

Different embodiments of the user model may be envisaged. For instance, the user model may comprise normalization factors which specify to which extend an intensity, i.e., an amplitude, of haptic content is scaled, as compared to the original, acquired haptic content, before rendering the modified haptic content to the user. The normalization factors may either be stored as look-up tables, e.g., categorized according to the type of haptic property (e.g., temperature, texture, stiffness, friction, motion, e.g., position, velocity, or acceleration, etc) and body part (e.g., hand, skin, arm, joint, wrist, knee, etc), or in the form of maps, with a certain spatial resolution.

Alternatively, or additionally, the user model may also comprise constant offsets which specify to which extend an intensity of haptic content is in increased or decreased, respectively, as compared to the original, acquired haptic content, before rendering the modified haptic content to the user.

Embodiments of the invention are advantageous in that an improved haptic perception, and thereby user experience, is achieved for users which anomalies in haptic perception. To this end, rather than rendering haptic content in a way such that users cannot distinguish between directly touching a physical object and sensing rendered haptic content representing the physical object, embodiments of the invention render haptic content such that the perceived haptic sensation of a user with anomalies in haptic perception resembles that of the user as if he/she did not suffer from any anomalies in haptic perception.

For instance, if the user suffers from degraded skin sensitivity, embodiments of the invention may render tactile content with increased intensity, thereby compensating for the reduced sensitivity. Similarly, if the user suffers from degraded motoric skills, e.g., stiff or blocked joints, embodiments of the invention may render kinesthetic content with reduced intensity, so as to alleviate the user from discomfort or pain.

According to another embodiment of the invention, a reaction of the user in response to sensing rendered haptic content is detected. The reaction of the user is detected using at least one sensor configured to detect a reaction of the user in response to sensing rendered haptic content. The at least one sensor may, e.g., be comprised in the haptic device, and may alternatively be comprised in a separate device, such as the user's smartphone, smartwatch, or wearable, which is communicatively connected with the haptic device. In response to detecting the reaction of the user, the model representing the anomalies in haptic perception of the user is adapted based on the detected reaction of the user. Advantageously, the model representing anomalies in haptic perception of the user is refined during usage of the haptic device, and is adjusted to reflect emerging and/or changing anomalies in haptic perception of the user.

According to an embodiment of the invention, a reaction of the user in response to sensing rendered haptic content is detected by rendering haptic content with increasing intensity until a reaction of the user is detected which is indicative of a haptic perception by the user, i.e., the user has sensed the rendered haptic content, and the model representing the anomalies in haptic perception of the user is adapted based on a current intensity of the rendered haptic content when the reaction of the user was detected. The reaction of the user which is indicative of a haptic perception by the user may, e.g., be an explicit confirmation by the user. This may, e.g., be a gesture performed by the user, a command spoken by the user, or the user pressing a button, so as to confirm that he/she has sensed the rendered haptic content. In this way, a lower threshold for haptic perception of the user may be established, and the user model may be updated accordingly. As an alternative, the reaction of the user which is indicative of a haptic perception by the user may, e.g., be a movement of body part, such as the user flexing a finger or an arm in response to sensing the rendered haptic content. In particular, the reaction of the user which is indicative of a haptic perception by the user may be a natural reflex of the user in response to sensing the rendered haptic content.

According to an embodiment of the invention, a reaction of the user in response to sensing rendered haptic content is detected by detecting a reaction of the user which is indicative of discomfort or pain perceived by the user, and the model representing the anomalies in haptic perception of the user is adapted so as to reduce an intensity of renditions of the modified haptic content as compared to renditions of the acquired haptic content. For instance, this may be the case if the user suffers from an increased tactile sensitivity, or a degrade in motoric skills. The user model may be adapted iteratively, e.g., by reducing the intensity of haptic renditions by 5% each time the user perceives the haptic rendition as discomforting or painful. Eventually the user model reaches a stage which reflects the upper threshold for haptic perception of the user. Advantageously, the model representing anomalies in haptic perception of the user is refined during usage of the haptic device, and is adjusted to reflect emerging and/or changing anomalies in haptic perception of the user.

According to an embodiment of the invention, a reaction of the user in response to sensing rendered haptic content is detected by detecting a reaction of the user in response to sensing rendered haptic content with known intensity. Further, the model representing the anomalies in haptic perception of the user is adapted by comparing the detected reaction of the user to at least one previously detected reaction of the user in response to sensing rendered haptic content with the same known intensity, and adapting the model representing the anomalies in haptic perception of the user so as to maintain a substantially constant reaction of the user in response to sensing rendered haptic content with the same known intensity over time. Thereby, the user model may be adjusted to reflect changes in haptic perception of the user over time. Advantageously, rendition of haptic content with a known intensity may be utilized as part of a calibration process which may be initiated by the user, by the haptic device, e.g., if the user model appears to be inaccurate, or at regular time intervals.

Even though advantages of the invention have in some cases been described with reference to embodiments of the first aspect of the invention, corresponding reasoning applies to embodiments of other aspects of the invention.

Further objectives of, features of, and advantages with, the invention will become apparent when studying the following detailed disclosure, the drawings, and the appended claims. Those skilled in the art realize that different features of the invention can be combined to create embodiments other than those described in the following. Brief description of the drawings

The above, as well as additional objects, features and advantages of the invention, will be better understood through the following illustrative and non-limiting detailed description of embodiments of the invention, with reference to the appended drawings, in which:

Fig. 1 shows a haptic glove, in accordance with an embodiment of the invention.

Fig. 2 shows a haptic exoskeleton-type of device, in accordance with another embodiment of the invention.

Fig. 3 illustrates rendering acquired haptic content, in accordance with embodiments of the invention.

Figs. 4A and 4B illustrate adapting the user model by rendering haptic content with increasing intensity, in accordance with embodiments of the invention. Fig. 5 shows an embodiment of the processing means comprised in the haptic device.

Fig. 6 shows another embodiment of the processing means comprised in the haptic device.

Fig. 7 shows a method of a haptic device, in accordance with embodiments of the invention.

All the figures are schematic, not necessarily to scale, and generally only show parts which are necessary in order to elucidate the invention, wherein other parts may be omitted or merely suggested.

Detailed description

The invention will now be described more fully herein after with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

In Fig. 1 , an embodiment of the haptic device is illustrated as a haptic glove 100. Haptic glove 100 comprises at least one haptic actuator 101 configured to render haptic content for sensing by a user of haptic glove 100, i.e., a person wearing haptic glove 100. In the present context, haptic content is a representation of one or more haptic properties, both tactile and kinesthetic, which may vary over space and time. Haptic content may, e.g., be presented by means of a sequence of values which are arranged in accordance with a haptic data format, as an analogue signal, or as a digital signal.

In Fig. 1 , haptic actuator 101 is illustrated as covering the entire inner surface of haptic glove 100, but one may easily envisage embodiments of haptic glove 100 comprising one or more haptic actuators which cover only part of the inner surface of glove 100, and accordingly only part over the wearer's hand. Haptic actuator 101 may be any type of actuator suitable for rendering haptic content to a user and may, e.g., be based on ultrasonic transducers, vibration motors (such as eccentric rotating mass motors and linear resonant actuators), electrostatic actuators, piezoelectric actuators, magnetic actuators, electric motors, and flexible actuated materials (such as elastometers, and shape memory alloy actuators). Throughout this disclosure, the terms "haptic actuator" and "haptic actuators" are used synonymously with reference to 101 .

Further with reference to Fig. 1 , the haptic device may also be embodied in the form of a haptic smartwatch 150, i.e., a smartwatch comprising a haptic actuator in the form of a haptic touchscreen 101 . In such a scenario, haptic content which is rendered is adapted to anomalies in haptic perception of the user's finger or fingers, or the user's wrist. Haptic touchscreens are known, e.g., from the TPad smartphone by Tangible Haptics (http://www.thetpadphone.com/), which comprises a piezoelectrically actuated touchscreen capable of generating various friction and texture effects via vibration. Alternatively, or additionally, haptic smartwatch 150 may comprise a haptic actuator which is separate from the touchscreen.

In Fig. 2, a further embodiment of the haptic device is illustrated as a haptic exoskeleton-type of device 200, e.g., like the EduExo by Beyond Robotics (https://www.eduexo.com/eduexo-kit/). Similar to haptic glove 100, haptic exoskeleton-type of device 200 comprises at least one haptic actuator 101 configured to render haptic content for sensing by a user of haptic exoskeleton-type of device 200, i.e., a person wearing haptic exoskeleton-type of device 200. Haptic actuator 101 may be any type of actuator suitable for rendering haptic content to a user, in particular kinesthetic content, such as stiffness and motion. Haptic exoskeleton-type of device 200 is in Fig. 2 illustrated as comprising two elongated members 208 which are attached, via straps 209, to a forearm and an upper arm of the user. By virtue of haptic actuator 101 , elongated members 209 may be rotated relative to each other, with haptic actuator 101 acting as a hinge, so as to render kinesthetic content to the user.

Further with reference to Fig. 2, the haptic device may also be embodied in the form of a haptic smartphone 250 comprising a haptic actuator in the form of a haptic touchscreen 101 , similar to haptic

smartwatch 150 described with reference to Fig. 1 . Alternatively, or additionally, haptic smartphone 250 may comprise a haptic actuator which is separate from the touchscreen.

In the following, embodiments of the haptic device are elucidated in more detail, using haptic glove 100, haptic smartwatch 150, haptic

exoskeleton-type of device 200, and haptic smartphone 250, as examples.

In addition to haptic actuator 101 , haptic device 100/150/200/250 (in the following referred to as 100-250) further comprises processing

means 102 which is operative to cause haptic device 100-250 to perform in accordance with embodiments of the invention set forth herein. Processing means 102 is described in further detail below, with reference to Figs. 5 and 6. Reference is also made to Fig. 3, which illustrates the rendition of haptic content in accordance with embodiments of the invention.

More specifically, haptic device 100-250 may be operative to acquire haptic content 342 from a haptic content source 301 . In Fig. 3, haptic content source 301 is illustrated as being separate from haptic device 100-250, and may, e.g., be another haptic device 100-250 which is associated with the same, or a different, user. For instance, a first haptic device (e.g., haptic smartwatch 150) may receive a haptic message, e.g., a Multimedia

Messaging Service (MMS) message or an email comprising haptic content, from a second haptic device (e.g., haptic smartphone 250). Alternatively, haptic device 100-250 may receive haptic content 342 from a database or a server 301 , e.g., a server 301 providing haptic content. For instance, server 301 may provide haptic content which is related to a video or a movie provided by server 301 , or a server associated with server 301 . Haptic content source 301 may either push haptic content 342 to haptic device 100- 250 (e.g., if haptic content source 301 is another haptic device), or provide haptic content 342 to haptic device 100-250 in response to a request 341 for haptic content received from haptic device 100-250 (e.g., if haptic content source 301 is a server providing haptic content).

Haptic content 342 may be received, and request 341 for haptic content may be transmitted, via a communications interface 103 comprised in haptic device 100-250 and a communications interface comprised in haptic content source 301 . Communications interface 103 may be any type of electronic circuitry, optionally in combination with software, configured to effect communications, i.e., exchange of data, with a corresponding communications interface over a medium. Such communications may either be effected over a wired medium or a wireless medium, either directly between two haptic devices 100-250, between a haptic device and a non- haptic device (e.g., haptic glove 100 and a non-haptic smartwatch or non- haptic smartphone), or via one or more communications networks, such as a Local Area Network (LAN) or the Internet. For example, communications interface 103 may be based on Ethernet, Universal Serial Bus (USB),

Lightning, FireWire, a short-range radio technology like Bluetooth, Near-Field Communication (NFC), Zigbee, a cellular radio technology like Global System for Mobile communications (GSM), Universal Mobile

Telecommunications System (UMTS), Long Term Evolution (LTE), or a 5G technology based on NR/NX.

As an alternative to receiving haptic content 342 from an external haptic content source 301 , haptic content may also be acquired 343 from a local storage comprised in haptic device 100-250, e.g., from a memory 503 comprised in an embodiment 500 of processing means 102 (see Fig. 5), in a haptic content module 602 comprised in another embodiment 600 of processing means 102 (see Fig. 6), or in a separate memory comprised in haptic device 100-250. For instance, this may be the case if previously received haptic content 342 is buffered or stored in a local storage. Even further, haptic content may also be acquired 343 from one or more haptic sensors comprised in haptic device 100-250. For instance, haptic glove 100 may be provided with haptic sensors on an outer surface of haptic glove 100, e.g., on one or more fingertips, and may be operative to measure a haptic property of a physical object which the user of haptic glove 100 touches, and to render the measured haptic property to the user using haptic actuator 101 either with reduced or increased intensity, based on the user model.

Further with reference to Figs. 1 to 3, haptic device 100-250 is operative to generate 314 modified haptic content, or to modify 314 the acquired haptic content, based on the acquired haptic content and a model representing anomalies in haptic perception of the user, the "user model". These haptic anomalies may, e.g., be caused by injuries, diseases, thicker skin, degraded motoric skills, permanent or temporary injuries, degraded or sensitive touch sensation, or other medical conditions of the user. The user model is preferably user specific, but may alternatively be shared among a group of users. For instance, this may be the case for a model which is suitable for a group of users suffering from a certain medical condition which is associated with certain, common anomalies in haptic perception.

Embodiments of the invention may refine such a generic model during use of the haptic device, as is described herein.

Different embodiments of the user model may be envisaged. For instance, the user model may comprise normalization factors, or scaling factors, which specify to which extend an intensity of haptic content is modified, or scaled, as compared to the original, acquired haptic content, before rendering 315 the modified haptic content to the user. As an example, a normalization factor of unity ("1 ") may indicate that a certain haptic property comprised in the acquired haptic content is not changed when generating 314 the modified haptic content, whereas a normalization factor of less than (more than) unity indicates that the modified haptic property has a decreased (increased) intensity as compared to the acquired haptic content. The normalization factors may either be stored as look-up tables, e.g., categorized according to the type of haptic property (e.g., temperature, texture, stiffness, friction, motion, etc) and body part (e.g., hand, skin, arm, joint, wrist, knee, etc), or in the form of maps, with a certain spatial resolution, categorized according to the type of haptic property. As an alternative, or in addition, to the normalization or scaling factors, the user model may also comprise offsets which specify a constant value by which an intensity of haptic content is increased or decreased, respectively, as compared to the original, acquired haptic content, before rendering the modified haptic content to the user.

As an example, the user model may comprise two parameters, a scaling factor a_p (x) and an offset b_v (x), which may be used to modify 314 acquired haptic content before rendering 315 as follows:

h_m(x, t) = a_p(x) ^■ h₀ (x, t) + b_p(x) .

Here, h_Q (x, t) represents one of possibly several haptic properties which are represented by the acquired haptic content, optionally varying with space (x) and/or time (t). Correspondingly, h_m(x, t) represents the modified haptic property. Scaling factor a_p (x) and offset b_p (x) may optionally be spatially dependent (indicated by argument "x", which denotes a position (x, y, z)) and/or specific for a certain body part (indicated by index "p"). Different alternatives for determining scaling factor a_p (x) and/or offset b_v (x) are described further below. It will be appreciated that embodiments of the invention may utilize only one of scaling factor a_p ( ) and offset b_v (x), and may alternatively use any other suitable equation for generating the modified haptic content based on the acquired haptic content and the user model. Further with reference to Figs. 1 to 3, haptic device 100-250 is further operative to render 315 the modified haptic content for sensing by the user, using haptic actuator 101 .

Hereinbefore, the functionality of haptic device 100-250 has been described with respect to modifying acquired haptic content based on a model representing the anomalies in haptic perception of the user, the user model. It was assumed that the user model was available to haptic device 100-250, either as a generic user model which reflects common anomalies in haptic perception for a certain medical condition, or as a user model which has been adapted to specific anomalies in haptic perception of the user. This adaption of the user model may, e.g., be part of a calibration procedure which is performed prior to, or in-between, ordinary usage of haptic device 100-250 (e.g., as calibration process 31 1-313 in Fig. 3), and may alternatively, or additionally, be performed during ordinary usage (e.g., as process 322-323 after rendering 315 haptic content in Fig. 3). Different alternatives for adapting the user model are described in the following.

According to an embodiment of the invention, haptic device 100-250 may further be operative to detect 312/322 a reaction of the user in response to sensing rendered haptic content 31 1/315, using at least one sensor configured to detect a reaction of the user in response to sensing rendered haptic content. The reaction of the user may either be a reaction in response to a rendition 31 1 of haptic content during a dedicated calibration procedure, or in reaction to haptic content rendered 315 during ordinary usage. The at least one sensor may, e.g., be a sensor 104 configured to detect a motion of a body part of the user, e.g., the user's hand or arm (e.g., a motion sensor, an accelerometer, a force sensor, a deformation sensor, a flex sensor, or the like), a microphone 105, or camera 106, and may be comprised in haptic device 100-250 or in a separate device, such as a smartwatch, or a smartphone, which is in wireless communications with a haptic device, e.g., haptic glove 100, via communications interface 103. Haptic device 100-250 is further operative to, in response to detecting a reaction of the user in response to sensing rendered haptic content, to adapt 313/323 the model representing the anomalies in haptic perception of the user based on the detected reaction of the user.

As an example, haptic device 100-250 may be operative to detect a reaction of the user in response to sensing rendered haptic content by rendering 31 1 haptic content with increasing intensity until a reaction of the user is detected 312 which is indicative of a haptic perception by the user, i.e., that the user has sensed the rendered 31 1 haptic content. As is illustrated in Figs. 4A and 4B, the intensity may either be increased continuously over time (see Fig. 4A), or step-wise (see Fig. 4B). The intensity in increased until the user reaction is detected 312 at time to, when the haptic content is rendered 31 1 with a current intensity /Ό. Haptic device 100-250 is further operative to adapt 313 the model representing the anomalies in haptic perception of the user based on the current intensity i₀ of the rendered 31 1 haptic content when the reaction of the user was detected 312. Thereby, a lower threshold for haptic perception by the user may be established.

As another example, the detected 312 reaction of the user may, e.g., be an explicit confirmation by the user that he/she has sensed the

rendered 31 1 haptic content. The user confirmation may, e.g., be a gesture performed by the user, such as a wrist flick, a hand gesture (e.g., forming a fist), an arm gesture (e.g., bending the arm), or a finger gesture (e.g., bending one or more fingers). Such a gesture may be detected using sensors comprised in haptic device 100-250, e.g., motion sensors 104 which are configured to detect a relative movement of the fingers of the user's hand, a relative movement of the forearm and the upper arm, a rotation of the hand, or the like. Alternatively, the gesture may be detected using a camera 106. As a further alternative, the gesture may be detected using motion

sensors 104 and/or a camera 106 comprised in a separate device, such as a smartwatch worn by the user (like smartwatch 150 shown in Fig. 1 ), or a smartphone held by the user (like smartphone 250 shown in Fig. 2), which is in wireless communications with a haptic device, e.g., haptic glove 100, via communications interface 103. In this case, haptic glove 100 is operative to receive information about a gesture performed by the user from

smartwatch 150, which is operative to detect the gesture using motion sensors 104 or camera 106.

The user confirmation may alternatively be a command spoken by the user, to indicate that he/she has sensed the rendered 31 1 haptic content. The spoken command is detected by a microphone 105 comprised in haptic device 100-250, or a microphone 105 comprised in a separate device such as a smartwatch, or a smartphone, which is in wireless communications with a haptic device, e.g., haptic glove 100, via communications interface 103. In this case, haptic glove 100 is operative to receive information about a spoken command from smartwatch 150, which is operative to detect the command spoken by the user using microphone 105.

The user confirmation may, as yet a further alternative, be the user pressing a button so as to confirm that he/she has sensed the rendered 31 1 haptic content. The button may e.g., be comprised in haptic device 100-250, either as a physical button or as a virtual button displayed on a (haptic or non-haptic) touchscreen comprised in smartwatch 150 and smartphone 250 (not illustrated in Figs. 1 and 2). Alternatively, the button may be comprised in a separate device, such as a smartwatch, or a smartphone, which is in wireless communications with a haptic device, e.g., haptic glove 100, via communications interface 103. In this case, haptic glove 100 is operative to receive information about a pressed button from smartwatch 150, which is operative to detect that the user has pressed a button.

According to another embodiment of the invention, haptic device 100- 250 may be operative to detect a reaction of the user in response to sensing rendered 31 1/315 haptic content by detecting 312/322 a reaction of the user which is indicative of discomfort or pain perceived by the user, and adapt 313/323 the model representing the anomalies in haptic perception of the user so as to reduce an intensity of renditions of the modified 314 haptic content, as compared to renditions of the acquired 342/343 haptic content. The reaction of the user which is indicative of pain may, e.g., be a motion of a body part of the user as a reaction to pain, which is detected by motion sensor 104 or camera 106, or an utterance which is indicative of discomfort or pain, which is detected by microphone 105. For instance, if the user is overly sensitive to haptic renditions, the user model may be adapted iteratively, e.g., by reducing the intensity of haptic renditions by 5% each time it is detected that the user perceives a haptic rendition as painful. Thereby, the user model will eventually reach a stage which reflects an upper threshold for haptic perception of the user.

According to a further embodiment of the invention, haptic

device 100-250 may be operative to detect a reaction of the user in response to sensing rendered haptic content by detecting 312/322 a reaction of the user in response to sensing rendered 31 1/315 haptic content with known intensity. Haptic device 100-250 is further operative to adapt 313/323 the model representing the anomalies in haptic perception of the user by comparing the detected 312/322 reaction of the user to at least one previously detected reaction of the user in response to sensing rendered haptic content with the same known intensity, and adapting the model representing the anomalies in haptic perception of the user so as to maintain a substantially constant reaction of the user in response to sensing rendered haptic content with the same known intensity over time. The haptic content with known intensity may, e.g., be haptic content which is used for calibration purposes. Alternatively, the haptic content with known intensity may, e.g., be a type of haptic content which is rendered occasionally, e.g., as part of a computer game which the user plays, or as a notification triggered by a service, an incoming call or message, or the like. By comparing detected reactions of the user in response to sensing the rendered haptic content with the same, known intensity over time, changes in haptic perception of the user can be discovered, and the user model can be adapted accordingly. For instance, if the detected reaction of the user is indicative of discomfort or pain, and previously detected reactions of the user have not been indicative of discomfort or pain, the user model may be adapted to reduce the intensity of rendered haptic content. Similarly, if a detected reaction of the user in comparison to previously detected reactions indicates that the user's haptic perception has become less sensitive, the user model may be adapted to increase the intensity of rendered haptic content.

According to yet another embodiment, haptic device 100 or 200 may be operative to determine an upper threshold for rendering kinesthetic content by measuring a movement of a body part of the user, e.g., one or more fingers or an arm of the user, and to use the measured values as one or more upper threshold values. This may, e.g., be performed as part of a calibration procedure, during which the user, optionally on request by haptic device 100 or 200, performs a movement of one or more body parts which he/she perceives as comfortable, i.e., not straining or painful. In that way, one or more upper thresholds for rendering kinesthetic content may be established, without the need to render haptic content at an intensity which potentially causes strain or pain for the user.

The model representing anomalies in haptic perception of the user, the user model, is preferably maintained in a local storage of haptic

device 100-250, e.g., in a memory 503 comprised in processing means 102 (see Fig. 5), in a haptic perception module 603 comprised in processing means 102 (see Fig. 6), or in a separate memory comprised in haptic device 100-250. As an alternative, the user model may be maintained in an external storage 302 which is accessible by haptic device 100-250 over communications interface 103, e.g., a cloud storage. Preferably, the user model is maintained both in a local storage and in external storage 302, and haptic device 100-250 is operative to synchronize changes of the user model so as to maintain substantially identical user models in both storage locations. More specifically, and with reference to Fig. 3, haptic device 100- 250 may be operative to push changes of the locally stored user model, which may result from adaptions 313/323 of the locally stored user model, as user model updates 351/354 to external storage 302. Haptic device 100-250 may further be operative to receive user model updates 353 from external storage 302, optionally in response to a request 352 for user model updates. It will be appreciated that haptic device 100-250 and external storage 302 may synchronize their respective copies of the user model, i.e., exchange information about the user model or changes thereof, either regularly or occasionally, e.g., triggered by adaptions 313/323 of the user model or depending on availability of bandwidth for communications via

communications interface 103. In particular, synchronization of the user model is not limited to the occasions illustrated in Fig. 3

Maintaining the user model in both a local storage and external storage 302 is advantageous in that excess latency in accessing the user model by haptic device 100-250 is avoided, if the user model is stored in a memory comprised in haptic device 100-250, while at the time allowing to synchronize changes to the user model across different haptic devices of a user. Thereby, changes to haptic perception of the user which are detected by a first haptic device of the user, e.g., haptic glove 100, can be made available to other haptic devices of the user, e.g., haptic exoskeleton-type of device 200. This is particularly advantageous if haptic devices with different haptic-rendition capabilities are used, as information about the respective local user model, or changes thereof, can be collected and shared among the haptic devices of a user in order to maintain a more general model of the user's anomalies in haptic perception.

In the following, embodiments of processing means 102 comprised in haptic device 100-250 are described with reference to Figs. 5 and 6. A first embodiment 500 of processing means 102 is shown in Fig. 5. Processing means 500 comprises a processing unit 502, such as a general- purpose processor, and a computer-readable storage medium 503, such as a Random-Access Memory (RAM), a Flash memory, or the like. In addition, processing means 500 comprises one or more interfaces 501 ("I/O" in Fig. 5) for controlling and/or receiving information from other components comprised in haptic device 100-250, such as haptic actuator 101 , communications module 103, and one or more sensors, e.g., sensor(s) 104, microphone 105, and camera 106. Memory 503 contains computer-executable

instructions 504, i.e., a computer program or software, to cause haptic device 100-250 to become operative in accordance with embodiments of the invention as described herein, when computer-executable instructions 504 are executed on processing unit 502.

An alternative embodiment 600 of processing means 102 is illustrated in Fig. 6. Similar to processing means 500, processing means 600 comprises one or more interfaces 601 ("I/O" in Fig. 6) for controlling and/or receiving information from other components comprised in haptic device 100-250, such as haptic actuator 101 , communications module 103, and one or more sensors, e.g., sensor(s) 104, microphone 105, and camera 106. Processing means 600 further comprises a haptic content module 602, a haptic perception module 603, and a haptic rendition module 604, which are configured to cause the haptic garment to become operative in accordance with embodiments of the invention as described herein.

In particular, haptic content module 602 is configured to acquire haptic content, haptic perception module 603 is configured to generate modified haptic content based on the acquired haptic content and a model

representing anomalies in haptic perception of the user, and haptic rendition module 604 is configured to render the modified haptic content using haptic actuator 101 . Optionally, haptic perception module 603 is further configured to detect a reaction of the user in response to sensing rendered haptic content, using at least one sensor configured to detect a reaction of the user in response to sensing rendered haptic content, and in response thereto adapt the model representing the anomalies in haptic perception of the user based on the detected reaction of the user.

For instance, haptic perception module 603 may be configured to detect a reaction of the user in response to sensing rendered haptic content by rendering haptic content with increasing intensity until a reaction of the user is detected which is indicative of a haptic perception by the user, and adapt the model representing the anomalies in haptic perception of the user based on a current intensity of the rendered haptic content when the reaction of the user was detected.

As another example, haptic perception module 603 may further be configured to detect a reaction of the user in response to sensing rendered haptic content by detecting a reaction of the user which is indicative of discomfort or pain perceived by the user, and adapt the model representing the anomalies in haptic perception of the user so as to reduce an intensity of renditions of the modified haptic content as compared to renditions of the acquired haptic content.

As a further example, haptic perception module 603 may be configured to detect a reaction of the user in response to sensing rendered haptic content by detecting a reaction of the user in response to sensing rendered haptic content with known intensity, compare the detected reaction of the user to at least one previously detected reaction of the user in response to sensing rendered haptic content with the same known intensity, and adapt the model representing the anomalies in haptic perception of the user so as to maintain a substantially constant reaction of the user in response to sensing rendered haptic content with the same known intensity over time. Modules 602-604 comprised in processing mean 600 may further be configured to perform additional or alternative operations in accordance with embodiments of the invention, as described herein.

Interfaces 501 and 601 , and modules 602-604, as well as any additional modules comprised in processing means 600, may be

implemented by any kind of electronic circuitry, e.g., any one, or a

combination of, analogue electronic circuitry, digital electronic circuitry, and processing means executing a suitable computer program, i.e., software.

In the following, embodiments 700 of the method of performed by a haptic device are described with reference to Fig. 7.

Method 700 comprises acquiring 751 haptic content, generating 752 modified haptic content based on the acquired haptic content and a model representing anomalies in haptic perception of the user, and rendering 753 the modified haptic content using at least one haptic actuator comprised in the haptic device and configured to render haptic content for sensing by a user of the haptic device.

Optionally, method 700 may further comprise detecting a reaction of the user in response to sensing rendered haptic content using at least one sensor configured to detect a reaction of the user in response to sensing rendered haptic content, and in response thereto adapting the model representing the anomalies in haptic perception of the user based on the detected reaction of the user (step 750 in Fig. 7).

For instance, the detecting a reaction of the user in response to sensing rendered haptic content may comprise rendering 71 1 haptic content with increasing 713 intensity until a reaction of the user is detected 712 which is indicative of a haptic perception by the user. The model representing the anomalies in haptic perception of the user is adapted 714 based on a current intensity of the rendered haptic content when the reaction of the user was detected 712. As another example, the detecting a reaction of the user in response to sensing rendered 721 haptic content may comprise detecting 722 a reaction of the user which is indicative of discomfort or pain perceived by the user. The model representing the anomalies in haptic perception of the user is adapted 724 so as to reduce an intensity of renditions of the modified haptic content as compared to renditions of the acquired haptic content.

As a further example, the detecting a reaction of the user in response to sensing rendered 741 haptic content may comprise detecting 742 a reaction of the user in response to sensing rendered haptic content with known intensity. The adapting the model representing the anomalies in haptic perception of the user may comprise comparing 743 the detected reaction of the user to at least one previously detected reaction of the user in response to sensing rendered haptic content with the same known intensity, and adapting 744 the model representing the anomalies in haptic perception of the user so as to maintain a substantially constant reaction of the user in response to sensing rendered haptic content with the same known intensity over time.

It will be appreciated that method 700 may comprise additional, or modified, steps in accordance with what is described throughout this disclosure. An embodiment of method 700 may be implemented as software, such as computer program 504, to be executed by a processing unit comprised in a haptic device, such as a device comprising a haptic touchscreen, or a haptic wearable, such as a haptic garment, a haptic glove, or an exoskeleton-type of haptic device, whereby the haptic device becomes operative in accordance with embodiments of the invention described herein.

The person skilled in the art realizes that the invention by no means is limited to the embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.

Claims

1 . A haptic device (100, 150; 200, 250) comprising:

at least one haptic actuator (101 ) configured to render haptic content for sensing by a user of the haptic device,

the haptic device being operative to:

acquire (342, 343) the haptic content,

generate (314) modified haptic content based on the acquired haptic content and a model representing anomalies in haptic perception of the user, and

render (315) the modified haptic content using the at least one haptic actuator (101 ) for sensing by the user.

2. The haptic device according to claim 1 , being further operative to: detect (312, 322) a reaction of the user in response to sensing rendered (31 1 , 315) haptic content, using at least one sensor (104-106) configured to detect a reaction of the user in response to sensing rendered haptic content, and

in response thereto adapt (313, 323) the model representing the anomalies in haptic perception of the user based on the detected reaction of the user.

3. The haptic device according to claim 2, being operative to:

detect a reaction of the user in response to sensing rendered haptic content by rendering (31 1 ) haptic content with increasing intensity until a reaction of the user is detected (312) which is indicative of a haptic perception by the user, and

adapt (313) the model representing the anomalies in haptic perception of the user based on a current intensity of the rendered haptic content when the reaction of the user was detected.

4. The haptic device according to claim 2, being operative to:

detect a reaction of the user in response to sensing rendered (31 1 , 315) haptic content by detecting (312, 322) a reaction of the user which is indicative of discomfort or pain perceived by the user, and

adapt (313, 323) the model representing the anomalies in haptic perception of the user so as to reduce an intensity of renditions of the modified haptic content as compared to renditions of the acquired haptic content.

5. The haptic device according to claim 2, being operative to:

detect a reaction of the user in response to sensing rendered haptic content by detecting (312, 322) a reaction of the user in response to sensing rendered (31 1 , 315) haptic content with known intensity, and

adapt (313, 323) the model representing the anomalies in haptic perception of the user by:

comparing the detected reaction of the user to at least one previously detected reaction of the user in response to sensing rendered haptic content with the same known intensity, and

adapting the model representing the anomalies in haptic perception of the user so as to maintain a substantially constant reaction of the user in response to sensing rendered haptic content with the same known intensity over time.

6. The haptic device according to any one of claims 2 to 5, wherein the at least one sensor is one of a motion sensor (104), an

accelerometer (104), a force sensor (104), a deformation sensor (104), a flex sensor (104), a microphone (105), and a camera (106).

7. The haptic device according to any one of claims 1 to 6, wherein the model representing the anomalies in haptic perception of the user is maintained in an external storage location (302) which is accessible by the haptic device (100, 150; 200, 250) over a communications interface comprised in the haptic device.

8. The haptic device according to any one of claims 1 to 7, being any one of a device (150; 250) comprising a haptic touchscreen (101 ), a haptic garment (100), a haptic glove (100), and an exoskeleton-type of haptic device (200).

9. A method (700) performed by a haptic device, the method comprising:

acquiring (342, 343; 751 ) haptic content,

generating (314; 752) modified haptic content based on the acquired haptic content and a model representing anomalies in haptic perception of the user, and

rendering (315; 753) the modified haptic content using at least one haptic actuator comprised in the haptic device and configured to render haptic content for sensing by a user of the haptic device.

10. The method according to claim 9, further comprising:

detecting (312, 322) a reaction of the user in response to sensing rendered (31 1 , 315) haptic content using at least one sensor configured to detect a reaction of the user in response to sensing rendered haptic content, and

in response thereto adapting (313, 323) the model representing the anomalies in haptic perception of the user based on the detected reaction of the user.

1 1 . The method according to claim 10, the detecting a reaction of the user in response to sensing rendered haptic content comprising

rendering (31 1 ; 71 1 ) haptic content with increasing (713) intensity until a reaction of the user is detected (312; 712) which is indicative of a haptic perception by the user, wherein the model representing the anomalies in haptic perception of the user is adapted (313; 714) based on a current intensity of the rendered haptic content when the reaction of the user was detected.

12. The method according to claim 10, the detecting a reaction of the user in response to sensing rendered (31 1 ,315; 721 ) haptic content comprising detecting (312, 322; 722) a reaction of the user which is indicative of discomfort or pain perceived by the user, wherein the model representing the anomalies in haptic perception of the user is adapted (31 1 , 323; 724) so as to reduce an intensity of renditions of the modified haptic content as compared to renditions of the acquired haptic content.

13. The method according to claim 10, the detecting a reaction of the user in response to sensing rendered haptic content comprising

detecting (312, 322; 742) a reaction of the user in response to sensing rendered (31 1 , 315; 741 ) haptic content with known intensity, and

the adapting (313, 323) the model representing the anomalies in haptic perception of the user comprising:

comparing (743) the detected reaction of the user to at least one previously detected reaction of the user in response to sensing rendered haptic content with the same known intensity, and

adapting (744) the model representing the anomalies in haptic perception of the user so as to maintain a substantially constant reaction of the user in response to sensing rendered haptic content with the same known intensity over time.

14. The method according to any one of claims 10 to 13, wherein the at least one sensor is one of a motion sensor, an accelerometer, a force sensor, a deformation sensor, a microphone, and a camera.

15. The method according to any one of claims 9 to 14, wherein the model representing the anomalies in haptic perception of the user is maintained in an external storage location which is accessible by the haptic device over a communications interface comprised in the haptic device.

16. The method according to any one of claims 9 to 15, the haptic device being any one of a device comprising a haptic touchscreen, a haptic garment, a haptic glove, and an exoskeleton-type of haptic device.

17. A computer program (504) comprising computer-executable instructions for causing a haptic device to perform the method according to any one of claims 9 to 16, when the computer-executable instructions are executed on a processing unit (502) comprised in the device.

18. A computer program product comprising a computer-readable storage medium (503), the computer-readable storage medium having the computer program (504) according to claim 17 embodied therein.