WO2013187977A1 - Skin stretch feedback devices, systems, and methods - Google Patents

Skin stretch feedback devices, systems, and methods Download PDF

Info

Publication number
WO2013187977A1
WO2013187977A1 PCT/US2013/032053 US2013032053W WO2013187977A1 WO 2013187977 A1 WO2013187977 A1 WO 2013187977A1 US 2013032053 W US2013032053 W US 2013032053W WO 2013187977 A1 WO2013187977 A1 WO 2013187977A1
Authority
WO
WIPO (PCT)
Prior art keywords
tactor
user
body
display device
shear display
Prior art date
Application number
PCT/US2013/032053
Other languages
French (fr)
Inventor
William R. Provancher
Markus N. MONTANDON
Landon T. Gwilliam
Original Assignee
The University Of Utah Research Foundation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US201261659421P priority Critical
Priority to US61/659,421 priority
Priority to US201261659452P priority
Priority to US61/659,452 priority
Priority to US61/660,162 priority
Priority to US201261660162P priority
Application filed by The University Of Utah Research Foundation filed Critical The University Of Utah Research Foundation
Publication of WO2013187977A1 publication Critical patent/WO2013187977A1/en
Priority claimed from US14/517,674 external-priority patent/US9317123B2/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/02Accessories
    • A63F13/06Accessories using player-operated means for controlling the position of a specific area display
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B6/00Tactile signalling systems, e.g. personal calling systems
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63FCARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1043Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being characterized by constructional details

Abstract

Implementations of the present invention relate devices, systems, and methods for displaying information about the direction and magnitude of position, movement, and/or resistive force experienced for an object. The present invention also provides a shear display device that can generate skin shear with one or more tactors, each moving in a two- or three-dimensional space. The movement of the tactors can represent to a user various information about an object.

Description

SKIN STRETCH FEEDBACK DEVICES, SYSTEMS, AND METHODS

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application claims the benefit of and priority to: U.S. Provisional Patent Application No. 61/659,421, filed June 13, 2012, entitled "Skin Stretch Feedback Devices, Systems, and Methods"; U.S. Provisional Application No. 61/659,452, filed June 14, 2012, entitled "Skin Stretch Feedback Devices, Systems, and Methods"; and U.S. Provisional Application No. 61/660,162, filed June 15, 2012, entitled "Skin Stretch Feedback Devices, Systems, and Methods." The entire content of each of the above-referenced applications is incorporated herein by this reference.

BACKGROUND OF THE INVENTION

1. The Field of the Invention

The present disclosure relates generally to haptic feedback devices and more specifically to the devices that provide tactile shear feedback.

2. Background and Relevant Art

The field of haptics is the science of interfacing with users via the sense of touch by applying forces, vibrations, or motions to a user. Haptic devices are increasingly used to provide a user with sensory input that conveys information about the surrounding environment. For instance, a haptic device can produce vibratory motions to provide the user, through his or her sense of touch, with various types of information. Haptic devices are commonly used in the field of gaming to provide sensory cues related to the user's environment.

[0002] Ordinarily, haptic devices are integrated into controllers (e.g., joystick), so the user can receive haptic feedback that relates to movements of the controller and/or an object being controlled. Typical haptic devices provide vibratory or force stimuli to display information to the user. In some instances, however, such stimuli can interfere with the user's ability to accurately and/or safely manipulate the controller that provides such haptic feedback. For example, force feedback on a controller that is used to operate a crane can result in unintended movements of the user's hand, which may, in turn, move the controller to an unintended position, thereby causing an accident. Similar results occur in video games when utilizing a force feedback joystick to control game play; however, this resulting loss of control has less catastrophic consequences. Hence, in some instances, vibration feedback and especially force feedback may reduce safety of operation of a controller that integrates such haptic feedback mechanisms. This is especially of concern in safety critical applications, such as robotic surgery or catheterization. Such concerns are the reason why force feedback of tool tip forces are not currently permitted in robotically enabled surgical applications.

[0003] Accordingly, there is a need for skin stretch feedback devices, systems, and methods.

BRIEF SUMMARY OF THE INVENTION

[0004] Implementations of the present invention provide devices, systems, and methods for displaying information about direction of movement, speed, resistive force experienced, other aspects of movement for an object, or combinations thereof. The device can also be used to display or feedback a wide variety of information that has magnitude and/or direction associated with it (e.g., temperature, motion, pressure, force, volume, proximity, other information, or combinations thereof) or give guidance about where a person should move (e.g., to push forward on an aircraft control stick to prevent stall from occurring). More specifically, the present disclosure provides a shear display device that can generate skin shear with one or more tactors moving in a two- or three- dimensional space. The movement of the tactors can represent to a user various aspects of an object (e.g., object being controlled by the user or a controlled object).

[0005] At least one embodiment of the present invention includes a shear display device for displaying tactile information and cues to a user. Such the device may include a body, a first motor and a crank coupled to the first motor. The device also may include a slider slidably positioned within the body, a first end of the slider being coupled to the crank in a manner that rotation of the first motor produces a linear movement of the slider. In addition, the device may include a tactor coupled to the slider.

[0006] One or more embodiments may include another shear display device for displaying tactile information and cues to a user. The device may incorporate a body, a first actuator assembly at least partially located within or secured to the body, the actuator assembly. The actuator assembly may include a sliding housing, a first motor secured to the sliding housing, a worm coupled to the first motor, and one or more gears engaged with the worm, the one or more gears being oriented substantially orthogonally relative to the worm. Additionally, the actuator assembly may include a cam coupled to the one or more gears, wherein the body includes a slot configured to accept the cam in a manner that rotation of the cam within the slot and produces movement of the sliding housing relative to the body. Moreover, the device may include a first tactor coupled to the sliding body in a manner that the first tactor can move relative to the body.

[0007] In addition, embodiments of the present invention may include yet another shear display device for displaying tactile information and cues to a user. The device may have a body sized and configured to be grasped by the user's hand and an actuator assembly at least partially located within or secured to the body. The actuator assembly may include a motor, a crank coupled to the motor, and a flexible spine having a first end coupled to the crank in a manner that rotation of the motor in a clockwise direction moves the flexible spine in a first direction and rotation of the crank in a counterclockwise direction moves the flexible spine in a second direction that is opposite to the first direction. The device also may include a first tactor coupled to a second end of the flexible spine in a manner that the flexible spine moves the first tactor in the first and second directions.

[0008] Yet another embodiment of the present invention may include one other shear display device for displaying tactile information and cues to a user. The device may have a body and a first tactor having a first area, the first tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a first density of mechanoreceptors. The device also may include a second tactor having a second area, the second tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a second density of mechanoreceptors. Furthermore, the second area may be greater than the first area, and the first density of mechanoreceptors may be greater than the second density of mechanoreceptors.

[0009] Embodiment also may include still one other shear display device for displaying tactile information and cues to a user. The device may incorporate a body having an elongated portion and a first tactor positioned along the elongated portion of the body, the first tactor being movable along a length of the elongated portion of the body. The device also may have a second tactor positioned along the elongated body, the second tactor being movable along the length of the elongated portion of the body, the second tactor being opposite to the first tactor. In addition, the device may include a third tactor positioned along the elongated body, the third tactor being movable along the length of the elongated portion of the body. In addition, the device may include a fourth tactor positioned along the elongated body, the fourth tactor being movable along the length of the elongated portion of the body. [0010] Additional or alternative embodiments may include one other shear display device for displaying tactile information and cues to a user. The device may have a body and a first tactor having a first area, the first tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a first density of mechanoreceptors. The device also may include a second tactor having a second area, the second tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a second density of mechanoreceptors. Furthermore, the first area may have a first proportion relative to the first density of mechanoreceptors, while the second area may have a second proportion relative to the second density of mechanoreceptors. In addition, the first proportion and the second proportion may be approximately the same. One or more embodiments may include a control system for controlling an object and receiving tactile feedback about the movement of the object, forces experienced by the object, torques experienced by the object, and combinations thereof. The system may include a shear display device including, which may have a body, an actuator assembly at least partially located with or secured to the body, and a first tactor coupled to the actuator assembly, the first tactor being movable in a two-dimensional or a three- dimensional space by the actuator assembly. The system also may include a controller operably connected to the shear display device, the controller being configured to receive instructions from the shear display device and to communicate the instructions to a controlled object.

[0011] Embodiments of the present invention also may involve a method for displaying movement information related to one or more objects as well as information about torque or rotational motion experienced thereby. The method may include receiving information about one or more of rotation of an object and torque experienced by the object, isolating a first portion of a user's skin relative to a body of a shear display device, and moving a first tactor of the shear display device in a first direction along a linear path, the first tactor being in contact with the isolated first portion of the user's skin. The method also may include isolating a second portion of a user's skin relative to a body of the shear display device and moving a second tactor of the shear display device in a second direction along a linear path, the second tactor being in contact with the isolated second portion of the user's skin, the second direction being opposite to the first direction.

[0012] Also, embodiments may include a method for displaying information about a change in size of an object. The method may include receiving information about the change in size of the object, isolating a first portion of a user's skin relative to a body of a shear display device, and moving a first tactor of the shear display device in a first direction along a linear path, the first tactor being in contact with the isolated first portion of the user's skin. The method also may include isolating a second portion of a user's skin relative to a body of a shear display device and moving a second tactor of the shear display device in a second direction along a linear path, the second tactor being in contact with the isolated second portion of the user's skin, the second direction being opposite to the first direction and away from or towards the first tactor.

[0013] Additional features and advantages of exemplary implementations of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of such exemplary implementations. The features and advantages of such implementations may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. These and other features will become more fully apparent from the following description and appended claims, or may be learned by the practice of such exemplary implementations as set forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to describe the manner in which the above-recited and other advantages and features of the invention can be obtained, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. For better understanding, the like elements have been designated by like reference numbers throughout the various accompanying figures. Understanding that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:

[0015] Figure 1A illustrates a perspective view of a shear display device in accordance with one implementation of the present invention;

[0016] Figure IB illustrates an exploded perspective view of the shear display device of Figure 1 A;

[0017] Figure 2 A illustrates a perspective view of a shear display device in accordance with another implementation of the present invention;

[0018] Figure 2B illustrates a perspective view of an actuation mechanism in accordance with one implementation of the present invention; [0019] Figure 2C illustrates a bottom perspective view of an actuator assembly that incorporates the actuation mechanism of Figure 2B;

[0020] Figure 2D illustrates a top perspective view of an actuator assembly that incorporates the actuation mechanism of Figure 2B;

[0021] Figure 3 A illustrates a perspective view of an actuator assembly in accordance with one or more implementations of the present invention;

[0022] Figure 3B illustrates a side view of the actuator assembly of Figure 3 A;

[0023] Figure 4 illustrates a perspective view of a shear display device in accordance with yet another implementation of the present invention;

[0024] Figure 5 illustrates a perspective view of a shear display device in accordance with still one other implementation of the present invention;

[0025] Figure 6 illustrates a perspective view of a shear display device in accordance with one or more implementations of the present invention;

[0026] Figure 7A illustrates a perspective view of a shear display device in accordance with yet one other implementation of the present invention;

[0027] Figure 7B illustrates a partial cutaway perspective view of the shear display device of Figure 7A;

[0028] Figure 8 illustrates a perspective view of a shear display device in accordance with still one other implementation of the present invention;

[0029] Figure 9 illustrates a perspective view of a shear display device in accordance with one or more implementations of the present invention;

[0030] Figure 10 illustrates a perspective view of a shear display device in accordance with an implementation of the present invention;

[0031] Figure 11 illustrates a perspective view of a shear display device in accordance with at least one other implementation of the present invention;

[0032] Figure 12 illustrates a perspective view of a control system that incorporates a shear display device in accordance with one implementation of the present invention; and

[0033] Figure 13 illustrates a chart of acts of a method of displaying information via tactile cues in accordance with one implementation of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Implementations of the present invention provide devices, systems, and methods for displaying information about direction of movement, speed, resistive force experienced, other aspects of movement for an object, or combinations thereof. The device can also be used to display or feedback a wide variety of information that has magnitude and/or direction associated with it (e.g., temperature, pressure, volume, proximity, other information, or combinations thereof) or give guidance about where a person should move (e.g., to push forward on an aircraft control stick to prevent stall from occurring). More specifically, the present disclosure provides a shear display device that can generate skin shear or skin stretch with one or more tactors moving in a two- or three- dimensional space. The tactors (or contactors) are the moving contacts between the shear display device and the user's skin, and the terms shear display, shear feedback, skin stretch, and skin stretch feedback are used interchangeably to refer to tactile cues provided via stretching one or more portions of the user' skin. The movement of the tactors can represent to a user various aspects of the movement of the object (e.g., object being controlled by the user or a controlled object). As used herein, the use of the term "controlled object" also includes the user's own body, limbs, arms, fingers, and hand.

[0035] For instance, the movement of the tactors can represent linear (or translational) direction and/or speed of the object's movement, or that a user should translate their hand or limb in two- or three-dimensional space. Furthermore, the movement of the tactors also can represent rotation of the object in two- or three- dimensional space, or that a user should rotate their hand or limb in two- or three- dimensional space. Similarly, movement of the tactors can display information about force and/or torque experienced by the object. With such information, the user may direct movement of the object (or their limbs) more accurately. For example, the user can control the amount of force applied by the controlled object onto another body as well as location and/or direction of the force. A controlled object can be any automated or semi- automated vehicle, tool, or other implement, movement of which can be directed by a controller. Examples of controlled objects include controlled servo and stepper motors or other actuators, computer controlled machines, vehicles, robots, etc. A controlled object could also include the human user, e.g., for using shear feedback to guide their limb motions during physical therapy.

[0036] Additionally, while providing information about various aspects of movement (e.g., acceleration, velocity, etc.) and/or location (e.g., orientation, position, etc.) of the controlled object, at least some embodiments of the shear display device do not generate gross motion of the user's hand (or other body part) that is in contact with the shear display device. In particular, the shear display device can be part of a controller (e.g., a joystick) that directs movement of the controlled object. The controller with shear feedback can allow the user to control or direct the controlled object, while displaying information about the controlled object's movement (i.e., through skin stretch rather than gross movement of the joystick). Moreover, the controller with shear feedback can display movement information (i.e., through skin stretch) without affecting the movement and/or location of the user's hand such as to interfere with the operation of the controller. In other words, in one or more embodiments, while the shear display device provides or (physically) displays information to the user, the shear display device does not move the user's limb, but rather only a small portion of the skin of the user's body, which may include the user's head, arm, hand, finger, other body part, or combinations thereof (for multiple tactors).

[0037] In some embodiments, the shear display device also can provide a simulated force feel to the user's hand, finger, or another body part. More specifically, the skin shear produced by the shear display device can be similar to the sensation otherwise felt by the user while using an actual tool, similar to the tool that is being controlled and operated by an automated system (e.g., a robot). Hence, for example, a surgeon can receive shear feedback while controlling an automated or robotically controlled scalpel, and such shear feedback can approximate the shear that a manual scalpel would produce on the skin of the surgeon's hand. Shear feedback can also be used to provide force information or guidance cues for procedures such as catheterization, rehabilitation, laser/bone/vascular etc. surgery or resection. For example, when used for rehabilitation, shear feedback can guide a person's limb motions as part of physical therapy, etc.

[0038] In some embodiments, a controller can detect when the user is not in contact with the tactor of the shear display device. For instance, the shear display device can include a touch sensor, which can recognize contact with user's skin (or lack thereof). This sensor could include, but is not limited to a capacitive sensor, electrostatic sensor, contact switch, force sensor, etc. Accordingly, the shear display device and/or the controller can respond to loss of contact with the user's skin, in a manner that would avoid displaying inaccurate information to the user. For example, if loss of tactor's contact with the user's skin is detected, the tactor can move to its default position (e.g., an origin or zero position, such as a center of a finger well) and cease all further operation, until contact is reestablished. Once contact is reestablished, the tactor can once again commence displaying information to the user. Additionally, the shear display device can alert the user about the loss of contact with the tactor as well as about reestablishment of contact (e.g., by providing an audible alert).

[0039] The tactor of a shear display can also be coupled to or have a force sensor embedded within it. This force sensor can be used for the above purpose of knowing when the user's hand is in contact with the shear tactor(s). The force sensor can also be used as a user input for the controller, similar in spirit as a ThinkPad laptop's TrackPoint cursor control sensor. Alternatively or in addition, this embedded force sensor can also be used to more generally measure user interaction forces with the shear display's tactor(s). These force sensors may provide a means to control translational and rotational input or motion to a device, object, robot, vehicle, or other computer controlled system. As will be described below, whereas like motion of multiple tactors can be used to provide translational information such as direction cues or display of object interaction force or motion, and the differential motion of multiple shear tactors can be used to present rotational information to the user, a user is also able to provide translational and rotational input to the force sensors on the multiple tactors in an analogous manner. That is, if the user applies the same force in the same direction to multiple tactors, this would cause a pure translational input to the control system, and if the user applies the same force in opposite directions (i.e., a force couple) to multiple tactors, this would cause a rotational input to the control system.

[0040] If different forces are applied to the two (or more) afore mentioned force sensors, the force direction and magnitude can be taken into account in determining the meaning of this input. One intuitive means to interpret having different inputs force vectors applied to multiple tactors would be to solve for the resultant forces and torques on the system of force sensors. Hence, one would examine the applied force vectors and the relative location of the force sensors in order to solve for the net translational force and moment applied to the system of force sensors. This can provide a means to provide multidimensional input control, including rotations, of a system using multiple 2- or 3- degree-of-freedom translational force sensors. This provides the ability to measure user input through applied forces on the shear display tactors, as opposed to just capturing the user's motion by tracking the position of the held or strapped-on shear display (e.g., by putting the shear display on a robot arm or by using some type of non-contact motion tracker). Furthermore, this scheme of capturing user input from one or more tactors with embedded force sensors is the "input" equivalent of moving the tactors together to give translational direction/motion/force cues or feedback and moving the tactors differentially to give rotational direction/motion/force etc., cues or feedback (which will be discussed further below).

[0041] Note, that in addition to tracking the position of the shear display by mounting it on the end of a robotic arm (with force feedback) or kinematic arm (without force feedback), it is also possible to utilize single or multiple shear displays within a shear display device or device with embedded shear feedback in combination with any type of motion tracking system to provide tactile feedback. This tactile feedback can be used to provide relevant situational or control information. For example, a device with single or multiple shear displays within it could be paired with a wireless camera system (e.g. Xbox Kinect) or other motion system (e.g., Nintendo Wii Wiimote motion/position sensor, or Sony Move motion sensor), other non-contact motion sensing (e.g., Polhemus, Razer Hydra, flock of birds inductive/electromagnetic motion sensors or ultrasound motions sensors) and used to provide feedback in a teleoperation, virtual reality, or gaming interaction. Information displayed via shear display can coordinate to the user's interaction within those environments, and/or the user's current motion. Other motion sensors can also be used to provide feedback via shear display based on user interaction in the above environments, such as tilt sensors, inertial sensors, gyros, accelerometers, magnetometers, or other position or motion sensors. Note that inertial sensing and some forms of non-contact position sensing also make it so that no explicit absolute position sensing is necessary, yet this sensor information can provide information that can be used by the control system and fed back to a user via shear feedback. Interaction from force sensors such as a force sensor in the user's chair or floor (e.g., Wii Fit board) can also be used to provide content to be portrayed via skin stretch feedback using a shear display. Again use of multiple force sensors on the tactors of a shear display device can be used to provide translational or rotational inputs to a control system.

[0042] Examples of types of information that can be portrayed along with how it can be displayed, includes: impact can be displayed by rapidly moving the tactor in the direction of the impact force, the velocity of an object or the person can be displayed via shear feedback by moving the tactor with a relative position proportional to the orientation and magnitude of the object's velocity vector. Forces are also similarly mapped to tactor motions by displaying a force vector as a scaled tactor displacement vector. The tilt of an object can also be portrayed via shear feedback by mapping the tilt angle or change in gravitational forces to a proportional amount of skin stretch feedback whose orientation corresponds to the direction an object is tilted (e.g., this could provide feedback as a user tilts their smart phone while playing a game). Another example would be to display skin stretch feedback in a driving game whose tactor motion is proportional to the inertial forces felt by the driver. As another example, if the orientation of the user's hands or arms is tracked then the orientation of the shear feedback could also be corrected to correspond to changes in the user's orientation with respect to the frame of reference of the forces, which may be useful while conducting upper extremity rehabilitation.

[0043] Any other quantities that have a vector and/or magnitude could also be displayed via shear feedback, e.g., life meter in a shooter game, sonar map that shows the location of enemies, which could be displayed by an outbound pulse-type motion in the direction of the target on the heads up map. This application could have performance advantages as the gamer won't need to refer to the visual heads-up display window as much to monitor this information. Likewise, skin stretch feedback could be used to point to an open player in a sports game using a radial tactor motion in the direction towards the open player and then return to center. The tactor motion could move repeatedly in an AC coupled fashion, or could simply move the tactor in the radial direction towards the open player and persistently point in the direction of interest until this information is no longer valid (before the tactor would return to center). Giving AC coupled cues has the advantage in that they are repeated and a user can sometimes (physically or cognitively) miss an initial tactor motion. The tactors location can also be used to provide feedback on the relative location or motion of an object within a game or virtual reality scenario.

[0044] It should be noted that various modes of user interaction are described herein. For example, the use of force feedback, shear display via skin stretch (using tactors), force detection (using tactors and force sensors), and other modes of user interaction are described. These descriptions are not meant to be limiting in any way. Force feedback, shear display via skin stretch, force detection, other user interaction modes, or combinations thereof may be used to interact with the user. Furthermore other modes of user interaction including visual feedback, auditory feedback, haptic feedback, olfactory feedback, even gustational (i.e. taste) feedback, other sensory feedback, or combinations thereof may be used to interact with the user. Haptic feedback, for example, may include vibrotactile feedback. In addition or alternative to these various types of feedback, other modes of receiving user feedback may be used. In addition or alternative to force detection, other user feedback detection may be used. For example, motion capture (including video), sound detection (including but not limited to voice recognition), other user feedback detection or combinations thereof. Furthermore, it should be clear that any of the user feedback modes and the user feedback detection modes may be used individually or in combination with each other.

[0045] In one embodiment, using shear (or skin stretch) feedback in combination with force feedback may reduce the amount of force feedback necessary to lower (or even safe) levels. This may, in the event that there are sensing errors with force feedback alone, reduce instabilities. Thus, using force feedback with skin stretch feedback may enhance/increase the perceived force or stiffness (or other force information such as friction, damping, etc.) reducing the amount of force feedback required. Providing force feedback in combination with skin stretch feedback may be more intuitive because the forces and force directions, though weak in some cases, can still be perceived and may help people more intuitively interpret the skin stretch cues, especially when moving with more spatial motions (i.e., not confined to a plane) or when judging real physical mechanical properties such as stiffness, damping, mass, friction, etc. Furthermore, when using torque feedback and shear feedback to provide rotational (i.e. torque or other rational indicators), less torque feedback may be necessary, due to the complementary rotational shear feedback (from differentially moving tactors) cues. Other advantages are also considered.

[0046] Furthermore, as described herein shear feedback may be used with various procedures. For example, shear feedback may be used to guide hand motions. This guidance may come through directional shear feedback cues, force/torque feedback cues, other cues, or combinations thereof. In the context of catheterization with a shear feedback and/or force feedback device, these cues may be used to help a user stay within a predetermined path.

[0047] For ease of description, the various modes of user interaction that are described herein have been generally presented. These modes of user interactions in addition to any other modes of user interactions may be used for these procedures.

[0048] Referring now to the figures, Figure 1A illustrates one embodiment of a shear display device 100a that has a tactor 110a, which can provide tactile cues and display information to a user. The tactor 110a can be at least partially located within a body 120a. In some instances, the body 120a can be cylindrical; however, other shapes of the body 120a can be suitable, depending on the particular use of the shear display device 100a. Generally, the body 120a can have a suitable configuration for being grasped by the user and/or to provide a reasonable means for the user to lay their hand and/or fingers on the shear display device 100a. The shear display device 100a also can include a well 130a, which can isolate the user's skin with respect to the body 120 and/or near or about the tactor 110a, such that the movement of the tactor 110a can stretch the user's isolated skin. For instance, the well 130a can isolate skin on the user's finger (or fingertip), such that the tactor 110a can create skin stretch thereon without physically moving the user's finger in any substantial manner. Among other things, the well 130a may be configured as an aperture or an orifice. In any event, the well 130a can have a suitable shape, size, and configuration (e.g., position and orientation relative to the tactor 110a) to restrain the user's skin relative to the tactor 110a and may also assist in grasping the shear display device 100a.

[0049] The tactor 110a can move in a two-dimensional space (or in three- dimensional space in some embodiments, though the discussion with respect to Figure 1 A will be mostly focused on two-dimensional space) and, when in contact with the user's skin, can cause skin stretch by such movements. In particular, the tactor 110a can move along X and/or Y axes, indicated in Figure 1A. Moreover, the tactor 1 10a can move simultaneously along both X and Y axes. As such, the tactor 110a can have linear or nonlinear movement in any direction. The skin stretch, experienced by the user can provide cues and information (e.g., directional information) to the user. For example, a linear movement of the tactor 110a can represent a linear movement of the object being controlled by the user. Such movement can be represented in any direction in a two- dimensional space.

[0050] Hence, via movement of the tactor 110a, the shear display device 100a can provide any number of directional cues or types of directional information. In at least one embodiment, the shear display device 100a can provide shear feedback that relates linear movement of an object (e.g., a controlled object, the shear feedback device 100a, or any other predetermined object or entity, whether real or virtual) with the movement of the tactor 110a. For instance, movement of the tactor 110a in a first direction can indicate to the user that an object also has moved in the first direction. Alternatively, movement of the tactor 110a in the first direction can signal to the user that a destination is located linearly in the first direction from the user or from a controlled object. In other words, movement of the tactor 110a also can indicate direction where it may be desirable for the user to move a controlled object. Accordingly, the shear display device 100a can guide the user (or user's hand) and can provide cues for controlling the controlled object (e.g., directing the controlled object to move in a certain desirable direction).

[0051] Likewise, the shear display device 100a can display force information.

For example, movement of the tactor 110a in the first direction can indicate a force applied to the controlled object, which is acting on the first object in the first direction. Hence, the user can quickly determine the direction of the force being applied to the controlled object. Additionally or alternatively, the shear display device 100a can indicate a relative amount of force experienced by the controlled object. For instance, relatively slow movement and/or relatively small displacement of the tactor 110a in the first direction can indicate a relatively small or insignificant force acting on the controlled object. By contrast, a relatively fast movement and/or relatively large displacement of the tactor 110a can indicate a relatively large force acting on the controlled object.

[0052] Similar to linear movements and forces, the shear display device 100a also can display rotational and/or torque information related to an object or entity (e.g., related to the controlled object). Rotational information can relate to in-plane rotation of an object (e.g., a controlled object, the shear feedback device 100a, or any other predetermined object or entity, whether real or virtual). For example, the tactor 110a can move in a substantially or approximately circular or semicircular manner about a predetermined point (e.g., about an original or default position of the tactor 110a or another point). In other words, the tactor 110a can move simultaneously along X and Y axes in a manner that produces a spiral, circular or semicircular movement. In any event, movement of the tactor 110a can appear to the user as a circular or semicircular movement.

[0053] Circular and/or semicircular movement of the tactor 110a can represent rotation of the controlled object and/or torque experienced thereby. More specifically, the rotational movement of the tactor 110a can represent, for instance, the rotation of the object or the magnitude of the torque applied thereto. In one example, the tactor 110a generally moves about and within the diameter of the well. In other embodiments, the diameter of the circular or semi-circular path may be used to communicate the orientation or rotation of the object or the magnitude of the torque applied thereto. In further embodiments, other shapes or paths of the tactor 1 10a may be used to describe the motion (i.e., force or velocity) of the controlled object.

[0054] Additionally or alliteratively, the tactor 110a can rotate about a point. In other words, rather than moving in a circular or semicircular path as described herein, the tactor 110a may rotate about a point. Such rotation (in place) also can represent rotation of an object and/or can convey information about the object's rotation and/or the torque experienced thereby. Similarly, to display the orientation of the object's rotation and/or the torque experienced thereby, the tactor 110a can have a related rotation or motion that can produce corresponding skin stretch in the user's hand, finger, or other body part. In other embodiments, a combination of the circular or semicircular movement as well as rotation of the tactor 110a may be used. For instance, the tactor 110a may rotate about its own axis as it also moves in a circular or semi-circular manner about a point.

[0055] In additional or alternative embodiments, the shear display device 100a also can provide information through a pattern or sequence of movements. For instance, the tactor 110a can have repeated movement or a series of repeated movements in a direction within the plane of the shear display. Such interrupted movement provides "AC coupled" or pulsing directional information (force, direction, displacement, motion, etc.). That is, movement of the tactor 110a can be pulsed with the direction and/or magnitude of the directional information to be provided. This directional information can also be done through sustained movements of the tactor, through "DC coupled" feedback. In this mode, the tactor 110a is moved to a position that represents the direction and/or magnitude of the directional information (force, direction, displacement, motion, etc.) and is held in this position until the tactor is moved back to its center position to indicate that that the user should stop moving, that force now zero, etc.).

[0056] In at least one embodiment, the information displayed by the shear display device 100a via movements of the tactor 110a that produce skin stretch can reflect the movement of the shear display device 100a as well as the movement of the controlled object or any other predetermined object or entity. In some instances, direction and/or speed of movement or acceleration of the tactor 110a can represent movement of the controlled object. For example, if the controlled object moves slowly, the tactor 110a also can move correspondingly slowly (i.e., at the same or proportional rate of speed as the controlled object) and in the same direction as the controlled object, thereby signaling to the user the speed and direction of movement of the controlled object. The tactor 110a could also signal the speed and direction of motion by moving the tactor 110a a first distance and in a first direction (e.g., from a center or default position of the tactor 110a). The first distance and the first direction can be proportional to and may correspond with the object's velocity. [0057] In some embodiments, the shear display device 100a also can be a controller that directs movements of a controlled object (i.e., sends information necessary to move or operate such controlled object). For instance, the shear display device 100a can include a mounting shaft 140a, which can couple to a corresponding control mechanism (e.g., a gimbaled sensor, a force feedback device such as a Phantom Robot Arm, etc.). As further described herein, the control mechanism can detect movements of the shear display device 100a and can send instruction to a controller for directing the controlled object.

[0058] In additional or alternative embodiments, the shear display device 100a also can send control signals or instructions to a controller without being physically connected to the control mechanism. For instance, the shear display device 100a can incorporate wired or wireless tracking mechanisms that can interface with or can be incorporated into the control mechanism and can detect movements and/or position (or change thereof) of the shear device 100a, which can be provided by the control mechanism as instructions for the controlled object. Furthermore, the tactor 110a can also include a force sensor in communication therewith. Consequently, the force sensor can send directional information to the control mechanism or to the controller, which can provide corresponding instruction to the controlled object.

[0059] In addition, as further described herein, the shear display device 100a can display information along a Z-axis. In other words, the shear display device 100a can provide information about three-dimensional movements, forces, torques, positions, etc. For instance, the tactor 1 10a can move outward or inward (i.e., toward or into the user's skin and away from the user's skin) to display information related to movement and/or force along the Z-axis of an object. Hence, movement of the tactor 110a along the Z-axis can provide information about the object, which is similar to the information described herein in connection with two-dimensional movement of the tactor 110a.

[0060] Moreover, movement of the tactor 110a along the Z-axis can be independent of the movements along the X and/or Y axes. Hence, the tactor 110a can move in any number of patterns or directions in three-dimensional space. For example, the tactor 110a can move in any one or more of the X-Y, X-Z, and Y-Z planes. Also, such movement can be along any desired path (e.g., circular path in the X-Z plane), which can indicate movement or position of as well as forces or torques experienced by the object. [0061] The tactor 110a can be actuated in a number of ways. For example, as illustrated in Figure IB, the actuator assembly of the shear display device 100a can include a crank-slider mechanism 150a connected to a first motor 160a. The crank-slider mechanism 150a can include a crank 170a connected to the first motor 160a and a slider 180a coupled to the tactor 110a. As the first motor 160a rotates, the crank 170a moves the slider 180a, thereby producing linear motion of the tactor 110a (e.g., along Y-axis). Accordingly, the crank mechanism 150a can produce linear motion of the tactor 110a in response to the rotation of the first motor 160a.

[0062] The actuator assembly of the shear display device 100a also can include a second motor 190a. The second motor 190a can be located within or secured to the slider 180a. In any event, the second motor 190a can move (e.g., along the Y-axis) with the slider 180a. The second motor 190a also can be coupled to the tactor 110a and can generate rotation of the tactor 110a, for example, with respect to a center axis of the slider 180a. Moreover, operation of the motor 160a and of the second motor 190a can be independent of each other. As such, tactor 110a can move independently in two- dimensions.

[0063] It should be noted, that the range of motion of the tactor 110a can be relatively small (e.g., 0-1 mm, 0-2 mm, 0-5mm). Accordingly, radial motion of the tactor 110a produced by the second motor 190a can appear as substantially linear motion to the user (e.g., as a linear motion along the X-axis). In other words, the first motor 160a may generate linear motion in the y-direction or along the Y-axis and the second motor can generate substantially linear motion in the x-direction or along the X-axis. The movement produced by the second motor 190a and by the first motor 160a (together with the crank mechanism 150a), when combined together, can produce any number of movements or movement patterns of the tactor 110a, such as the movements and movement patterns described herein. Particularly, the tactor 110a can be moved in a linear manner in any direction. Similarly, the tactor 110a also can be moved in a nonlinear manner in any direction. For example, the tactor 110a can be moved in a circular semicircular or other nonlinear manner.

[0064] The movement produced by the second motor 190a and by the first motor

160a (together with the crank mechanism 150a), when combined together, can produce any number of movements or movement patterns of the tactor 110a. Particularly, the tactor 110a can be moved in a linear manner in any direction. Similarly, the tactor 110a also can be moved in a nonlinear manner in any direction. For example, the tactor 110a can be moved in a circular semicircular or other nonlinear manner.

[0065] Although the description herein relates to a shear display device that has an approximately cylindrical form factor, it should be appreciated that this invention is not so limited. Moreover, in light of this disclosure it should be appreciated that the form factor of the shear display device can vary from one embodiment to another. For instance, as illustrated in Figure 2A, at least one embodiment includes a shear display device 100b that has a flat or box-like (e.g., rectangular) body 120b. Except as described herein, the shear display device 100b and its components and elements can be similar to or the same as the shear display device 100a (Figures 1A-1B) and its respective components and elements.

[0066] For example, as illustrated in Figure 2A, the shear display device 100a can include the tactor 110b located within a well 130b, which can at least partially constrain user's skin in contact therewith. The tactor 1 10b can display the same or similar information as can be displayed by the tactor 110a of the shear display device 110a (Figures 1 A- IB). Moreover, the tactor 110b can represent such information by the same or similar movements and movement sequences as described herein in connection with the shear display device 110a (Figures 1A-1B). In certain applications, however, the rectangular body 120b of the shear display device 100b may present a user with a more convenient or ergonomic interface than, for instance, the shear display device that has a cylindrical configuration.

[0067] Moreover, the well 130b and the tactor 110b can be located essentially anywhere on the body 120b. For instance, the well 130b and the tactor 110b can be located near one or more edges of the body 120b (e.g., near a corner of the body 120). It should be appreciated, however, that the well 130b and the tactor 110b also can be located away from one or more edges of the body 120 (e.g., near a center point of the body 120), as may be desirable for a particular application.

[0068] Also, flat or rectangular form factor of the shear display device 100b can allow for additional or alternative actuation mechanisms or actuator assemblies (e.g., as compared with the shear display device 100a (Figures 1A-1B)), which can move the tactor 110b relative to the well 130b. In one example, as illustrated in Figure 2B, an actuator assembly of the shear display device 100b can include one or more actuator assemblies, which can comprise a cam actuation mechanism 150b. More specifically, the cam actuation mechanism 150b can include first and second motors 160b, 190b which can move or rotate respective first and second cams 170b, 180b. As described herein in more detail, rotation of the first and second cams 170b, 180b can result in movement of the tactor 110b relative to the body 120b. As shown in Figures 2B-2D, the first and second cams 170b, 180b are shown as eccentric circular pins that move within a slot to provide relative motion.

[0069] In one embodiment, the first motor 160b can be coupled to the first cam

170b through a series of gears. For example, the first motor 160b can have a worm 162b coupled to a shaft thereof, which engages a worm gear. The worm 162b can be engaged with a worm/spur gear 172b of a first diameter. In some embodiments, the connection between the worm 162b and the worm/spur gear 172b can be a reducer and can provide mechanical advantage (i.e., can produce higher torque at the rotation of the first gear 172b than produced by the first motor 160b). As such, one rotation of the worm gear 162b can produce less than one rotation of the spur gear 172b. The significant mechanical advantage that may be provided by a worm gear can reduce the required space and also may reduce the meshing velocity of spur gears at the next stage of the transmission. The reduced meshing velocity of the spur gears can greatly reduce the noise produced by a geared transmission, and the meshing of worm gears in the first stage of the transmission is inherently quieter than the meshing of spur gears. Use of helical gears rather than standard spur gears can further reduce the transmission noise.

[0070] Additionally or alternatively, the spur gear 172b can be coupled to or operatively connected with the first cam 170b, as described further below. For example, the spur gear 172b can be coupled to a spur gear 174b of a second diameter (e.g., the spur gears 172b, 174b can be coupled together and may rotate together about a shaft). Moreover, the spur gear 174b can be engaged with a spur gear 176b, which can be coupled with the first cam 170b. The spur gears 172b and 176b can have substantially the same diameter, while the spur gear 174b can have a smaller diameter. Accordingly, connection between the spur gears 172b, 174b, 176b also can be a reducer and can provide mechanical advantage.

[0071] Mechanical advantage provided by the connection between the worm 162b and the worm/spur gear 172b and/or by the connection between the spur gears 172b, 174b, 176b can transfer more force to the movement of the tactor. Additionally, such connection can reduce the angle of rotation of the first cam 170b relative to the rotation of the first motor 160b. Consequently, such connection also can provide additional control and may enhance precision or accuracy of positioning and/or moving the tactor. [0072] The second motor 190b can be coupled to the second cam 180b in a similar manner, as the first motor 160b may be coupled to the first cam 170b, as described herein. Additionally, it should be appreciated that the first and second motors 160b, 190b can have any number of suitable connections or coupling configurations with the respective first and second cams 170b, 180b. Such connections can include direct or direct drive connections, crank-slider connections, belt-pulley connection, chain-sprocket connections, etc. In any event, the first and second motors 160b, 190b can rotate respective first and second cams 170b, 180b, which can produce motion of the tactor relative to the body of the shear display device.

[0073] For example, as illustrated in Figure 2C, the cam actuation mechanism

150b (Figure 2B) can be housed in a sliding housing 200b. More specifically, the cam actuation mechanism can be secured to and/or within the sliding housing 200b. In addition, the first cam 170b can be slidably and/or rotatably secured to or within the body of the shear display device (e.g., the first cam 170b can be secured within a slot in the body). Accordingly, in response to clockwise rotation of the first cam 170b, the sliding housing 200b can be pushed in a first direction along an X-axis, while in response to counterclockwise rotation of the first cam 170b, the sliding housing 200b can be pushed in a second, opposite direction along the X-axis.

[0074] In some embodiments, the sliding housing 200b can have grooves 210b, which can guide the sliding housing 200b along the X-axis. Thus, the sliding housing 200b can move along the X-axis in response to rotation of the first motor. Furthermore, the tactor can be secured or coupled to the sliding housing 200b. Consequently, movement of the sliding housing 200b can result in the corresponding movement of the tactor relative to the body of the shear display device.

[0075] It should be appreciated that the first cam 170b (and the second cam 180b

(Figure 2B) can provide mechanical advantage. Moreover, the first cam 170b (and the second cam 180b (Figure 2B) can be configured such as to provide the greatest mechanical advantage at the farthest point of travel of the sliding housing 200b (and of the upper slide 220b (Figure 2B), respectively). Accordingly, as the tactor moves and the user's skin stretches, the tactor can experience resistance due to the stretch of the user's skin, which can increase as the tactor moves away from a default position and can peak at the farthest position of travel. Such increase in resistance can be at least in part accommodated by correspondingly increasing mechanical advantage provided by the first cam 170b (and the second cam 180b (Figure 2B)). [0076] Also, as the first cam 170b (or the second cam 180b (Figure 2B)) rotates

(e.g., within a slot in the body of the shear display device), the sliding housing (and/or the upper slide 220b (Figure 2B)) can move in a first direction and/or in an opposite direction. In some embodiments, the first cam 170b (and/or the second cam 180b (Figure 2B)) as well as corresponding slot or receiving channel in the body of the shear device can be configured such that the first cam 170b (and/or the second cam 180b (Figure 2B)) can fully rotate therein. As such, if a motor or controller fails in a manner that provides continuous rotation to the first cam 170b (and/or the second cam 180b (Figure 2B)), the sliding housing 200b (or the upper slide 220b (Figure 2B), as applicable) can continuously oscillate, without exceeding travel limits of the shear display device and/or damaging components or elements thereof.

[0077] In additional or alternative embodiments, an upper slide 220b can be slidably coupled to the sliding housing 200b. Thus, the second cam can move the upper slide 220b relative to the sliding housing 200b as well as relative to the body of the shear display device. More specifically, as illustrated in Figure 2D, the second cam 180b can be rotatably secured within a slot 230b and the upper slide 220b. Hence, as the second cam 180b rotates about an axis thereof, the second cam 180b can push or pull the upper slide 220b along the Y-axis.

[0078] In some embodiments, the sliding housing 200b can include guiding channels 240b (e.g., the guiding channels 240b can have a gib-like configuration). The upper slide 220b can include guiding protrusions 250b that can fit into the guiding channel 240b. Accordingly, the upper slide 220b can move linearly along the Y-axis relative to the sliding housing 200b as well as relative to the body of the shear display device. Additionally, when the sliding housing 200b moves along the X-axis, relative to the body of the shear display device, the upper slide 220b can move together with the sliding housing 200b.

[0079] In one or more embodiments, the tactor 110b can be secured to the upper slide 220b. Consequently, when the upper slide 220b moves along the Y-axis, in response to rotation of the second motor, the tactor 110b also can move along the Y-axis. Likewise, when the sliding housing 200b moves along the X-axis, in response to rotation of the first motor, the tactor 110b can move along the X-axis.

[0080] Hence, rotation of the first and second motors can actuate movement of the tactor 110b along the X and Y axes. Furthermore, the first and second motors can move the tactor 110b in any number of paths and/or sequences or patterns. For instance, the first and/or second motors can be a servo motors connected to and controlled by a controller, which can provide instructions to the first and second motors to move the tactor 110b in a manner that displays information to the user, as described herein.

[0081] The above description relates to providing one-, two- and three- dimensional information by producing skin shear or skin stretch through movement of one or more tactors in a single plane, which is substantially parallel with the user's skin. This invention, however, is not so limited. For example, as illustrated in Figures 3A-3B, one embodiment of an actuation mechanism or an actuator assembly 260, which can actuate a tactor 110 in a manner that can provide one-, two-, or three-dimensional information to the user through movement of the tactor 110 in a plane substantially orthogonal with respect to the user's skin described herein. It should be appreciated that the actuator assembly 260 can be incorporated into any one of the shear display devices described herein, including the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, 100k, 100η (Figures 1A-2B, 4-11) irrespective of the particular shape of their respective bodies. Moreover, although the illustrated embodiment of the actuator assembly 260 incorporates a cylindrical or a point tactor 110, it should be noted that the actuator assembly 260 can move any one of the tactors described herein, irrespective of the size and/or shape thereof. Although the focus of the foregoing description of embodiments of shear display devices focuses generally on the movement of tactors in the x- and y-directions, any embodiment herein may be combined to provide combinations of circular/semi-circular movement, rotation, planar movement, other tactor movement or combinations thereof with movement or cues provided in the z direction as well as gross movement when used with a device such as a force feedback device.

[0082] Accordingly, such movement of the tactor can apply pressure onto the user's skin. More specifically, the actuator assembly 260 can incorporate a cam 270 and a motor 280 that can rotate the cam 270, thereby causing the tactor 110 to move outward (i.e., toward the user's skin along the Z-axis). For example, a counter clockwise rotation of the cam 270 can cause the tactor 110 to move outward. The tactor 110 can be returned to its original position by rotating the cam 270 in the opposite direction (e.g., counterclockwise), such as to lower the tactor 110.

[0083] In some embodiments, the tactor 110 may be coupled with a sliding shim and a spring (e.g., a conical spring) 290. The spring 290 may keep the tactor 110 generally in contact with the cam 270 when the cam 270 moves from its largest diameter toward its smallest diameter. In other embodiments, the cam 270 and the tactor 110 can be connected (e.g., via a T-slot connection). Accordingly, as the cam 270 rotates (e. g., clockwise) to lower the tactor 110, the cam 270 can pull the tactor 110 downward. Alternatively, the tactor 110 can be spring-loaded, such that rotation of the cam 270, which is uncoupled from the tactor 110, may allow the spring to lower the tactor 110.

[0084] The tactor 110 may be operatively associated with actuator assemblies that can move the tactor 110 in a plane substantially parallel with the user's skin (e.g., in the X- Y plane). Accordingly, the tactor 110 can move in three-dimensional space-i.e., in a plane parallel to the user's skin as well as in a plane perpendicular to the user's skin. For example, the actuator assembly 260 can include first and second motors 160k, 190k, which can move the tactor 110 along respective X and Y axes. Particularly, the tactor 110 can be coupled to the first motor 160k via a first crank-slider mechanism 170k and to the second motor 190k via a second crank slider mechanism 190k. As the first motor 160k rotates the crank of the first crank-slider mechanism 170k, the first slider can move the tactor 110 along the X-axis. Likewise, as the second motor 180k rotates the crank of the second crank-slider mechanism 190k, the second slider can move the tactor 110 along the Y-axis. It should be appreciated that the tactor 110 may be operatively associated with any one of the above-described mechanisms, which can move the tactor 110 along the X and/or Y axes.

[0085] In other embodiments, the shear display device can include a set of wedges. As the wedges are moved toward one another, the first wedge slides onto the second, thereby raising the tactor. Conversely, as the wedges move away from each other, the tactor can be lowered. Similar to the cam 270 (Figures 3A-3B), the first wedge can be coupled to the tactor, thereby pulling the tactor as the first wedge slides down along the second wedge. Alternatively, the tactor can be spring-loaded; hence, the spring can force the tactor downward as the first wedge slides down the second wedge.

[0086] In addition to or in lieu of moving the tactor along the Z-axis, relational information can be displayed to the user by providing changes in pressure on the user's skin. For instance, the change (increase/decrease) in pressure can correspond to a correlating change in upward/downward direction of movement of the controlled object. Additionally or alternatively, the increase/decrease in pressure can correspond with increase/decrease in force experienced by the controlled object from its environment.

[0087] In one embodiment, the shear display device can convey a sensation of increased upward pressure by reducing the area of the tactor that contacts the user's skin. For example, the shape of the tactor can be changed, thereby reducing the area of the tactor that contacts the user's skin. In one instance, the tactor can have a substantially curved or spherical outer surface. By reducing the radius (or contact area) of the curved or spherical surface (i.e., outer surface) that defines the tactor, the user can experience greater pressure applied to the portion of the skin that contacts the tactor. Conversely, by increasing the radius of the sphere defining the tactor, a greater area will contact the user's skin, thereby decreasing the pressure felt by the user.

[0088] To increase and decrease the radius of the curved or spherical surface defining the tactor, in one or more embodiments, the tactor includes a flexible outer shell that is connected to at least two connectors (e.g., two tendons). The tendons may be connected to a shortening device. For example, the shortening device can be a pulley coupled to an actuator. As the pulley reduces the length of the tendons, the flexible outer shell is reduced in radius by the tactor reducing the contact area of the outer surface.

[0089] In other embodiments, the tactor may include an inflatable balloon or membrane, which can be inflated and deflated, thereby changing the area of the tactor and varying the pressure sensed by the user. Alternatively, the tactor can comprise a domed shell actuated by piezo-electrical elements. In any event, the shape of the shell or membrane of the tactor can change in a manner that may provide the user with a sensation of increased or decreased pressure on the user's skin.

[0090] In another embodiment, the shear display device can incorporate a tactor comprising multiple concentric rings. Such concentric rings can move up or down, thereby increasing and decreasing the area that is in contact with user's skin. Accordingly, whether by moving the tactor outward/inward (toward and away from the user's skin) and/or by decreasing/increasing the area of the tactor, the user can sense a change in pressure on the skin that is in contact with the tactor. For example, the tactor may be in a base configuration where the tactor is z-axis neutral. In some embodiments, the concentric rings move outward in the z-direction.

[0091] Moreover, the concentric rings can cooperate to form a contoured surface.

A contoured surface may create a sense of surface curvature or changing contact area, which may be used to portray differing forces, direction cues, pressure, etc. The concentric rings also can cooperate to provide the same outward movement in the z- direction, but only the centermost concentric ring remains in the outward position (the other rings move away from the finger pad). This provides a smaller surface area of the tactor which can be used to create a sense of surface curvature or changing contact area which may also be used to create a sense of surface curvature or changing contact area, which may be used to portray differing forces, direction cues, pressure, etc.

[0092] In other embodiments, the shear display device can include multiple tactors, which can be located in the same or in one or more different planes. For example, as illustrated in Figure 4, a shear display device 100c can incorporate the first tactor 110c' as well as a second tactor 110c". Except as otherwise described herein, the shear display device 100c and its components and elements can be similar to or the same as any one of the shear display devices 100a, 100b (Figures 1A-2D) and their respective components and elements (e.g., actuator assemblies). More specifically, the second tactor 110c"can be positioned substantially orthogonally relative to the first tactor 110c'. Similarly, the control system can incorporate the shear display devices that have multiple tactors. As described herein in further detail, a control system can include the force feedback device and the shear display device 100c connected to the force feedback device.

[0093] Additionally or alternatively, in some embodiments, as illustrated in Figure 5, a shear display device lOOd can include multiple opposing tactors, such as a first tactor l lOd' and a second tactor l lOd". Except as otherwise described herein, the shear display device lOOd and its components and elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c (Figures 1A-2A and4) and their respective components and elements (e.g., actuator assemblies). More specifically, the first and second tactors l lOd', l lOd" can be directly opposite each other. As shown in Figure 5, the tactors l lOd', l lOd" are opposite to each other and centered approximately about the same axis (e.g., about Z axis). Such a configuration of the shear display device lOOd can allow the shear display device lOOd to display various movement information to the user, which can be presented in a more intuitive manner.

[0094] For instance, the user can perceive relative movement of the first and second tactors l lOd', l lOd", which can provide various information to the user. Particularly, moving the first tactor l lOd' and the second tactor l lOd" in opposite directions within their respective X-Y planes (i.e., in two-dimensional motion) can inform the user about rotational motion of the object (e.g., controlled object, shear display device 1 lOd, etc.). For example, movement of the first tactor 110d' and the second tactor 1 lOd" in opposite directions along the Y axis can display rotational motion about the X axis. It should be appreciated that, because the first and second tactors l lOd', l lOd" are spaced apart along the X-axis, the user can experience a torque-like sensation, produced by the relative movement of the first and second tactors 110d', 1 lOd" in opposite directions. [0095] In other embodiments, the first and second tactors 110d', 1 lOd" may move in opposite directions in three-dimensional motion. For example, the first tactor l lOd' may move in a first direction along the Y axis and the second tactor 1 lOd" may move in a second, opposite direction along the Y-axis, while the first tactor l lOd' and the second tactor l lOd" both move inward (i.e., while the first tactor l lOd' moves along the Z-axis, away from the user's skin and the second tactor 1 lOd" moves along the Z-axis away from the user's skin) in an arc-like motion.

[0096] Additionally or alternatively, the first and second tactors l lOd', l lOd" of the shear display device lOOd can move in opposite directions to display rotation about another axis (e.g., about an axis that is concentric with the shear display device lOOd). Hence, for example, when the user rotates the cylindrical shear display device lOOd about the Y axis, the first tactor l lOd' and the second tactor l lOd" can move in opposite directions the X axis, thereby displaying rotational motion of the shear display device l lOd and/or of the object, such as the control object. Furthermore, other rotational motions can be display to the user by moving the first and second tactors 110d', 1 lOd" in a similar manner (i.e., by relative motion of the first and second tactors 1 lOd, 1 lOd").

[0097] Accordingly, the shear display device lOOd can provide the user with information about rotation of an object about X and/or Y axes. For example, counterclockwise rotation of the shear display device l lOd about the Y-axis can be displayed to the user by moving the first tactor 100d' in a first direction along the X-axis, while moving the second tactor l lOd" in a second, opposite direction along the X-axis. Such movement of the first and second tactors l lOd', l lOd" can create a sensation of torque and can convey relevant rotational information to the user.

[0098] It should be appreciated that in addition to or in lieu of movements described herein, which can display rotational information to the user, the first and second tactors l lOd', l lOd" of the shear display device lOOd can move in the same or similar manner as the tactors of the shear display devices 100a, 100b, 100c (Figures 1A-3). Accordingly, the shear display device lOOd also can convey the same information as a single tactor shear display device. For instance, to display rotation about the Z axis, the first and/or second tactors l lOd', l lOd" can move in a spiral or circular path and/or can rotate about various axes, as described herein. More specifically, for instance, to indicate a clockwise motion of the shear display device 1 lOd and/or of the controlled object, first tactor l lOd' can be moved in a clockwise spiral or circular motion about the center or another point in the well. Similarly, the opposing, second tactor l lOd" can move in an opposite direction, namely counterclockwise relative to a view looking at that tactor (to indicate the same motion) about the same or another point. Furthermore, circular and/or rotational movements of the first and second tactors l lOd', l lOd" can be synchronized, such as to display the same rotation on both sides of the shear display device l lOd. In other words, the first tactor l lOd' and the second tactor l lOd" may both rotate and/or move clockwise, counterclockwise, or otherwise simultaneously.

[0099] Likewise, as described herein in connection with the shear display device that has a single tactor, the shear display device l lOd also can display translational or linear motion of the shear display device l lOd and/or of the controlled object or user's hand. For instance, both the first tactor l lOd' and the second tactor l lOd" can move in the same direction and at the same speed or acceleration to indicate translational motion. In at least one example, both the first tactor 110d' and the second tactor 1 lOd" can move toward the user (i.e., upward on the page) indicating corresponding linear movement of the controlled object. Alternatively, both the first tactor l lOd' and the second tactor l lOd" can move to toward the user (i.e., toward the left of the page) indicating corresponding linear movement of the controlled object. Similarly, both the first tactor l lOd' and the second tactor l lOd" can move to the user's left, thereby displaying corresponding movement of the controlled object. Also, both the first tactor 110d' and the second tactor l lOd" can move to the user's right, thereby displaying corresponding movement of the controlled object.

[00100] Similarly, as described below in further detail, the first and second tactors

110d', 1 lOd" can move out of plane (i.e., in a first or second, opposite direction along the Z-axis). Moreover, any one of the actuation mechanisms or actuator assemblies described herein can be incorporated into or used in the shear display device lOOd. In some embodiments, the first and second tactors l lOd', l lOd" may be coupled (hard- or soft- coupled, i.e., physically or via a controller) such that the first and second tactors l lOd', l lOd" move together. For example, the first and second tactors l lOd', 110d"may both move in the same direction (i.e., to the right, but with the first tactor l lOd' moving outward in the z direction and the second tactor 1 lOd" moving inward in the z-direction).

[00101] In addition to or in lieu of the actuator assembly 260 (Figures 3A-3B), as mentioned above, the shear display device lOOd can include any number of suitable actuator assemblies and/or actuator mechanisms. In one embodiment, the first and second tactors l lOd', l lOd" can move in opposite directions along the Z-axis in a synchronized and/or in an independent manner. Such movement, for example, can convey force or pressure experienced by the controlled object. As noted above, location and movement of the first and second tactors 110d', 1 lOd" can indicate to the user the magnitude of force or pressure experienced by and/or the location or movement or size of an object.

[00102] In an embodiment, the actuator assembly of the shear display device lOOd can include a sliding wedge (or multiple sliding wedges). For instance, the shear display device lOOd may include a wedge actuator that moves the wedges with respect to each other. In one example, the wedge actuator may move a left wedge away from the wedge actuator. As such, a right wedge may stay in its relative longitudinal location. In other embodiments, the right wedge also may be actuated by the wedge actuator or another actuator in the opposite direction than the left wedge. Furthermore, as the larger portion of the left wedge approaches the larger portion of the right wedge, both first and second tactors l lOd', HOd" can move outward (e.g., the first tactor l lOd' moves in a first direction and the second tactor 1 lOd" moves in a second, opposite direction).

[00103] Additionally or alternatively, shims or other mechanisms may be used to interface with the sliding wedges. The first and second tactors l lOd', l lOd" may be operatively associated with actuator mechanisms and/or actuator assemblies that can move the first and second tactors l lOd', l lOd" in a plane substantially parallel with the user's skin (e.g., actuation mechanism 150b (Figures 2B-2D)). Accordingly, the first and second tactors 110d', 1 lOd" can move in three-dimensional space— i.e., in a plane parallel to the user's skin as well as in a plane perpendicular to the user's skin.

[00104] In a further embodiment, the first and second tactors 110d', 1 lOd" may be coupled together, but may be able to move both in the same direction and in opposite directions along the Z-Axis. Any number of mechanisms may be used that can provide coupled motion in both the opposite and the same directions. For instance, the shear display device lOOd may use a tapered eccentric cam that is both eccentric about and tapered along its longitudinal axis. The shear display device lOOd may include an actuation mechanism (not shown) that may both rotate and longitudinally move the tapered eccentric cam.

[00105] For example, as the tapered eccentric cam is moved toward the first and second tactors l lOd', l lOd", both the first and second tactors l lOd', l lOd" can move outward along the Z-axis. Likewise, as the tapered eccentric cam is moved away from the first and second tactors l lOd', l lOd", both the first and second tactors l lOd', l lOd" can move inward along the Z-axis. Also, as the tapered eccentric cam is rotated, the first and second tactors 110d', 1 lOd" can both move in the same direction. In other words, the first tactor 110d' can move outward and the second tactor 1 lOd" can move inward.

[00106] Where the tapered eccentric cam both longitudinally moves and rotates, the first and second tactors 110d', 1 lOd" may move in the same direction (i.e., outward or inward) but at a different rate and/or a different amount. For example, when the tapered eccentric cam is rotated to a neutral position (e.g., where each of the first and second tactors l lOd', l lOd" may be the same distance from the center of the tapered eccentric cam) the tactor to which the larger diameter portion of the tapered eccentric cam is approaching will move farther outward as the tapered eccentric cam is longitudinally advanced and rotated such that the larger diameter portion of the tapered eccentric cam approaches that tactor. Likewise, from an advanced position position (i.e., where one of the first and second tactors l lOd', l lOd" abutd the larger diameter portion of the tapered eccentric cam and where the largest portion of the taper is abutting both first and second tactors 110d', 1 lOd"), as the tapered eccentric cam moves toward the neutral position and longitudinally away from the first and second tactors l lOd', l lOd", the tactor that was closest the largest portion of the tapered eccentric cam may move faster and further inward as the tapered eccentric cam is longitudinally advanced and rotated such that the larger diameter portion of the tapered eccentric cam moves away from that tactor.

[00107] In addition, shims or other mechanisms may be used to interface with the tapered eccentric cam. As noted above, the first and second tactors 110d', 1 lOd" may be operatively associated with actuator assemblies that can move the tactors in a plane substantially parallel with the user's skin. Accordingly, the tactors can move in three- dimensional space— i.e., in a plane parallel to the user's skin as well as in a plane perpendicular to the user's skin.

[00108] Consequently, the shear display device lOOd also can display information about force experienced by an object about the Z-axis. Specifically, in one embodiment, the first and second tactors 110d', 1 lOd" can move outward, thereby displaying to the user increased pressure along the Z-axis. Conversely, the first and second tactors 110d', 1 lOd" can move inward, which may signal to the user a decrease in force experienced by an object. Moreover, outward movement of the first and second tactors l lOd', l lOd" may signal to the user growing, zooming, or size increase of an object. Similarly, inward movement of the first and second tactors l lOd', l lOd" can signal to the user shrinking, zooming out, or size decrease of an object. [00109] As noted herein, the shape of the body of the shear display device can vary from one embodiment to another. For instance, as illustrated in Figure 6, a flat or rectangular shear display device lOOe also can have opposing first and second tactors 110e', 1 lOe". Except as otherwise described herein, the shear display device lOOe and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd (Figures 1A-5) and their respective components and elements. Furthermore, the first and second tactors l lOe', HOe" can exhibit relative motion (e.g., similar to the tactor movements described in connection with the shear display device lOOd (Figure 5)) to display rotation of an object about X and/or Y axes.

[00110] Moreover, the shear display device also can provide the same or similar information as any one of the shear display devices 100a, 100b, 100c, lOOd (Figures 1A- 4). For example, the first and/or second tactors 1 10e', 1 lOe" can move in a linear manner to indicate linear directional or force cues. Additionally or alternatively, the first and/or second tactors 110e', 1 lOe can move along spiral, circular or semicircular paths to provide rotational information about the Z axis. In any event, the shear display device lOOe can provide the same information as can be provided by a single tactor shear display device as well as additional information that can be presented by differential relative movement of the first and second tactors 110e', 1 lOe.

[00111] In one embodiment, the shear display device lOOe can be coupled to or integrated with a tool or an instrument. For instance, the shear display device lOOe can be coupled to a stylus, which can be used to send instructions to a controller. Furthermore, as described herein in more detail, the shear display device lOOe can be coupled to a controller and/or force feedback device. Hence, for example, the shear display device lOOe, together with the controller and/or force feedback device, can be used in robotic procedures (e.g., surgery, catheter insertion, etc.).

[00112] As mentioned herein, the body of the shear display device can have any number of suitable configurations, shapes, and sizes. Furthermore, among other things, locations of the tactors and wells on the body of the shear display device can vary from one embodiment to another. In addition, the shape and size of the tactors can vary from one embodiment to the next and may depend on location of the tactor on the shear display device, portion of the skin intended to be contacted by the tactors, etc. For instance, as illustrated in Figure 7A, a shear display device lOOf can have a first and second tactors 11 Of, 11 Of located on a joystick- like body 120f. Except as otherwise described herein, the shear display device lOOf and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe (Figures 1A- 6) and their respective components and elements.

[00113] More specifically, the shear display device lOOf can have the first tactor

I lOf located on an upper end thereof, such that the first tactor 1 lOf can engage the user's thumb. The first tactor 11 Of can be surrounded by a first well 13 Of, which can restrain the skin on the user's thumb, in a manner that isolates skin movement and produces skin stretch in response to the movement of the first tactor 11 Of. Moreover, the first tactor

I I Of can be offset from a center axis of the body 120f. For example, a right-handed shear display device lOOf can have the first tactor 1 lOf offset to the left of the center axis of the body 120f.

[00114] As such, the location of the first tactor 1 lOf can allow the user's thumb to remain in a more natural position. In other words, in some embodiments, the user's thumb can remain in a relaxed or un-flexed state when in contact with the first tactor 1 lOf . Such a configuration can provide better isolation of the user's skin relative to the first tactor 1 lOf , which can lead to more accurate and/or sensitive feedback for the user. Moreover, such configuration also can provide improved angular accuracy of user's interpretation of the skin stretch cues from the first tactor 11 Of (as compared with the configuration where the tactor is aligned with the center axis of the body).

[00115] Likewise, for a left-handed shear display device lOOf, the first tactor 1 lOf can be offset to the right of the center axis of the body 120f, such that the user's left thumb can remain in a relaxed state while in contact with the first tactor 1 lOf . It should be appreciated that other embodiments can include the first tactor 11 Of located at other suitable locations on the upper end of the shear display device lOOf (e.g., aligned with the center axis of the body 120f). In any event, the first tactor 11 Of can be located on the body 120f such that the user is capable of placing a left or a right thumb on the first tactor 1 lOf (and in the well 130f) while holding the body 120f of the shear display device lOOf with the corresponding hand.

[00116] The first tactor 11 Of can be capable of movements similar to or the same as the movements of any one of the tactors described herein. Hence, the first tactor 1 lOf can display the same or similar information as any one of the tactors mentioned herein. In addition to the first tactor 11 Of, as noted herein, the shear display device lOOf can include the second tactor 11 Of". In one embodiment, the second tactor 11 Of can move substantially along the length of the body 120f. Particularly, the tactor 11 Of can be in contact with the user's palm and can provide any number of tactile cues, represented by a substantially vertical movement of the second tactor 1 lO '.

[00117] In one or more embodiments, the second tactor 1 lO ' can have a plate- or bar-like shape. As such, the second tactor 1 lO ' can engage a larger portion of the user's skin. Such a configuration can be particularly advantageous on portions of the user's skin that have a relatively low density of mechanoreceptors. For example, skin on the user's palm may have a lower density of mechanoreceptors or nerve endings than the skin on the user's fingertips. Accordingly, in one embodiment, the second tactor 1 lO ' can recruit or engage substantially the same or similar number of mechanoreceptors on the user's palm as recruited or engaged by the first tactor 1 lOf on the user's fingertip. Such a balanced engagement can allow the shear display device lOOf to provide tactile cues to the user's fingertip(s), fmgerpads, and palm, which appear with similar or substantially the same intensity to the user.

[00118] Hence, the second tactor 11 Of can be sufficiently larger than the first tactor 11 Of, to allow the second tactor 11 Of to contact sufficiently larger portion of the user's palm. For instance, the second tactor 11 Of can be approximately a .75" x 1.5" rectangle (i.e., can have an approximate area of 1.125 in ), while the first tactor 11 Of can have an approximately 0.25" diameter (i.e., can have an approximate area of 0.05 in ). In other examples, the second tactor 11 Of may have a substantially rectangular shape with sides greater than 1.5" and/or smaller than .75". It should be also appreciated that the second tactor 11 Of can have any number of other shapes, which may vary from one embodiment to the next (e.g., square, triangular, trapezoid, irregular, etc.). The greater area of the second tactor 11 Of, as compared with the area of the first tactor 11 Of, can allow the second tactor 11 Of to recruit sufficient number of mechanoreceptors to compensate for lower density of the mechanoreceptors on the portion of the skin in contact with the second tactor 1 lOf .

[00119] In additional or alternative embodiments the second tactor 11 Of can include ridges or texture, which can create sufficient friction between the user's palm and the second tactor 11 Of while also increasing the experienced sensations. Alternatively, the second tactor 11 Of can comprise a material that has sufficient coefficient of friction to prevent or reduce slippage between the tactor 1 lOf and the user's palm. For instance, the second tactor 11 Of can comprise rubber, neoprene, silicone, and the like. In any event, the second tactor 11 Of can stretch the user's skin, thereby activating mechanoreceptors thereon. [00120] Furthermore, as the user grasps the body 120f, portions of the body 120f surrounding or adjacent to the second tactor 1 lO ' can at least partially restrain the user's skin surrounding the skin that is in contact with the second tactor 1 lO '. In at least one implementation, the portions of the body 120f that surround the second tactor 11 Of can comprise material that exhibits a relatively high coefficient of friction (e.g., roughened surface) or may be tacky, such as to prevent or limit movement of the skin adjacent to the portion of the user's skin that is in contact with the second tactor 11 Of . In any case, when the user's hand grasps the body 120f, the skin in contact with the second tactor 11 Of can be at least partially isolated in a manner that allow the second tactor 11 Of to stretch the skin.

[00121] Accordingly, in some embodiments, the shear display device lOOf can produce skin stretch without incorporating a well into the body 120f thereof. In other words, the act of grasping the body 120f can sufficiently isolate the user's skin to allow the tactor 11 Of to stretch an isolated portion of the user's skin. Alternatively, however, the body 120f can incorporate a well about the tactor 1 lOf , and such well also can isolate the user's skin.

[00122] The first and/or second tactors 1 lO , 1 lOf can be actuated in any number of ways, which can include any one or more of the actuation mechanisms and actuator assemblies described herein. Additionally, for example, the second tactor 11 Of can be coupled to a motor 160f, as illustrated in Figure 6B. Particularly, in one embodiment, a resilient flexible spine 170f can be coupled to a crank 180f secured to the shaft of the motor 160f. As the shaft of the motor 160f rotates the crank 180f in a first direction, the crank 180f can push the spine 170f upward, thereby moving the second tactor 11 Of upward. Conversely, as the shaft of the motor 160f rotates the crank 180f in a second, opposite direction, the crank 180f can pull the spine 170f downward, thereby moving the second tactor 1 lOf downward.

[00123] In some embodiments, the body 120f can have a channel 190f, which can restrain and/or guide the spine 170f therein. Furthermore, in at least one embodiment, the body 120f can have a curvilinear configuration, which may provide an ergonomic fit with the user's hand. Thus, the channel 190f also can have a curvilinear configuration, as the path of the channel 190f may generally follow the outside geometry of the body 120f. In any event, however, the spine 170f can have sufficient flexibility to move up and down within the channel 190f, thereby moving the second tactor 11 Of in corresponding directions. It should be appreciated that, in at least one embodiment, the tactor 1 lOf can move in any number of directions and patterns, linear and nonlinear, as described herein in connection with other tactors.

[00124] It should be appreciated that the shear display device lOOf can allow the user to "power grip" the body 120f, while the second tactor 11 Of may continue moving and transmitting tactile information to the user (i.e., stretching user's skin). In other words, the user's hand can apply a relatively large amount of compressive force onto the body as well as onto the second tactor 11 Of without impeding or interfering with the operation of the second tactor 11 Of and of the shear display device lOOf. Accordingly, the shear display device lOOf can continue providing tactile cues to the user when the user applies relatively large force onto the body 120f.

[00125] As noted herein, the shear display device lOOf can be incorporated into a control system, as described herein in further detail. For example, the shear display device lOOf can include a connecting portion 200f, which can couple the shear display device lOOf to a controller and/or force feedback device. Accordingly, the user can receive tactile cues from the first and second tactors 11 Of, 11 Of and also can receive force feedback.

[00126] In additional or alternative embodiments, the shear display device can include multiple large tactors. For example, as noted herein, large tactors can provide enhanced sensation to the user when in contact with the user's skin that has a relatively low density of mechanoreceptors. For example, as illustrated in Figure 8, a shear display device lOOg can incorporate two plate-like tactors HOg', HOg". Except as otherwise described herein, the shear display device lOOf and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf (Figures 1A-7B) and their respective components and elements.

[00127] As noted herein, a body 120g of the shear display device lOOg can have any number of suitable configurations. In one example, the body 120g can be substantially rectangular or bar-like. Other embodiments can include a cylindrical, spherical, or any other number of configurations for the body 120g. Additionally, the tactors HOg', HOg" can be located essentially anywhere on the body 120g, as may be more suitable or desirable for a particular application. In one embodiment, the tactors 110g', 1 lOg" are located near one edge of the body 120g.

[00128] The tactors 110g', 1 lOg" can be similar to or the same as the second tactor

1 lOf (Figures 7A-7B). However, the tactors 110g', 110g"can exhibit that same or similar movements as any one of the tactors described herein and may be actuated in the same or similar manner as any one of such tactors (i.e., the shear display device lOOg can incorporate any one of the actuator assemblies described herein). Accordingly, the tactors l lOg', HOg" can move up and down along one edge of the body 120g (along Y axis). Furthermore, the tactors HOg', HOg" can move toward and away from the edge of the body 120g (along X Axis). Hence, the tactors HOg', HOg" can exhibit any number of movement path, patterns, and sequences described herein.

[00129] Furthermore, the tactors HOg', HOg" can display the same or similar information as described above in connection with multiple opposing tactors. To mention a few: relative movement of the tactors HOg', HOg" can display rotational information such as rotary motions and torques; likewise, the tactors HOg', HOg" can display linear motion and/or force information.

[00130] Moreover, as noted above in connection with the shear display device lOOf

(Figures 7A-7B), the shear display device lOOg can allow the user to apply sufficiently large grasping force onto the body 120g. At the same time, the tactors HOg', HOg" can continue moving relative to the body 120g and displaying tactile information to the user. Consequently, such configuration can be employed in applications where the body 120g of the shear display device lOOg may be subjected to relatively large grasping forces from the user. For example, the shear display device lOOg can be incorporated into controller used in virtual games of tennis, baseball, golf, sword fighting, etc. It should be appreciated, however, that the shear display device lOOg can be incorporate into any number of controller or devices, which require or allow the user to apply relatively large grasping force onto the body 120g of the shear display device lOOg. In addition, as noted above, the tactors HOg', HOg" can provide sufficient stimulation of user's mechanoreceptors at locations of low or lower sensitivity (e.g., lower than fingertips). Accordingly, the shear display device lOOf can also be used in applications where user's gripping force may not be relatively large and may provide skin stretch cues to portion of the skin with lower sensitivity (e.g., user's palm).

[00131] As previously noted, the body of the shear display device can have any number of configurations. Moreover, a shear display device may incorporate any number of tactors (one, two, three, four, etc.), which may be positioned and/or oriented on the body of the shear display device in any number of suitable configurations. In one example, as illustrated in Figure 9, a shear display device lOOh can have a joystick- like body 120f and may include first, second, and third tactors 110h', HOh", HOj. Except as otherwise described herein, the shear display device lOOh and its components or elements can be similar to or the same as any of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg (Figures 1A-8). For instance, the joystick-like body 120h may be similar to or the same as the joystick-like body 120f (Figures 7A-7B).

[00132] In one embodiment, the first and second tactors 11 Oh', 11 Oh" may move upward and downward along a length of the body 120h. Such movement may approximately follow the lengthwise curvature or surface of the body 120h. In additional or alternative embodiments, the first and second tactors 11 Oh', 11 Oh" may move approximately perpendicular to the length of the body 120h. In still further embodiments, the first and second tactors 110h', 1 lOh" may move about the body 120h in a manner that the path of the first and second tactors 11 Oh', 11 Oh" approximately follows the curvature of the perimeter or surface of the body 120f.

[00133] It should be appreciated that the third tactor 1 lOj can move in a similar or the same manner and can provide similar or the same tactile cues as the first tactor 1 lOf (Figures 7A-7B). Similarly, the first and second tactors 11 Oh', 11 Oh" can move in a similar or the same manner and can provide similar or the same tactile cues as the first and second tactors HOg', HOg". Accordingly, the shear display device lOOh can incorporate cues provided by the shear display device lOOf, lOOg (Figures 7A-8) in a single device. For example, the shear display device lOOh can display torque or rotation experienced by an object by moving the first and second tactors 110h', 11 Oh" in opposite directions (e.g., along the length of the body 120, about the perimeter of the body 120, etc.). Alternatively, linear movement or force experienced by an object can be displayed by the shear display device lOOh as movement of the first and second tactors 110h', 1 lOh" in the same direction.

[00134] The shear display device lOOh can be used in any number of applications. For instance, the shear display device lOOh may be used as an element or component of a control stick of an airplane (e.g., the body 120h of the shear display device can operate as the control stick). In one embodiment, the first and second tactors 11 Oh', 11 Oh" may provide instruction to a pilot (e.g., to a student pilot) regarding how to move the control stick and/or navigate the airplane. In one example, to signal to the pilot to pull back on the control stick, the first tactor 11 Oh' can move downward along the body 120h, while the second tactor 1 lOh" moves upward. Conversely, to signal to the pilot to push forward on the control stick, the first tactor 11 Oh' can move upward along the body 120h, while the second tactor 1 lOh" moves downward. [00135] In yet another embodiment, the shear display device can include four tactors. For example, as illustrated in Figures 10A-10B, a shear display device 100k can include a partially cylindrical body 120k and four tactors positioned about the body 120k. Except as otherwise described herein, the shear display device 100k and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh (Figures 1A-9). Particularly, the shear display device can include first and second tactors 110k', 110k" located along the X-axis of the shear display device 100k and opposite to one another. In addition, the shear display device can include third and fourth tactors 110m', 110m" positioned along the Y-axis of the shear display device and opposite to each other. Moreover, in some embodiments, the first and second tactors 110k', 110k" may have an approximately orthogonal orientation relative to the third and fourth tactors 110m', 110m".

[00136] In an embodiment, the first and second tactors 110k', 110k" can move along the length of the body 120k (i.e., in a direction along or parallel the Z-axis of the shear display device 100k). Likewise, the third and fourth tactors 110m', 110m" can move along the length of the body 120k. In at least one embodiment, such movement of the first, second, third, and fourth tactors 110k', 110k", 110m', 110m" can approximately follow the contour or the surface of the body 120k.

[00137] Furthermore, embodiments of the present invention can include the first and second tactors 110k', 110k" that may move about the body 120k. For instance, the first and second tactors 110k', 110k" can rotate about the body 120k (e.g., about the Z- axis of the shear display device 100k). Such rotation of the first and second tactors 110k', 110k" can approximately follow the contour or the surface of the body 120k. Moreover, such rotation may be synchronized in a manner that the first and second tactors 110k', 110k" rotate together, as a single unit. Similarly, the third and fourth tactors 110m', 110m" can rotate about the body 120k in the same or similar manner as the first and second tactors 110k', 110k". In addition, the first and second tactors 110k', 110k" and the third and fourth tactors 110m', 110m" can rotate about the body 120k together.

[00138] In one embodiment, the body 120k may be sized and configured to allow the user to grasp the body 120k together with the first, second, third, and fourth tactors 110k', 110k", 110m', 1 10m". In other words, the user can grasp the body 120k in a manner that the user's hand contacts the first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m", which may transmit tactile information to the user. Also, the body 120k may isolate portions of the user's skin that are in contact with the respective first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m", such that the first, second, third, and fourth tactors 110k', 110k", 110m', 110m" may produce skin stretch on the skin portions in contact therewith.

[00139] Accordingly, various linear and/or nonlinear movements of the first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m" may provide tactile cues or information to the user. In one example, the first and second tactors 110k', 110k" can move in opposite directions along or parallel to the Z-axis to signal rotation of an object about the Y-axis. Alternatively, such movement can signal to the user to rotate the shear display device lOOh (e.g., in order to move an object), as described below in further detail. Linear movements of the first and second tactors 110k', 110k" can represent linear movements and/or forces experienced by an object or cues to move an object, as described herein.

[00140] Similar to the first and second tactors 110k', 110k", the third and fourth tactors 110m', 110m" can move in opposite directions along the Z-axis to signal to the user rotation or torque about the X-axis. Likewise, such movement also can signal to the user to rotate the shear display device about the Y-axis. In additional or alternative embodiments, as noted above, the first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m" may rotate about the body 120k (e.g., about the Z-axis). Among other things, such rotation can signal to the user rotation and/or torque about the Z-axis experienced by an object. Also, such rotation may signal to the user to rotate the shear display device 100k about the Z-axis. It should be appreciated that one can create torque sensations about axes that lie between the X- and Y-axes by moving multiple tactors together. For example, the first and second tactors 110k' and 110k" can be moved upwards, while the third and fourth tactors 1 10m' 110m" are moved downwards to provide a sensation of torque for the user, which would rotate the top of the body 120k about an axis that is approximately 45 degrees between the X- and Y-Axes.

[00141] An exemplary application of the shear display device 100k may include controlling an object and receiving cues regarding desired or necessary movements for such object and/or for the shear display device 100k. For example, the shear display device 100k may incorporate or may be integrated with a wireless remote or controller, such a Wii remote. In other words, moving the shear display device 100k may send instructions (e.g., movement instructions) to a controlled object. Accordingly, in one embodiment, the user may receive cues or suggestions about where to move the shear display device 100k (and, thus, the controlled object), as described above. [00142] Additionally or alternatively, the shear display device 100k can provide correctional or training cues regarding an optimal or improved movement in a particular application. For instance, the shear display device 100k can provide correctional or training cues for a tennis swing (e.g., the shear display device 100k can represent or can be incorporated into a handle of a tennis racket and may provide tactile cue regarding where and/or how to move the tennis racket). Similarly, the shear display device 100k can provide any number of corrective or training cues, which may improve user's movements in connection with any number of activities (e.g., golf, baseball, fishing, etc.).

[00143] The rotational and translational degrees of freedom communicated by the shear display device 100k could also include assisting in orienting or pointing an object such as a satellite dish or camera. That is, the shear display device 100k can supply the degrees of freedom for pointing the camera with rotations about the X- and Y-Axes, it could suggest how to zoom the lens with translational cues along the Z-axis, and could suggest changes in focus with cues to rotate the camera lens, with rotational cues about the Z-Axis. Such cues could be especially advantageous for individuals with vision impairments.

[00144] As mentioned above, the shear display device can have any number of tactors located on any portion thereof. Figure 11 illustrates another embodiment of a shear display device, which incorporates multiple tactors on the same side thereof. Particularly, Figure 11 illustrates a shear display device 100η that incorporates first and second tactors 11 On', 11 On" on the top of a body 120n. Except as otherwise described herein, the shear display device 100η and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, 100k (Figures lA-10) and their respective components and elements.

[00145] To display in-plane rotation of the shear display device 100η, the first and second tactors 11 On', 11 On" can move in opposing directions. More specifically, the first and second tactors 1 ΙΟη', 110η" can move in opposing directions along their respective Y' and Y" axes. This movement can generate torque about a center point between the first and second tactors 11 On', 11 On". Such torque can indicate to the user that the shear display device 100η and/or the controlled object is rotating or that they should rotate their hand about a Z-axis (not shown), which can be perpendicular to the X'-Y' plane.

[00146] As noted above, the displacement or movement of the first and second tactors 11 On', 11 On" also can indicate the location, distance, displacement, or motion of the controlled object. Moreover, moving the first and second tactors 11 On', 11 On" with opposing motions (e.g. on their respective Y' and Y" axes) also can indicate torque being applied to the controlled object or that the user should rotate their hand about that respective axis. Hence, location or movement of the first and second tactors 1 lOn', 110η" can indicate the amount of torque being applied to the controlled object. In contrast to moving of the first and second tactors 11 On', 11 On" in opposing directions, which can indicate rotation or torque, movement of the first and second tactors 11 On', 11 On" in the same direction (e.g., along Y' and Y" axes, X' and X" axes, or any parallel directions), can indicate translational/linear movement of the shear display device 100η and/or of an object, such as the controlled object.

[00147] Additionally or alternatively, the first and second tactors 11 On', 11 On" can move in opposite directions on an axis that lies along the direction between the center point of each of the tactors to display various tactile cues to the user. For example, the first and second tactors 11 On', 11 On" can move radially away from each other to display zooming out or an increase in distance (virtual or otherwise) between two objects (e.g., between an object and the controlled object; between the user and an object, etc.) or increase in the size of an object. Such movement also can display to the user tensile forces experienced by an object or provide similar force/pressure cues.

[00148] Conversely, the first and second tactors 11 On', 11 On" can move radially toward each other, thereby signaling to the user zooming in or a reduction in distance (virtual or otherwise) between two objects or the decreasing size of an object. Likewise, such movement of the first and second tactors 11 On', 11 On" can indicate compressive forces experienced by an object or provide similar force/pressure cues. For instance, as a tool passes through a narrowing and is compressed, the first and second tactors 11 On', 110η" can move toward each other, thereby indicating compression of the tool.

[00149] In one embodiment, the first and second tactors 11 On', 11 On" are offset from each other in both the x- and y-direction. Such a configuration can provide tactile feedback to the user's index and middle fingertips, which are commonly offset in a similar manner as the first and second tactors 11 On', 11 On". This offset configuration can be advantageous if the shear display device 100η was a computer mouse or a similar device.

[00150] In other embodiments, the first and second tactors 11 On', 11 On" can be substantially aligned with each other. In other words, the shear display device 100η can have a continuous edge, and the first and second tactors 1 lOn', 110η" can be positioned at approximately the same distance from the edge. In any case, the user can position or orient the shear display device 100η in any manner relative to the user's hand and/or fingertips, such as to accommodate a particular placement of the user's fingertips on the first and second tactors 1 ΙΟη', 110η".

[00151] Additional or alternative embodiments include the shear display device

100η that comprises multiple unconnected bodies, each of which can incorporate one of the first and second tactors 11 On', 11 On". In other words, the shear display device 100η can comprise multiple unconnected shear display devices, which can be place near each other, such that the user can place fingertips on the desired tactors. Such otherwise unconnected shear display devices can function together (e.g., by receiving commands from the same controller) and can produce the same movement of the first and second tactors 1 ΙΟη', 110η" as a single body shear display device 100η.

[00152] Furthermore, it should be appreciated that the shear display device 100η can include first and second tactors 11 On', 11 On" that have any number suitable shapes and sizes, which may vary from one implementation to another. For example, the shear display device 100η can include bar- or plate-like first and second tactors (e.g., similar to the first and second tactors HOg', HOg" (Figure 8). The plate-like first and second tactors can move in any number of ways described herein and can provide the same or similar information as described herein. Additionally, the plate-like first and second tactors can allow the user to apply a relatively large compressive force to the shear display device 100η, while continuing to display information to the user. Furthermore, as noted above, the plate-like tactors can recruit more mechanoreceptors and, thus, can provide additional sensation in the areas of relatively low density of mechanoreceptors (e.g., palm, wrist, etc.), which can facilitate greater accuracy of the user's perception of the displayed information.

[00153] Hence, movements of the tactor can display one-, two-, three-dimensional, and generally multi-dimensional information about movement of the controlled object. As described herein, the shear display device can be connected to or form a part of a controller, which can control the controlled object. Moreover, the controller also can provide force feedback to the user, as further described herein. For ease of description of the controller, reference is made to a "force feedback device," which can incorporate both the control functionality and can provide force feedback. Furthermore, the force feedback device can provide force feedback on any one or more of three axes in a three- dimensional space as well as torque feedback about any one of the three axes. For example, the feedback can be such as to provide a certain amount of resistance to movement of a shear display device. It should be appreciated, however, that shear display device can be coupled to or integrated with a controller that can provide either of the control functionality or force feedback. Figure 12 illustrates a control system 300 that includes the shear display device 100a connected to a force feedback device 310. The force feedback device 310 may include a commercially available force feedback device, such as the Phantom Robot Arm.

[00154] The force feedback device 310 can provide resistive force in response to the movement of the shear display device 100a. Such resistive force, for example, can signal a resistance (as well as magnitude thereof) encountered by the controlled object (where this object could be a physical object or a simulated/virtual object). For example, when used with a remote scalpel that is cutting through subcutaneous tissue and then encounters bone, the force feedback device 310 may provide greater (or in this example infinite) resistance to further advancement in the direction of the bone. Additionally, the force feedback device 310 can sense movements of the shear display device 100a in one-, two-, three-, and up to six-dimensional space (i.e., linear movements in along any one or more axes in a three-dimensional space plus rotational movements about such axes in the three-dimensional space). In other words, as the user moves the shear display device 100a, the force feedback device 310 can provide instructions to a controller (or to the controlled object), and the instructions can correspond to the user's movements of the shear display device 100a. Thus, the force feedback device 310 can provide information to the user about position and/or movement of the controlled object as well as provide instruction to the controlled object related to the movement thereof (e.g., as sensed by the force feedback device 310 from the movement of the shear display device 100a).

[00155] Moreover, in addition to or in lieu of force feedback, the control system

300 also can provide the user with information about the movement of an object, such as the controlled object. More specifically, the control system 300 can display position of the controlled object, forces experienced by of the controlled object, etc., via the shear display device 100a. For example, the force feedback device 310 can detect rotation of the shear display device 100a about the Z-axis.

[00156] Similarly, the force feedback device 310 can detect rotation of the shear display device 100a about the X-axis. As noted above, as the force feedback device 310 detects rotation of the shear display device 100a about and/or linear movement of the shear display device 100a in the X-, Y-, and/or Z-axes (or any other axes), the force feedback device 310 can translate such linear movements and/or rotations, and direct the controlled object in a corresponding manner. [00157] Similarly, the tactor 110a and its motion can display a representative force experienced by the controlled object. Hence, in lieu of providing force feedback in response to force experienced by the controlled object (via the force feedback device 310), resistance to linear movement and/or rotation of a multi-tactor shear display device (e.g., shear display device 100k, Figures 10A-10B) can be presented as skin shear (i.e., stretching the skin) by the tactor 110a. For instance, as noted above, force may be displayed by displacing the tactor 110a from a center or default position to a second position. Also, the shear display device 100a can display torque by rotating the tactor 110a about its axis and/or moving the tactor 110a along a circular path. In one example, direction and distance of movement (or speed or acceleration) of the tactor 110a from its center position can indicate the forces experienced by the controlled object. In other words, relatively larger movements of the tactor 110a can represent a relatively greater amount of force experienced by the controlled object. Conversely, a relatively smaller movement of the tactor 110a can represent a relatively lower amount of force experienced by the controlled object.

[00158] As described herein, a shear display device provides localized sensations, which do not affect or move the user's hand. Hence, providing haptic feedback via skin shear instead of force feedback can reduce the risk associated with unintentional hand movement, which can occur in response to receiving force feedback from the force feedback device 310. For example, in various medical applications, unintentional hand movements during a procedure where a physician controls a medical device with the help of the control system (e.g., during a surgery) can present huge safety concerns and can lead to devastating consequences. Accordingly, the shear display device 100a can reduce potential accidents that may occur during various procedures, which can be especially relevant to high risk procedures.

[00159] Additionally, the control system 300 can provide a combined force feedback and shear feedback. For example, the force feedback device 310 can prevent movement of the shear feedback device 100a in a certain direction (e.g., along X-axis beyond a predetermined point or a limit), and the shear feedback device 100a can provide shear feedback (e.g., by moving in the tactor 110a in a direction away from the predetermined limit). As such, the control system 300 can reduce the amount of force applied in the force feedback, while the user can experience sufficient tactile feedback to for accurate responses to such feedback. Combined force feedback and skin stretch can enhance user's sensation and can lead to more accurate interpretation of the cues as compared with providing only the force feedback or skin stretch cues.

[00160] As noted above, reduction of force feedback can lead to improved accuracy of user's movements during user controlled tasks. For instance, the user can guide the controlled object along a path that is in part dictated by the tactile feedback from the control system 300. Thus, increased accuracy of user's movements can result in increased accuracy of the control object's adherence to the path. In one example, the control system 300 can be used in surgical procedures, where the tactile feedback can represent the controlled object's interaction with the surrounding environment (e.g., resistance or forces experienced by the controlled object). Accordingly, increased accuracy of movement of the controlled device as well as reduced or eliminated unintentional movements (e.g., movements that can result from excessive force feedback) of the shear display device 110a, which can control the controlled object, can lead to safer surgical procedures.

[00161] As noted above, the control system 300 can include any of the shear display devices described herein. In one example, the control system 300 may incorporate a multi-tactor shear display device (e.g., the shear display device lOOd (Figure 5)). Hence, in some instances, the user can receive cues from the multi-tactor shear display device (as described herein) and can move the shear display device in a direction and/or to a position indicated by such cues. For example, the user may rotate the shear display device about the Z-axis (e.g., which would result from the shear feedback provided to a user). It should be noted that, in some situations, the shear display device can move only about the Z-axis, while remaining substantially stationary otherwise (e.g., via force feedback that prevents linear movement and rotation about the X and Y axes.

[00162] As described herein, rotation of the shear display device 100a about the Z- axis can direct the controlled object to perform certain function. For example the controlled object can be directed to rotate. In response to such rotation the controlled object can experience torque applied by its environment. Hence, providing shear feedback can inform the user about the amount of torque experienced by or the orientation or relative movement of the controlled object. Moreover, the information about the torque experienced by the controlled object also can be displayed by a combined skin shear and force feedback. [00163] In some embodiments, the control system 300 can be configured to partially restrict the user's ability to move the shear display device in three- or six- dimensional space. In other words, the force feedback device 310 can apply force to prevent the user's movements of the shear display device in one or more directions and/or prevent rotations about one or more axes. For example, the control system 200a may restrict all linear movement of the shear display device and can restrict rotation about X-, Y- or Z- axes. Hence, for instance, the shear display device can be allowed to rotate only about the Z-axis. In other words, the force/torque feedback device 310 can provide resistance to movement of the shear display device in a manner that would effectively constrain the movements of the shear display device to a predetermined plane, surface, or three-dimensional surface or area (e.g., allowing movements only on the surface or on one side of the plane or surface).

[00164] Such restrictions also can confine movement of the shear display device to a particular confinement plane or surface. Thus, the user may be able to move the shear display device only in a two-dimensional plane. Similarly, the control system 300 can restrict the user's movements of the shear display device to a surface, which can have a three-dimensional profile.

[00165] In other embodiments, the control system 300 can prevent the user from moving the shear display device past a predetermined safety plane or surface in the three- dimensional space. More specifically, the control system 300 can allow the user to move the shear display device to any location on the one side of the safety plane or surface and prohibit movement of the shear display device beyond that side of the safety plane or surface. A surface of an object can be obtained (e.g., using three-dimensional scanning techniques) and can be used as a safety surface for the control system 300. Such safety surface can protect the object from inadvertent contact or impact by the controlled object.

[00166] Guiding the motion and orientation of the user while the user holds the shear display device such that the user can only move within a plane or surface can also be very advantageous cognitively and for task performance. This reduces the task space such that a user is no longer required to reason and control for six degrees of freedom of motion, but can simply focus on two degrees of freedom. This also allows the axis of the shear display to always be controlled to be in the most advantageous orientation to present useful feedback to the user.

[00167] In one aspect of the control system, the center of the shear display device

(i.e., location of the tactor and/or where the user grips the shear display) would be placed at the center of rotation of the force/torque feedback robot's gimbal. By placing the shear display device at the center of the gimbal, if torques are applied to the shear display device to guide its motions, these torques will be less likely to result in potentially large translational forces being generated when the robot arm pushes on the user's hand. Hence this centered configuration has potential safety advantages.

[00168] Furthermore, providing torque feedback at the force feedback device 310 can provide a means of controlling the orientation of the shear display device 100a such that it always lies in the predetermined or desired orientation. All three axes of rotation can be similarly controlled such that the orientation of the shear display device can be suggested to or controlled for the user. Specifically, rotation of the shear display device can be controlled about any one or more of the X, Y, and Z axes.

[00169] Providing rotational guidance for the orientation of the shear display device and, thus, for the user's hand provides a compromise in system complexity and safety. As noted above, in safety critical applications force feedback is often not used due to concerns of feedback instabilities. Hence, torque feedback can provide an effective means to guide the orientation of the shear display device to be placed in a desired or preferred orientation within a three-dimensional space with three translations and three rotations, for conveying task specific information to the shear display device. Unlike force feedback, where transitional motion can result from feedback instabilities, torque feedback has a lower potential to cause safety issues.

[00170] Accordingly, Figures 1-12 and the corresponding text, provide a number of different components and mechanisms for displaying movement, direction, force, and torque information to a user via tactile cues provided by the shear display device and/or by the control system. In addition to the foregoing, some embodiments of the present invention also can be described in terms of flowcharts comprising acts and steps in a method for accomplishing a particular result. For example, Figure 13 illustrates a flowchart of one exemplary method of providing above-described information with tactile cues using principles of the present invention. The acts of Figure 13 are described herein with reference to the components and diagrams of Figures 1 through 12.

[00171] For example, as illustrated in Figure 13, the method can involve an act 400 of receiving information about an object. As mentioned above, such object can be a real (or physical) object or can be a virtual (i.e., computer-generated) object. Furthermore, such an object can be an object being controlled by the user, such as the controlled object, an object used to control one or more other objects (e.g., shear display device in combination with a controller), or any other object being observed by the user.

[00172] In addition, the particular information received can vary from one embodiment to another. For instance, such information can include information about the object's movements. Particularly, such information can include information about the object's direction, velocity, and acceleration. Additionally or alternatively, such information can include information about the forces, torques, or pressure experienced by or applied to the object.

[00173] The method can further involve an act 410 of displaying the information to the user via one or more tactile cues. More specifically, the above-described information about the object can be displayed to the user via predetermined one or more skin stretch cues, which can be provided by the shear display device (e.g. shear display device 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, 100k, 100η). In other words, the shear display device can stretch one or more portions of the user's skin to provide tactile sensations that can correspond with the information about the object and can be interpreted by the user as such. For example, the shear display device can move one or more tactors (e.g. tactors 110a, 110b, 110c*, 110c", HOd*, 110d", HOe*, 110e", 110f, l lOf", 110g', 110g", l lOh', 1 lOh", etc.) in a first direction and at a first speed in along a linear path to indicate linear in a first direction and at a first speed in along a linear path to indicate linear movement of the object and particular speed thereof. Alternatively or additionally, the shear display device can move one or more tactors in a first direction and at a first speed and/or by a first displacement along a linear path to indicate a force experienced by the object (i.e., magnitude of the force and/or direction thereof).

[00174] Additionally or alternatively, the shear display device can move one or more tactors along a curved, circular, or semicircular path to indicate rotation of or a torque experienced by the object. Moreover, as described herein, the shear display device can include multiple tactors (e.g. 110c*, 110c", HOd*, 110d", HOe*, HOe", HOf, l lOf", 110g', 1 lOg", 110h', 1 lOh", etc.). Hence, the shear display device can move the tactors in a coordinated manner to display linear movement, force, rotation, torque, and combinations thereof. In one example, opposing tactors can move in opposite directions (e.g., along X- or Y-axis), as described herein, to indicate rotation of and/or torque experienced by the object. For instance, directions of the movement of the tactors and the direction of a perceivable torque created about a center point therebetween can provide information about the object's rotation. Furthermore, speed and/or acceleration as well as displacement of the tactors (i.e., the manner in which the user's skin is stretched) can provide information about the torque experienced by the object.

[00175] Likewise, tactors facing in the same or similar direction (e.g., adjacent tactors) also can provide rotational information. As noted above, in one example, one or more of the tactors can rotate about the respective axes thereof or may move in a circular or semicircular paths to display information about the object's rotation and/or torque experienced thereby. Additionally or alternatively, the tactors can move in opposite directions along a linear or substantially linear paths (e.g., along X- or Y-axis), thereby producing a perception of torque about a point therebetween. Accordingly, such movements can provide information about the object's rotational movement and/or torque experienced thereby.

[00176] In some embodiments, as described herein, the tactors also can move toward and away from the user's skin. Hence, the tactors can provide three-dimensional information about linear movement or forces experienced by the object. Additionally or alternatively, the tactors can provide information about rotational movement and/or torques experienced by the object about any one or more of the three-dimensional axes (i.e., the shear display device 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, etc., can provide four-, five, and six-dimensional information to the user).

[00177] Embodiments of the present invention also can include and/or can be implemented in performing medical procedures, such as upper extremity rehabilitation, surgery, catheter insertion, etc. More specifically, in one example, a shear display device (e.g. shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, etc.) can signal to the user the direction of movement (i.e., where to move) for a desired advancement of the tool or catheter in the body. Also, the shear display device can inform the user about the forces and/or torques experienced by the catheter or other controlled tool or object. Accordingly, the shear display device can improve the safety of medical procedures, such as catheter insertion and guidance.

[00178] In some embodiments, the shear display device can display impact between two objects (e.g., between a controlled object and another object) in a different manner than a force applied onto the controlled object. Specifically, upon impact, the shear display device can produce a relatively large initial stretch of the user's skin (i.e., a large displacement of the tactors (e.g. tactors 110c', 110c", 110d', 110d", 110e', 110e", HOf, 110f", HOg*, 110g", HOh*, 110h", etc.)), which can be proportional to the impact (e.g., to the speed of the controlled object at the time of impact and/or mass thereof), followed by displaying further tactor motion in proportion to the penetration of the virtual object that is contacted. In other words, a larger initial stretch of the user's skin can produce an enhanced or more accurate perception of impact. It should be noted that a single or multiple tactors (as applicable) can move in unison to produce skin stretch of multiple portions of the user's skin.

[00179] In some embodiments, the displacement of the tactor and the corresponding amount of skin stretch can be proportional to the force or torque experienced by the object. For instance, a force of 2 N can be displayed by displacing the tactor by 1 mm, while a force of 4 N can be displayed by displacing the tactor by 2 mm. Alternatively, relationships between the force experienced by the object and the displacement of the tactor can be non-linear (e.g., logarithmic, quadratic, etc.). Thus, small amounts of force can be displayed to the user by sufficiently large displacement of the tactor, such that the user can perceive such forces. At the same time, larger forces also can be displayed by proportionately less displacement by the tactor without running out of travel. Hence a non-linear mapping of forces (or other quantities mentioned above) to tactor displacements has the advantage of being responsive for displaying lower force levels where the skin is not substantially stretched, while not prematurely saturating at higher force levels. This extends the range of forces that can be displayed to a user and has the advantage of meeting a user's expectations of increased skin stretch when experiencing increasing forces or large forces. This can better maintain the causality of the user experience than using a high force-to-tactor-displacement gain and experiencing early tactor saturation at high forces.

[00180] The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

O 2013/187977 - Page 50 - PCT/US2013/032053 CLAIMS We claim:
1. A shear display device for displaying tactile information and cues to a user, the device comprising:
a body;
a first motor;
a crank coupled to the first motor;
a slider slidably positioned within the body, a first end of the slider being coupled to the crank in a manner that rotation of the first motor produces a linear movement of the slider; and
a tactor coupled to the slider.
2. The device of claim 1, further comprising a second motor coupled to or secured within the slider in a manner that the second motor moves linearly together with the slider, and the second motor being coupled to the tactor in a manner that rotation of the second motor produces rotational movement of the tactor.
3. The device of claim 2, further comprising a well located on the body, the well at least partially surrounding the tactor.
4. The device of claim 2, wherein one or more of the first motor and the second motor are servo motors.
5. The device of claim 2, wherein the body has an approximately cylindrical shape, and the rotational movement of the tactor approximately follows the cylindrical shape of the body.
6. A shear display device for displaying tactile information and cues to a user, the device comprising:
a body;
a first actuator assembly at least partially located within or secured to the body, the actuator assembly comprising:
a sliding housing;
a first motor secured to the sliding housing;
a worm coupled to the first motor;
one or more gears engaged with the worm, the one or more gears being oriented substantially orthogonally relative to the worm; O 2013/187977 - Page 51 - PCT/US2013/032053 a cam coupled to the one or more gears, wherein the body includes a slot configured to accept the cam in a manner that rotation of the cam within the slot and produces movement of the sliding housing relative to the body; and a first tactor coupled to the sliding body in a manner that the first tactor can move relative to the body.
7. The device of claim 6, wherein the one or more gears comprise a first gear engaged with the worm, the first gear being operably coupled to the cam in a manner that rotation of the first gear rotates the cam.
8. The device of claim 7, wherein the one or more gears further comprise:
a second gear coupled to the first gear in a manner that rotation of the first gear in a clockwise direction produces rotation of the second gear in the clockwise direction; and a third gear coupled to the cam in a manner that rotation of the third gear in a counterclockwise direction produce rotation of the cam in the counterclockwise direction, the third gear being engaged with the second gear in a manner that rotation of the second gear in the clockwise direction produces rotation of the third gear in the counterclockwise direction.
9. The device of claim 6, further comprising a well surrounding the tactor.
10. The device of claim 6, wherein the first tactor has an approximately cylindrical shape.
11. The device of claim 6, wherein the first tactor has an approximately plate-like shape.
12. The device of claim 6, further comprising:
a second actuator assembly at least partially located within or secured to the body; and
a second tactor operably coupled to the second actuator assembly.
13. The device of claim 12, wherein the first actuator assembly is the same as the second actuator assembly.
14. The device of claim 12, wherein the first tactor and the second tactor face in the same direction.
15. The device of claim 12, wherein the first tactor faces in an opposite direction than the second tactor.
16. The device of claim 12, wherein the first and second tactors are oriented at a non- parallel angle relative to each other. O 2013/187977 - Page 52 - PCT/US2013/032053
17. The device of claim 6, wherein the body includes one or more non-planar surfaces, and the movement of the tactor relative to the body approximately follows at least one non-planar surface of the one or more non-planar surfaces of the body.
18. A shear display device for displaying tactile information and cues to a user, the device comprising:
a body sized and configured to be grasped by the user's hand;
an actuator assembly at least partially located within or secured to the body, the actuator assembly comprising:
a motor;
a crank coupled to the motor; and
a flexible spine having a first end coupled to the crank in a manner that rotation of the motor in a clockwise direction moves the flexible spine in a first direction and rotation of the crank in a counterclockwise direction moves the flexible spine in a second direction that is opposite to the first direction; and a first tactor coupled to a second end of the flexible spine in a manner that the flexible spine moves the first tactor in the first and second directions.
19. The device of claim 18, wherein the first tactor is located outside of the body.
20. The device of claim 18, wherein the first tactor has a plate-like shape.
21. The device of claim 18, wherein the body is further configured to restrain at least a portion of skin on user's hand relative to the first tactor.
22. The device of claim 18, wherein the body has a joystick-like configuration.
23. The device of claim 22, further comprising a second tactor located on an upper portion of the body in a manner that the second tactor can be engaged by the user's thumb.
24. The device of claim 23, wherein the second tactor is offset from a centerline of the body.
25. The device of claim 18, wherein the body includes at least one non-planar surface, and movement of the first tactor approximately follows the at least one non-planer surface when the first tactor moves in one or more of the first and second directions. O 2013/187977 - Page 53 - PCT/US2013/032053
26. The device of claim 18, further comprising a second tactor, the second tactor being opposite to the first tactor, the second tactor being movable in the first and second directions.
27. A shear display device for displaying tactile information and cues to a user, the device comprising:
a body;
a first tactor having a first area, the first tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a first density of mechanoreceptors;
a second tactor having a second area, the second tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a second density of mechanoreceptors; and
wherein:
the second area is greater than the first area, and
the first density of mechanoreceptors is greater than the second density of mechanoreceptors .
28. A shear display device for displaying tactile information and cues to a user, the device comprising:
a body having an elongated portion;
a first tactor positioned along the elongated portion of the body, the first tactor being movable along a length of the elongated portion of the body;
a second tactor positioned along the elongated body, the second tactor being movable along the length of the elongated portion of the body, the second tactor being approximately opposite to the first tactor; and
a third tactor positioned along the elongated body, the third tactor being movable along the length of the elongated portion of the body.
29. The device of claim 28, further comprising a fourth tactor positioned along the elongated body, the fourth tactor being movable along the length of the elongated portion of the body, the fourth tactor being opposite to the third tactor.
30. A shear display device for displaying tactile information and cues to a user, the device comprising:
a body; O 2013/187977 - Page 54 - PCT/US2013/032053 a first tactor having a first area, the first tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a first density of mechanoreceptors;
a second tactor having a second area, the second tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a second density of mechanoreceptors; and
wherein:
the first area has a first proportion relative to the first density of mechanoreceptors;
the second area has a second proportion relative to the second density of mechanoreceptors; and
the first proportion and the second proportion are approximately the same.
31. A control system for controlling an object and receiving tactile feedback about the movement of the object, forces experienced by the object, torques experienced by the object, and combinations thereof, the system comprising:
a shear display device including:
a body;
an actuator assembly at least partially located with or secured to the body; and
a first tactor coupled to the actuator assembly, the first tactor being movable in a two-dimensional or a three-dimensional space by the actuator assembly; and
a controller operably connected to the shear display device, the controller being configured to receive instructions from the shear display device and to communicate the instructions to a controlled object.
32. A method for displaying movement information related to one or more objects as well as information about forces and torques experienced thereby, the method comprising:
receiving information about one or more of movement of an object, forces experienced by the object, and torques experienced by the object;
displaying the information to a user by concurrently stretching a plurality of portions of user's skin with a plurality of tactors of one or more shear display devices; wherein: O 2013/187977 - Page 55 - PCT/US2013/032053 at least one tactor of the plurality of tactors moves in a two-dimensional space or in a three-dimensional space.
33. A method for displaying movement information related to one or more objects as well as information about torque or rotational motion experienced thereby, the method comprising:
receiving information about one or more of rotation of an object and torque experienced by the object;
isolating a first portion of a user's skin relative to a body of a shear display device; moving a first tactor of the shear display device in a first direction along a linear path, the first tactor being in contact with the isolated first portion of the user's skin; isolating a second portion of a user's skin relative to a body of the shear display device; and
moving a second tactor of the shear display device in a second direction along a linear path, the second tactor being in contact with the isolated second portion of the user's skin, the second direction being opposite to the first direction.
34. The method of claim 33, further comprising:
isolating a third portion of a user's skin relative to a body of a the shear display device; and
moving a third tactor of the shear display device in the second direction along a linear path, the third tactor being in contact with the isolated third portion of the user's skin.
35. A method for displaying information about a change in size of an object, the method comprising:
receiving information about the change in size of the object;
isolating a first portion of a user's skin relative to a body of a shear display device; moving a first tactor of the shear display device in a first direction along a linear path, the first tactor being in contact with the isolated first portion of the user's skin; isolating a second portion of a user's skin relative to a body of a shear display device; and
moving a second tactor of the shear display device in a second direction along a linear path, the second tactor being in contact with the isolated second portion of the user's skin, the second direction being opposite to the first direction and away from or towards the first tactor.
PCT/US2013/032053 2012-06-13 2013-03-15 Skin stretch feedback devices, systems, and methods WO2013187977A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US201261659421P true 2012-06-13 2012-06-13
US61/659,421 2012-06-13
US201261659452P true 2012-06-14 2012-06-14
US61/659,452 2012-06-14
US201261660162P true 2012-06-15 2012-06-15
US61/660,162 2012-06-15

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
EP13803464.0A EP2862151A4 (en) 2012-06-13 2013-03-15 Skin stretch feedback devices, systems, and methods
US14/517,674 US9317123B2 (en) 2012-06-13 2014-10-17 Skin stretch feedback devices, systems, and methods
US15/043,140 US9767659B2 (en) 2012-06-13 2016-02-12 Skin stretch feedback devices, systems, and methods
US15/675,076 US10152853B2 (en) 2012-06-13 2017-08-11 Skin stretch feedback devices, systems, and methods
US16/179,707 US20190073038A1 (en) 2012-06-13 2018-11-02 Skin stretch feedback devices, systems, and methods

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/517,674 Continuation-In-Part US9317123B2 (en) 2012-06-13 2014-10-17 Skin stretch feedback devices, systems, and methods

Publications (1)

Publication Number Publication Date
WO2013187977A1 true WO2013187977A1 (en) 2013-12-19

Family

ID=49758596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/032053 WO2013187977A1 (en) 2012-06-13 2013-03-15 Skin stretch feedback devices, systems, and methods

Country Status (2)

Country Link
EP (1) EP2862151A4 (en)
WO (1) WO2013187977A1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9317123B2 (en) 2012-06-13 2016-04-19 University Of Utah Research Foundation Skin stretch feedback devices, systems, and methods
US9946350B2 (en) 2014-12-01 2018-04-17 Qatar University Cutaneous haptic feedback system and methods of use
EP3213539A4 (en) * 2014-11-25 2018-06-20 Nokia Technologies Oy Apparatuses, methods and computer programs for providing vehicle feedback

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4591868A (en) * 1982-04-09 1986-05-27 National Industries, Inc. Collapsible motor operated antenna
WO2009018330A2 (en) 2007-07-30 2009-02-05 University Of Utah Research Foundation Shear tactile display system for communicating direction and other tactile cues
US20090139359A1 (en) 2007-12-04 2009-06-04 Cybernet Systems Corporation Cable driven tactor
US20100161079A1 (en) 2008-12-22 2010-06-24 The Board Of Trustees Of The Leland Stanford Junior University Skin stretch tactile feedback device
US20110115754A1 (en) * 2009-11-17 2011-05-19 Immersion Corporation Systems and Methods For A Friction Rotary Device For Haptic Feedback
US8004052B2 (en) * 2003-12-29 2011-08-23 Vladimir Vaganov Three-dimensional analog input control device
US20120038468A1 (en) * 2007-07-30 2012-02-16 University Of Utah Multidirectional controller with shear feedback
US8125453B2 (en) * 2002-10-20 2012-02-28 Immersion Corporation System and method for providing rotational haptic feedback

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4591868A (en) * 1982-04-09 1986-05-27 National Industries, Inc. Collapsible motor operated antenna
US8125453B2 (en) * 2002-10-20 2012-02-28 Immersion Corporation System and method for providing rotational haptic feedback
US8004052B2 (en) * 2003-12-29 2011-08-23 Vladimir Vaganov Three-dimensional analog input control device
US20120038468A1 (en) * 2007-07-30 2012-02-16 University Of Utah Multidirectional controller with shear feedback
WO2009018330A2 (en) 2007-07-30 2009-02-05 University Of Utah Research Foundation Shear tactile display system for communicating direction and other tactile cues
US20090139359A1 (en) 2007-12-04 2009-06-04 Cybernet Systems Corporation Cable driven tactor
US20100161079A1 (en) 2008-12-22 2010-06-24 The Board Of Trustees Of The Leland Stanford Junior University Skin stretch tactile feedback device
US20110115754A1 (en) * 2009-11-17 2011-05-19 Immersion Corporation Systems and Methods For A Friction Rotary Device For Haptic Feedback

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2862151A4 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9317123B2 (en) 2012-06-13 2016-04-19 University Of Utah Research Foundation Skin stretch feedback devices, systems, and methods
US9767659B2 (en) 2012-06-13 2017-09-19 The University Of Utah Research Foundation Skin stretch feedback devices, systems, and methods
US10152853B2 (en) 2012-06-13 2018-12-11 University Of Utah Research Foundation Skin stretch feedback devices, systems, and methods
EP3213539A4 (en) * 2014-11-25 2018-06-20 Nokia Technologies Oy Apparatuses, methods and computer programs for providing vehicle feedback
US9946350B2 (en) 2014-12-01 2018-04-17 Qatar University Cutaneous haptic feedback system and methods of use

Also Published As

Publication number Publication date
EP2862151A1 (en) 2015-04-22
EP2862151A4 (en) 2016-01-13

Similar Documents

Publication Publication Date Title
Biggs et al. Haptic interfaces
Prattichizzo et al. Towards wearability in fingertip haptics: a 3-dof wearable device for cutaneous force feedback
US7821498B2 (en) Interface for controlling a graphical image
Tavakoli et al. A force reflective master-slave system for minimally invasive surgery
US7731588B2 (en) Remote vehicle control system
CA2254854C (en) Method and apparatus for providing simulated physical interactions within computer generated environments
Massie Initial haptic explorations with the phantom: Virtual touch through point interaction
Iwata et al. Project FEELEX: adding haptic surface to graphics
Lederman et al. Sensing and displaying spatially distributed fingertip forces in haptic interfaces for teleoperator and virtual environment systems
CN102596086B (en) Minimally invasive surgery system master finger tracking device
KR101009865B1 (en) System and method for providing force feedback to a user
Laycock et al. Recent developments and applications of haptic devices
US20040108992A1 (en) Isotonic-isometric haptic feedback interface
Richard et al. Effect of frame rate and force feedback on virtual object manipulation
US5709219A (en) Method and apparatus to create a complex tactile sensation
US8918211B2 (en) Medical robotic system providing sensory feedback indicating a difference between a commanded state and a preferred pose of an articulated instrument
US7131073B2 (en) Force feedback applications based on cursor engagement with graphical targets
US20030210259A1 (en) Multi-tactile display haptic interface device
JP6314134B2 (en) User interface for robot training
Prattichizzo et al. Cutaneous force feedback as a sensory subtraction technique in haptics
US6946812B1 (en) Method and apparatus for providing force feedback using multiple grounded actuators
US9174344B2 (en) Method and apparatus for haptic control
Pacchierotti et al. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery
US6061004A (en) Providing force feedback using an interface device including an indexing function
Hinckley et al. Cooperative bimanual action

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13803464

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2013803464

Country of ref document: EP

NENP Non-entry into the national phase in:

Ref country code: DE