EP2862151A1 - Dispositifs, systèmes et méthodes de rétroaction d'étirement de peau - Google Patents

Dispositifs, systèmes et méthodes de rétroaction d'étirement de peau

Info

Publication number
EP2862151A1
EP2862151A1 EP13803464.0A EP13803464A EP2862151A1 EP 2862151 A1 EP2862151 A1 EP 2862151A1 EP 13803464 A EP13803464 A EP 13803464A EP 2862151 A1 EP2862151 A1 EP 2862151A1
Authority
EP
European Patent Office
Prior art keywords
tactor
user
display device
shear display
skin
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP13803464.0A
Other languages
German (de)
English (en)
Other versions
EP2862151A4 (fr
Inventor
William R. Provancher
Markus N. MONTANDON
Landon T. Gwilliam
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Utah Research Foundation UURF
Original Assignee
University of Utah Research Foundation UURF
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Utah Research Foundation UURF filed Critical University of Utah Research Foundation UURF
Publication of EP2862151A1 publication Critical patent/EP2862151A1/fr
Publication of EP2862151A4 publication Critical patent/EP2862151A4/fr
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/24—Constructional details thereof, e.g. game controllers with detachable joystick handles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B6/00—Tactile signalling systems, e.g. personal calling systems
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1043—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being characterized by constructional details

Definitions

  • the present disclosure relates generally to haptic feedback devices and more specifically to the devices that provide tactile shear feedback.
  • haptics The field of haptics is the science of interfacing with users via the sense of touch by applying forces, vibrations, or motions to a user.
  • Haptic devices are increasingly used to provide a user with sensory input that conveys information about the surrounding environment. For instance, a haptic device can produce vibratory motions to provide the user, through his or her sense of touch, with various types of information.
  • Haptic devices are commonly used in the field of gaming to provide sensory cues related to the user's environment.
  • haptic devices are integrated into controllers (e.g., joystick), so the user can receive haptic feedback that relates to movements of the controller and/or an object being controlled.
  • controllers e.g., joystick
  • Typical haptic devices provide vibratory or force stimuli to display information to the user.
  • such stimuli can interfere with the user's ability to accurately and/or safely manipulate the controller that provides such haptic feedback.
  • force feedback on a controller that is used to operate a crane can result in unintended movements of the user's hand, which may, in turn, move the controller to an unintended position, thereby causing an accident. Similar results occur in video games when utilizing a force feedback joystick to control game play; however, this resulting loss of control has less catastrophic consequences.
  • vibration feedback and especially force feedback may reduce safety of operation of a controller that integrates such haptic feedback mechanisms. This is especially of concern in safety critical applications, such as robotic surgery or catheterization. Such concerns are the reason why force feedback of tool tip forces are not currently permitted in robotically enabled surgical applications.
  • Implementations of the present invention provide devices, systems, and methods for displaying information about direction of movement, speed, resistive force experienced, other aspects of movement for an object, or combinations thereof.
  • the device can also be used to display or feedback a wide variety of information that has magnitude and/or direction associated with it (e.g., temperature, motion, pressure, force, volume, proximity, other information, or combinations thereof) or give guidance about where a person should move (e.g., to push forward on an aircraft control stick to prevent stall from occurring).
  • the present disclosure provides a shear display device that can generate skin shear with one or more tactors moving in a two- or three- dimensional space. The movement of the tactors can represent to a user various aspects of an object (e.g., object being controlled by the user or a controlled object).
  • At least one embodiment of the present invention includes a shear display device for displaying tactile information and cues to a user.
  • a shear display device for displaying tactile information and cues to a user.
  • the device may include a body, a first motor and a crank coupled to the first motor.
  • the device also may include a slider slidably positioned within the body, a first end of the slider being coupled to the crank in a manner that rotation of the first motor produces a linear movement of the slider.
  • the device may include a tactor coupled to the slider.
  • One or more embodiments may include another shear display device for displaying tactile information and cues to a user.
  • the device may incorporate a body, a first actuator assembly at least partially located within or secured to the body, the actuator assembly.
  • the actuator assembly may include a sliding housing, a first motor secured to the sliding housing, a worm coupled to the first motor, and one or more gears engaged with the worm, the one or more gears being oriented substantially orthogonally relative to the worm.
  • the actuator assembly may include a cam coupled to the one or more gears, wherein the body includes a slot configured to accept the cam in a manner that rotation of the cam within the slot and produces movement of the sliding housing relative to the body.
  • the device may include a first tactor coupled to the sliding body in a manner that the first tactor can move relative to the body.
  • embodiments of the present invention may include yet another shear display device for displaying tactile information and cues to a user.
  • the device may have a body sized and configured to be grasped by the user's hand and an actuator assembly at least partially located within or secured to the body.
  • the actuator assembly may include a motor, a crank coupled to the motor, and a flexible spine having a first end coupled to the crank in a manner that rotation of the motor in a clockwise direction moves the flexible spine in a first direction and rotation of the crank in a counterclockwise direction moves the flexible spine in a second direction that is opposite to the first direction.
  • the device also may include a first tactor coupled to a second end of the flexible spine in a manner that the flexible spine moves the first tactor in the first and second directions.
  • Yet another embodiment of the present invention may include one other shear display device for displaying tactile information and cues to a user.
  • the device may have a body and a first tactor having a first area, the first tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a first density of mechanoreceptors.
  • the device also may include a second tactor having a second area, the second tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a second density of mechanoreceptors.
  • the second area may be greater than the first area
  • the first density of mechanoreceptors may be greater than the second density of mechanoreceptors.
  • Embodiment also may include still one other shear display device for displaying tactile information and cues to a user.
  • the device may incorporate a body having an elongated portion and a first tactor positioned along the elongated portion of the body, the first tactor being movable along a length of the elongated portion of the body.
  • the device also may have a second tactor positioned along the elongated body, the second tactor being movable along the length of the elongated portion of the body, the second tactor being opposite to the first tactor.
  • the device may include a third tactor positioned along the elongated body, the third tactor being movable along the length of the elongated portion of the body.
  • the device may include a fourth tactor positioned along the elongated body, the fourth tactor being movable along the length of the elongated portion of the body.
  • Additional or alternative embodiments may include one other shear display device for displaying tactile information and cues to a user.
  • the device may have a body and a first tactor having a first area, the first tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a first density of mechanoreceptors.
  • the device also may include a second tactor having a second area, the second tactor being positioned and oriented relative to the body to engage a portion of the user's skin having a second density of mechanoreceptors.
  • the first area may have a first proportion relative to the first density of mechanoreceptors
  • the second area may have a second proportion relative to the second density of mechanoreceptors.
  • the first proportion and the second proportion may be approximately the same.
  • One or more embodiments may include a control system for controlling an object and receiving tactile feedback about the movement of the object, forces experienced by the object, torques experienced by the object, and combinations thereof.
  • the system may include a shear display device including, which may have a body, an actuator assembly at least partially located with or secured to the body, and a first tactor coupled to the actuator assembly, the first tactor being movable in a two-dimensional or a three- dimensional space by the actuator assembly.
  • the system also may include a controller operably connected to the shear display device, the controller being configured to receive instructions from the shear display device and to communicate the instructions to a controlled object.
  • Embodiments of the present invention also may involve a method for displaying movement information related to one or more objects as well as information about torque or rotational motion experienced thereby.
  • the method may include receiving information about one or more of rotation of an object and torque experienced by the object, isolating a first portion of a user's skin relative to a body of a shear display device, and moving a first tactor of the shear display device in a first direction along a linear path, the first tactor being in contact with the isolated first portion of the user's skin.
  • the method also may include isolating a second portion of a user's skin relative to a body of the shear display device and moving a second tactor of the shear display device in a second direction along a linear path, the second tactor being in contact with the isolated second portion of the user's skin, the second direction being opposite to the first direction.
  • embodiments may include a method for displaying information about a change in size of an object.
  • the method may include receiving information about the change in size of the object, isolating a first portion of a user's skin relative to a body of a shear display device, and moving a first tactor of the shear display device in a first direction along a linear path, the first tactor being in contact with the isolated first portion of the user's skin.
  • the method also may include isolating a second portion of a user's skin relative to a body of a shear display device and moving a second tactor of the shear display device in a second direction along a linear path, the second tactor being in contact with the isolated second portion of the user's skin, the second direction being opposite to the first direction and away from or towards the first tactor.
  • Figure 1A illustrates a perspective view of a shear display device in accordance with one implementation of the present invention
  • Figure IB illustrates an exploded perspective view of the shear display device of Figure 1 A
  • Figure 2 A illustrates a perspective view of a shear display device in accordance with another implementation of the present invention
  • Figure 2B illustrates a perspective view of an actuation mechanism in accordance with one implementation of the present invention
  • Figure 2C illustrates a bottom perspective view of an actuator assembly that incorporates the actuation mechanism of Figure 2B;
  • Figure 2D illustrates a top perspective view of an actuator assembly that incorporates the actuation mechanism of Figure 2B;
  • Figure 3 A illustrates a perspective view of an actuator assembly in accordance with one or more implementations of the present invention
  • Figure 3B illustrates a side view of the actuator assembly of Figure 3 A
  • Figure 4 illustrates a perspective view of a shear display device in accordance with yet another implementation of the present invention
  • Figure 5 illustrates a perspective view of a shear display device in accordance with still one other implementation of the present invention
  • Figure 6 illustrates a perspective view of a shear display device in accordance with one or more implementations of the present invention
  • Figure 7A illustrates a perspective view of a shear display device in accordance with yet one other implementation of the present invention
  • Figure 7B illustrates a partial cutaway perspective view of the shear display device of Figure 7A
  • Figure 8 illustrates a perspective view of a shear display device in accordance with still one other implementation of the present invention.
  • Figure 9 illustrates a perspective view of a shear display device in accordance with one or more implementations of the present invention.
  • Figure 10 illustrates a perspective view of a shear display device in accordance with an implementation of the present invention
  • Figure 11 illustrates a perspective view of a shear display device in accordance with at least one other implementation of the present invention
  • Figure 12 illustrates a perspective view of a control system that incorporates a shear display device in accordance with one implementation of the present invention.
  • Figure 13 illustrates a chart of acts of a method of displaying information via tactile cues in accordance with one implementation of the present invention.
  • Implementations of the present invention provide devices, systems, and methods for displaying information about direction of movement, speed, resistive force experienced, other aspects of movement for an object, or combinations thereof.
  • the device can also be used to display or feedback a wide variety of information that has magnitude and/or direction associated with it (e.g., temperature, pressure, volume, proximity, other information, or combinations thereof) or give guidance about where a person should move (e.g., to push forward on an aircraft control stick to prevent stall from occurring).
  • the present disclosure provides a shear display device that can generate skin shear or skin stretch with one or more tactors moving in a two- or three- dimensional space.
  • the tactors are the moving contacts between the shear display device and the user's skin, and the terms shear display, shear feedback, skin stretch, and skin stretch feedback are used interchangeably to refer to tactile cues provided via stretching one or more portions of the user' skin.
  • the movement of the tactors can represent to a user various aspects of the movement of the object (e.g., object being controlled by the user or a controlled object).
  • controlled object also includes the user's own body, limbs, arms, fingers, and hand.
  • the movement of the tactors can represent linear (or translational) direction and/or speed of the object's movement, or that a user should translate their hand or limb in two- or three-dimensional space.
  • the movement of the tactors also can represent rotation of the object in two- or three- dimensional space, or that a user should rotate their hand or limb in two- or three- dimensional space.
  • movement of the tactors can display information about force and/or torque experienced by the object. With such information, the user may direct movement of the object (or their limbs) more accurately. For example, the user can control the amount of force applied by the controlled object onto another body as well as location and/or direction of the force.
  • a controlled object can be any automated or semi- automated vehicle, tool, or other implement, movement of which can be directed by a controller.
  • Examples of controlled objects include controlled servo and stepper motors or other actuators, computer controlled machines, vehicles, robots, etc.
  • a controlled object could also include the human user, e.g., for using shear feedback to guide their limb motions during physical therapy.
  • the shear display device can be part of a controller (e.g., a joystick) that directs movement of the controlled object.
  • the controller with shear feedback can allow the user to control or direct the controlled object, while displaying information about the controlled object's movement (i.e., through skin stretch rather than gross movement of the joystick).
  • the controller with shear feedback can display movement information (i.e., through skin stretch) without affecting the movement and/or location of the user's hand such as to interfere with the operation of the controller.
  • the shear display device while the shear display device provides or (physically) displays information to the user, the shear display device does not move the user's limb, but rather only a small portion of the skin of the user's body, which may include the user's head, arm, hand, finger, other body part, or combinations thereof (for multiple tactors).
  • the shear display device also can provide a simulated force feel to the user's hand, finger, or another body part. More specifically, the skin shear produced by the shear display device can be similar to the sensation otherwise felt by the user while using an actual tool, similar to the tool that is being controlled and operated by an automated system (e.g., a robot). Hence, for example, a surgeon can receive shear feedback while controlling an automated or robotically controlled scalpel, and such shear feedback can approximate the shear that a manual scalpel would produce on the skin of the surgeon's hand. Shear feedback can also be used to provide force information or guidance cues for procedures such as catheterization, rehabilitation, laser/bone/vascular etc. surgery or resection. For example, when used for rehabilitation, shear feedback can guide a person's limb motions as part of physical therapy, etc.
  • an automated system e.g., a robot
  • a controller can detect when the user is not in contact with the tactor of the shear display device.
  • the shear display device can include a touch sensor, which can recognize contact with user's skin (or lack thereof).
  • This sensor could include, but is not limited to a capacitive sensor, electrostatic sensor, contact switch, force sensor, etc. Accordingly, the shear display device and/or the controller can respond to loss of contact with the user's skin, in a manner that would avoid displaying inaccurate information to the user.
  • the tactor can move to its default position (e.g., an origin or zero position, such as a center of a finger well) and cease all further operation, until contact is reestablished. Once contact is reestablished, the tactor can once again commence displaying information to the user. Additionally, the shear display device can alert the user about the loss of contact with the tactor as well as about reestablishment of contact (e.g., by providing an audible alert).
  • its default position e.g., an origin or zero position, such as a center of a finger well
  • the shear display device can alert the user about the loss of contact with the tactor as well as about reestablishment of contact (e.g., by providing an audible alert).
  • the tactor of a shear display can also be coupled to or have a force sensor embedded within it.
  • This force sensor can be used for the above purpose of knowing when the user's hand is in contact with the shear tactor(s).
  • the force sensor can also be used as a user input for the controller, similar in spirit as a ThinkPad laptop's TrackPoint cursor control sensor.
  • this embedded force sensor can also be used to more generally measure user interaction forces with the shear display's tactor(s).
  • These force sensors may provide a means to control translational and rotational input or motion to a device, object, robot, vehicle, or other computer controlled system.
  • a user is also able to provide translational and rotational input to the force sensors on the multiple tactors in an analogous manner. That is, if the user applies the same force in the same direction to multiple tactors, this would cause a pure translational input to the control system, and if the user applies the same force in opposite directions (i.e., a force couple) to multiple tactors, this would cause a rotational input to the control system.
  • This provides the ability to measure user input through applied forces on the shear display tactors, as opposed to just capturing the user's motion by tracking the position of the held or strapped-on shear display (e.g., by putting the shear display on a robot arm or by using some type of non-contact motion tracker). Furthermore, this scheme of capturing user input from one or more tactors with embedded force sensors is the "input" equivalent of moving the tactors together to give translational direction/motion/force cues or feedback and moving the tactors differentially to give rotational direction/motion/force etc., cues or feedback (which will be discussed further below).
  • a shear display device or device with embedded shear feedback in addition to tracking the position of the shear display by mounting it on the end of a robotic arm (with force feedback) or kinematic arm (without force feedback), it is also possible to utilize single or multiple shear displays within a shear display device or device with embedded shear feedback in combination with any type of motion tracking system to provide tactile feedback.
  • This tactile feedback can be used to provide relevant situational or control information.
  • a device with single or multiple shear displays within it could be paired with a wireless camera system (e.g.
  • Motion sensors can coordinate to the user's interaction within those environments, and/or the user's current motion.
  • Other motion sensors can also be used to provide feedback via shear display based on user interaction in the above environments, such as tilt sensors, inertial sensors, gyros, accelerometers, magnetometers, or other position or motion sensors.
  • inertial sensing and some forms of non-contact position sensing also make it so that no explicit absolute position sensing is necessary, yet this sensor information can provide information that can be used by the control system and fed back to a user via shear feedback.
  • Interaction from force sensors such as a force sensor in the user's chair or floor (e.g., Wii Fit board) can also be used to provide content to be portrayed via skin stretch feedback using a shear display.
  • force sensors on the tactors of a shear display device can be used to provide translational or rotational inputs to a control system.
  • Examples of types of information that can be portrayed along with how it can be displayed includes: impact can be displayed by rapidly moving the tactor in the direction of the impact force, the velocity of an object or the person can be displayed via shear feedback by moving the tactor with a relative position proportional to the orientation and magnitude of the object's velocity vector. Forces are also similarly mapped to tactor motions by displaying a force vector as a scaled tactor displacement vector.
  • the tilt of an object can also be portrayed via shear feedback by mapping the tilt angle or change in gravitational forces to a proportional amount of skin stretch feedback whose orientation corresponds to the direction an object is tilted (e.g., this could provide feedback as a user tilts their smart phone while playing a game).
  • Another example would be to display skin stretch feedback in a driving game whose tactor motion is proportional to the inertial forces felt by the driver.
  • the orientation of the shear feedback could also be corrected to correspond to changes in the user's orientation with respect to the frame of reference of the forces, which may be useful while conducting upper extremity rehabilitation.
  • any other quantities that have a vector and/or magnitude could also be displayed via shear feedback, e.g., life meter in a shooter game, sonar map that shows the location of enemies, which could be displayed by an outbound pulse-type motion in the direction of the target on the heads up map.
  • shear feedback e.g., life meter in a shooter game, sonar map that shows the location of enemies, which could be displayed by an outbound pulse-type motion in the direction of the target on the heads up map.
  • This application could have performance advantages as the gamer won't need to refer to the visual heads-up display window as much to monitor this information.
  • skin stretch feedback could be used to point to an open player in a sports game using a radial tactor motion in the direction towards the open player and then return to center.
  • the tactor motion could move repeatedly in an AC coupled fashion, or could simply move the tactor in the radial direction towards the open player and persistently point in the direction of interest until this information is no longer valid (before the tactor would return to center).
  • Giving AC coupled cues has the advantage in that they are repeated and a user can sometimes (physically or cognitively) miss an initial tactor motion.
  • the tactors location can also be used to provide feedback on the relative location or motion of an object within a game or virtual reality scenario.
  • force feedback shear display via skin stretch (using tactors), force detection (using tactors and force sensors), and other modes of user interaction are described. These descriptions are not meant to be limiting in any way.
  • Force feedback, shear display via skin stretch, force detection, other user interaction modes, or combinations thereof may be used to interact with the user.
  • other modes of user interaction including visual feedback, auditory feedback, haptic feedback, olfactory feedback, even gustational (i.e. taste) feedback, other sensory feedback, or combinations thereof may be used to interact with the user.
  • Haptic feedback for example, may include vibrotactile feedback.
  • other modes of receiving user feedback may be used.
  • other user feedback detection may be used.
  • motion capture including video
  • sound detection including but not limited to voice recognition
  • other user feedback detection or combinations thereof.
  • any of the user feedback modes and the user feedback detection modes may be used individually or in combination with each other.
  • using shear (or skin stretch) feedback in combination with force feedback may reduce the amount of force feedback necessary to lower (or even safe) levels. This may, in the event that there are sensing errors with force feedback alone, reduce instabilities. Thus, using force feedback with skin stretch feedback may enhance/increase the perceived force or stiffness (or other force information such as friction, damping, etc.) reducing the amount of force feedback required.
  • Providing force feedback in combination with skin stretch feedback may be more intuitive because the forces and force directions, though weak in some cases, can still be perceived and may help people more intuitively interpret the skin stretch cues, especially when moving with more spatial motions (i.e., not confined to a plane) or when judging real physical mechanical properties such as stiffness, damping, mass, friction, etc.
  • torque feedback and shear feedback to provide rotational (i.e. torque or other rational indicators)
  • less torque feedback may be necessary, due to the complementary rotational shear feedback (from differentially moving tactors) cues.
  • Other advantages are also considered.
  • shear feedback may be used with various procedures.
  • shear feedback may be used to guide hand motions. This guidance may come through directional shear feedback cues, force/torque feedback cues, other cues, or combinations thereof. In the context of catheterization with a shear feedback and/or force feedback device, these cues may be used to help a user stay within a predetermined path.
  • Figure 1A illustrates one embodiment of a shear display device 100a that has a tactor 110a, which can provide tactile cues and display information to a user.
  • the tactor 110a can be at least partially located within a body 120a.
  • the body 120a can be cylindrical; however, other shapes of the body 120a can be suitable, depending on the particular use of the shear display device 100a.
  • the body 120a can have a suitable configuration for being grasped by the user and/or to provide a reasonable means for the user to lay their hand and/or fingers on the shear display device 100a.
  • the shear display device 100a also can include a well 130a, which can isolate the user's skin with respect to the body 120 and/or near or about the tactor 110a, such that the movement of the tactor 110a can stretch the user's isolated skin.
  • the well 130a can isolate skin on the user's finger (or fingertip), such that the tactor 110a can create skin stretch thereon without physically moving the user's finger in any substantial manner.
  • the well 130a may be configured as an aperture or an orifice.
  • the well 130a can have a suitable shape, size, and configuration (e.g., position and orientation relative to the tactor 110a) to restrain the user's skin relative to the tactor 110a and may also assist in grasping the shear display device 100a.
  • a suitable shape, size, and configuration e.g., position and orientation relative to the tactor 110a
  • the tactor 110a can move in a two-dimensional space (or in three- dimensional space in some embodiments, though the discussion with respect to Figure 1 A will be mostly focused on two-dimensional space) and, when in contact with the user's skin, can cause skin stretch by such movements.
  • the tactor 110a can move along X and/or Y axes, indicated in Figure 1A.
  • the tactor 1 10a can move simultaneously along both X and Y axes.
  • the tactor 110a can have linear or nonlinear movement in any direction.
  • the skin stretch, experienced by the user can provide cues and information (e.g., directional information) to the user.
  • a linear movement of the tactor 110a can represent a linear movement of the object being controlled by the user. Such movement can be represented in any direction in a two- dimensional space.
  • the shear display device 100a can provide any number of directional cues or types of directional information.
  • the shear display device 100a can provide shear feedback that relates linear movement of an object (e.g., a controlled object, the shear feedback device 100a, or any other predetermined object or entity, whether real or virtual) with the movement of the tactor 110a.
  • an object e.g., a controlled object, the shear feedback device 100a, or any other predetermined object or entity, whether real or virtual
  • movement of the tactor 110a in a first direction can indicate to the user that an object also has moved in the first direction.
  • movement of the tactor 110a in the first direction can signal to the user that a destination is located linearly in the first direction from the user or from a controlled object.
  • movement of the tactor 110a also can indicate direction where it may be desirable for the user to move a controlled object.
  • the shear display device 100a can guide the user (or user's hand) and can provide cues for controlling the controlled object (e.g., directing the controlled object to move in a certain desirable direction).
  • the shear display device 100a can display force information.
  • movement of the tactor 110a in the first direction can indicate a force applied to the controlled object, which is acting on the first object in the first direction.
  • the shear display device 100a can indicate a relative amount of force experienced by the controlled object. For instance, relatively slow movement and/or relatively small displacement of the tactor 110a in the first direction can indicate a relatively small or insignificant force acting on the controlled object.
  • a relatively fast movement and/or relatively large displacement of the tactor 110a can indicate a relatively large force acting on the controlled object.
  • the shear display device 100a also can display rotational and/or torque information related to an object or entity (e.g., related to the controlled object).
  • Rotational information can relate to in-plane rotation of an object (e.g., a controlled object, the shear feedback device 100a, or any other predetermined object or entity, whether real or virtual).
  • the tactor 110a can move in a substantially or approximately circular or semicircular manner about a predetermined point (e.g., about an original or default position of the tactor 110a or another point).
  • the tactor 110a can move simultaneously along X and Y axes in a manner that produces a spiral, circular or semicircular movement.
  • movement of the tactor 110a can appear to the user as a circular or semicircular movement.
  • Circular and/or semicircular movement of the tactor 110a can represent rotation of the controlled object and/or torque experienced thereby. More specifically, the rotational movement of the tactor 110a can represent, for instance, the rotation of the object or the magnitude of the torque applied thereto. In one example, the tactor 110a generally moves about and within the diameter of the well. In other embodiments, the diameter of the circular or semi-circular path may be used to communicate the orientation or rotation of the object or the magnitude of the torque applied thereto. In further embodiments, other shapes or paths of the tactor 1 10a may be used to describe the motion (i.e., force or velocity) of the controlled object.
  • the tactor 110a can rotate about a point.
  • the tactor 110a may rotate about a point.
  • Such rotation also can represent rotation of an object and/or can convey information about the object's rotation and/or the torque experienced thereby.
  • the tactor 110a can have a related rotation or motion that can produce corresponding skin stretch in the user's hand, finger, or other body part.
  • a combination of the circular or semicircular movement as well as rotation of the tactor 110a may be used.
  • the tactor 110a may rotate about its own axis as it also moves in a circular or semi-circular manner about a point.
  • the shear display device 100a also can provide information through a pattern or sequence of movements.
  • the tactor 110a can have repeated movement or a series of repeated movements in a direction within the plane of the shear display.
  • Such interrupted movement provides "AC coupled” or pulsing directional information (force, direction, displacement, motion, etc.). That is, movement of the tactor 110a can be pulsed with the direction and/or magnitude of the directional information to be provided.
  • This directional information can also be done through sustained movements of the tactor, through "DC coupled" feedback.
  • the tactor 110a is moved to a position that represents the direction and/or magnitude of the directional information (force, direction, displacement, motion, etc.) and is held in this position until the tactor is moved back to its center position to indicate that that the user should stop moving, that force now zero, etc.).
  • the directional information force, direction, displacement, motion, etc.
  • the information displayed by the shear display device 100a via movements of the tactor 110a that produce skin stretch can reflect the movement of the shear display device 100a as well as the movement of the controlled object or any other predetermined object or entity.
  • direction and/or speed of movement or acceleration of the tactor 110a can represent movement of the controlled object. For example, if the controlled object moves slowly, the tactor 110a also can move correspondingly slowly (i.e., at the same or proportional rate of speed as the controlled object) and in the same direction as the controlled object, thereby signaling to the user the speed and direction of movement of the controlled object.
  • the tactor 110a could also signal the speed and direction of motion by moving the tactor 110a a first distance and in a first direction (e.g., from a center or default position of the tactor 110a).
  • the first distance and the first direction can be proportional to and may correspond with the object's velocity.
  • the shear display device 100a also can be a controller that directs movements of a controlled object (i.e., sends information necessary to move or operate such controlled object).
  • the shear display device 100a can include a mounting shaft 140a, which can couple to a corresponding control mechanism (e.g., a gimbaled sensor, a force feedback device such as a Phantom Robot Arm, etc.).
  • the control mechanism can detect movements of the shear display device 100a and can send instruction to a controller for directing the controlled object.
  • the shear display device 100a also can send control signals or instructions to a controller without being physically connected to the control mechanism.
  • the shear display device 100a can incorporate wired or wireless tracking mechanisms that can interface with or can be incorporated into the control mechanism and can detect movements and/or position (or change thereof) of the shear device 100a, which can be provided by the control mechanism as instructions for the controlled object.
  • the tactor 110a can also include a force sensor in communication therewith. Consequently, the force sensor can send directional information to the control mechanism or to the controller, which can provide corresponding instruction to the controlled object.
  • the shear display device 100a can display information along a Z-axis.
  • the shear display device 100a can provide information about three-dimensional movements, forces, torques, positions, etc.
  • the tactor 1 10a can move outward or inward (i.e., toward or into the user's skin and away from the user's skin) to display information related to movement and/or force along the Z-axis of an object.
  • movement of the tactor 110a along the Z-axis can provide information about the object, which is similar to the information described herein in connection with two-dimensional movement of the tactor 110a.
  • movement of the tactor 110a along the Z-axis can be independent of the movements along the X and/or Y axes.
  • the tactor 110a can move in any number of patterns or directions in three-dimensional space.
  • the tactor 110a can move in any one or more of the X-Y, X-Z, and Y-Z planes.
  • such movement can be along any desired path (e.g., circular path in the X-Z plane), which can indicate movement or position of as well as forces or torques experienced by the object.
  • the tactor 110a can be actuated in a number of ways.
  • the actuator assembly of the shear display device 100a can include a crank-slider mechanism 150a connected to a first motor 160a.
  • the crank-slider mechanism 150a can include a crank 170a connected to the first motor 160a and a slider 180a coupled to the tactor 110a.
  • the crank 170a moves the slider 180a, thereby producing linear motion of the tactor 110a (e.g., along Y-axis).
  • the crank mechanism 150a can produce linear motion of the tactor 110a in response to the rotation of the first motor 160a.
  • the actuator assembly of the shear display device 100a also can include a second motor 190a.
  • the second motor 190a can be located within or secured to the slider 180a. In any event, the second motor 190a can move (e.g., along the Y-axis) with the slider 180a.
  • the second motor 190a also can be coupled to the tactor 110a and can generate rotation of the tactor 110a, for example, with respect to a center axis of the slider 180a.
  • operation of the motor 160a and of the second motor 190a can be independent of each other. As such, tactor 110a can move independently in two- dimensions.
  • the range of motion of the tactor 110a can be relatively small (e.g., 0-1 mm, 0-2 mm, 0-5mm). Accordingly, radial motion of the tactor 110a produced by the second motor 190a can appear as substantially linear motion to the user (e.g., as a linear motion along the X-axis).
  • the first motor 160a may generate linear motion in the y-direction or along the Y-axis and the second motor can generate substantially linear motion in the x-direction or along the X-axis.
  • the movement produced by the second motor 190a and by the first motor 160a when combined together, can produce any number of movements or movement patterns of the tactor 110a, such as the movements and movement patterns described herein.
  • the tactor 110a can be moved in a linear manner in any direction.
  • the tactor 110a also can be moved in a nonlinear manner in any direction.
  • the tactor 110a can be moved in a circular semicircular or other nonlinear manner.
  • the crank mechanism 150a when combined together, can produce any number of movements or movement patterns of the tactor 110a.
  • the tactor 110a can be moved in a linear manner in any direction.
  • the tactor 110a also can be moved in a nonlinear manner in any direction.
  • the tactor 110a can be moved in a circular semicircular or other nonlinear manner.
  • shear display device 100b that has a flat or box-like (e.g., rectangular) body 120b.
  • the shear display device 100b and its components and elements can be similar to or the same as the shear display device 100a ( Figures 1A-1B) and its respective components and elements.
  • the shear display device 100a can include the tactor 110b located within a well 130b, which can at least partially constrain user's skin in contact therewith.
  • the tactor 1 10b can display the same or similar information as can be displayed by the tactor 110a of the shear display device 110a ( Figures 1 A- IB).
  • the tactor 110b can represent such information by the same or similar movements and movement sequences as described herein in connection with the shear display device 110a ( Figures 1A-1B).
  • the rectangular body 120b of the shear display device 100b may present a user with a more convenient or ergonomic interface than, for instance, the shear display device that has a cylindrical configuration.
  • the well 130b and the tactor 110b can be located essentially anywhere on the body 120b.
  • the well 130b and the tactor 110b can be located near one or more edges of the body 120b (e.g., near a corner of the body 120).
  • the well 130b and the tactor 110b also can be located away from one or more edges of the body 120 (e.g., near a center point of the body 120), as may be desirable for a particular application.
  • an actuator assembly of the shear display device 100b can include one or more actuator assemblies, which can comprise a cam actuation mechanism 150b. More specifically, the cam actuation mechanism 150b can include first and second motors 160b, 190b which can move or rotate respective first and second cams 170b, 180b.
  • first and second cams 170b, 180b can result in movement of the tactor 110b relative to the body 120b.
  • first and second cams 170b, 180b are shown as eccentric circular pins that move within a slot to provide relative motion.
  • the first motor 160b can be coupled to the first cam
  • the first motor 160b can have a worm 162b coupled to a shaft thereof, which engages a worm gear.
  • the worm 162b can be engaged with a worm/spur gear 172b of a first diameter.
  • the connection between the worm 162b and the worm/spur gear 172b can be a reducer and can provide mechanical advantage (i.e., can produce higher torque at the rotation of the first gear 172b than produced by the first motor 160b).
  • one rotation of the worm gear 162b can produce less than one rotation of the spur gear 172b.
  • the significant mechanical advantage that may be provided by a worm gear can reduce the required space and also may reduce the meshing velocity of spur gears at the next stage of the transmission.
  • the reduced meshing velocity of the spur gears can greatly reduce the noise produced by a geared transmission, and the meshing of worm gears in the first stage of the transmission is inherently quieter than the meshing of spur gears.
  • Use of helical gears rather than standard spur gears can further reduce the transmission noise.
  • the spur gear 172b can be coupled to or operatively connected with the first cam 170b, as described further below.
  • the spur gear 172b can be coupled to a spur gear 174b of a second diameter (e.g., the spur gears 172b, 174b can be coupled together and may rotate together about a shaft).
  • the spur gear 174b can be engaged with a spur gear 176b, which can be coupled with the first cam 170b.
  • the spur gears 172b and 176b can have substantially the same diameter, while the spur gear 174b can have a smaller diameter. Accordingly, connection between the spur gears 172b, 174b, 176b also can be a reducer and can provide mechanical advantage.
  • connection between the worm 162b and the worm/spur gear 172b and/or by the connection between the spur gears 172b, 174b, 176b can transfer more force to the movement of the tactor. Additionally, such connection can reduce the angle of rotation of the first cam 170b relative to the rotation of the first motor 160b. Consequently, such connection also can provide additional control and may enhance precision or accuracy of positioning and/or moving the tactor.
  • the second motor 190b can be coupled to the second cam 180b in a similar manner, as the first motor 160b may be coupled to the first cam 170b, as described herein.
  • first and second motors 160b, 190b can have any number of suitable connections or coupling configurations with the respective first and second cams 170b, 180b.
  • Such connections can include direct or direct drive connections, crank-slider connections, belt-pulley connection, chain-sprocket connections, etc.
  • the first and second motors 160b, 190b can rotate respective first and second cams 170b, 180b, which can produce motion of the tactor relative to the body of the shear display device.
  • the cam 150b ( Figure 2B) can be housed in a sliding housing 200b. More specifically, the cam actuation mechanism can be secured to and/or within the sliding housing 200b.
  • the first cam 170b can be slidably and/or rotatably secured to or within the body of the shear display device (e.g., the first cam 170b can be secured within a slot in the body). Accordingly, in response to clockwise rotation of the first cam 170b, the sliding housing 200b can be pushed in a first direction along an X-axis, while in response to counterclockwise rotation of the first cam 170b, the sliding housing 200b can be pushed in a second, opposite direction along the X-axis.
  • the sliding housing 200b can have grooves 210b, which can guide the sliding housing 200b along the X-axis.
  • the sliding housing 200b can move along the X-axis in response to rotation of the first motor.
  • the tactor can be secured or coupled to the sliding housing 200b. Consequently, movement of the sliding housing 200b can result in the corresponding movement of the tactor relative to the body of the shear display device.
  • first cam 170b and the second cam 180b (Figure 2B) can be configured such as to provide the greatest mechanical advantage at the farthest point of travel of the sliding housing 200b (and of the upper slide 220b ( Figure 2B), respectively). Accordingly, as the tactor moves and the user's skin stretches, the tactor can experience resistance due to the stretch of the user's skin, which can increase as the tactor moves away from a default position and can peak at the farthest position of travel. Such increase in resistance can be at least in part accommodated by correspondingly increasing mechanical advantage provided by the first cam 170b (and the second cam 180b ( Figure 2B)). [0076] Also, as the first cam 170b (or the second cam 180b ( Figure 2B)) rotates
  • the sliding housing (and/or the upper slide 220b ( Figure 2B)) can move in a first direction and/or in an opposite direction.
  • the first cam 170b (and/or the second cam 180b ( Figure 2B)) as well as corresponding slot or receiving channel in the body of the shear device can be configured such that the first cam 170b (and/or the second cam 180b ( Figure 2B)) can fully rotate therein.
  • the sliding housing 200b (or the upper slide 220b ( Figure 2B), as applicable) can continuously oscillate, without exceeding travel limits of the shear display device and/or damaging components or elements thereof.
  • an upper slide 220b can be slidably coupled to the sliding housing 200b.
  • the second cam can move the upper slide 220b relative to the sliding housing 200b as well as relative to the body of the shear display device. More specifically, as illustrated in Figure 2D, the second cam 180b can be rotatably secured within a slot 230b and the upper slide 220b. Hence, as the second cam 180b rotates about an axis thereof, the second cam 180b can push or pull the upper slide 220b along the Y-axis.
  • the sliding housing 200b can include guiding channels 240b (e.g., the guiding channels 240b can have a gib-like configuration).
  • the upper slide 220b can include guiding protrusions 250b that can fit into the guiding channel 240b. Accordingly, the upper slide 220b can move linearly along the Y-axis relative to the sliding housing 200b as well as relative to the body of the shear display device. Additionally, when the sliding housing 200b moves along the X-axis, relative to the body of the shear display device, the upper slide 220b can move together with the sliding housing 200b.
  • the tactor 110b can be secured to the upper slide 220b. Consequently, when the upper slide 220b moves along the Y-axis, in response to rotation of the second motor, the tactor 110b also can move along the Y-axis. Likewise, when the sliding housing 200b moves along the X-axis, in response to rotation of the first motor, the tactor 110b can move along the X-axis.
  • rotation of the first and second motors can actuate movement of the tactor 110b along the X and Y axes.
  • the first and second motors can move the tactor 110b in any number of paths and/or sequences or patterns.
  • the first and/or second motors can be a servo motors connected to and controlled by a controller, which can provide instructions to the first and second motors to move the tactor 110b in a manner that displays information to the user, as described herein.
  • FIG. 3A-3B one embodiment of an actuation mechanism or an actuator assembly 260, which can actuate a tactor 110 in a manner that can provide one-, two-, or three-dimensional information to the user through movement of the tactor 110 in a plane substantially orthogonal with respect to the user's skin described herein.
  • the actuator assembly 260 can be incorporated into any one of the shear display devices described herein, including the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, 100k, 100 ⁇ ( Figures 1A-2B, 4-11) irrespective of the particular shape of their respective bodies.
  • the illustrated embodiment of the actuator assembly 260 incorporates a cylindrical or a point tactor 110, it should be noted that the actuator assembly 260 can move any one of the tactors described herein, irrespective of the size and/or shape thereof.
  • any embodiment herein may be combined to provide combinations of circular/semi-circular movement, rotation, planar movement, other tactor movement or combinations thereof with movement or cues provided in the z direction as well as gross movement when used with a device such as a force feedback device.
  • the actuator assembly 260 can incorporate a cam 270 and a motor 280 that can rotate the cam 270, thereby causing the tactor 110 to move outward (i.e., toward the user's skin along the Z-axis).
  • a counter clockwise rotation of the cam 270 can cause the tactor 110 to move outward.
  • the tactor 110 can be returned to its original position by rotating the cam 270 in the opposite direction (e.g., counterclockwise), such as to lower the tactor 110.
  • the tactor 110 may be coupled with a sliding shim and a spring (e.g., a conical spring) 290.
  • the spring 290 may keep the tactor 110 generally in contact with the cam 270 when the cam 270 moves from its largest diameter toward its smallest diameter.
  • the cam 270 and the tactor 110 can be connected (e.g., via a T-slot connection). Accordingly, as the cam 270 rotates (e. g., clockwise) to lower the tactor 110, the cam 270 can pull the tactor 110 downward.
  • the tactor 110 can be spring-loaded, such that rotation of the cam 270, which is uncoupled from the tactor 110, may allow the spring to lower the tactor 110.
  • the tactor 110 may be operatively associated with actuator assemblies that can move the tactor 110 in a plane substantially parallel with the user's skin (e.g., in the X- Y plane). Accordingly, the tactor 110 can move in three-dimensional space-i.e., in a plane parallel to the user's skin as well as in a plane perpendicular to the user's skin.
  • the actuator assembly 260 can include first and second motors 160k, 190k, which can move the tactor 110 along respective X and Y axes.
  • the tactor 110 can be coupled to the first motor 160k via a first crank-slider mechanism 170k and to the second motor 190k via a second crank slider mechanism 190k.
  • the first motor 160k rotates the crank of the first crank-slider mechanism 170k
  • the first slider can move the tactor 110 along the X-axis.
  • the second motor 180k rotates the crank of the second crank-slider mechanism 190k
  • the second slider can move the tactor 110 along the Y-axis.
  • the tactor 110 may be operatively associated with any one of the above-described mechanisms, which can move the tactor 110 along the X and/or Y axes.
  • the shear display device can include a set of wedges. As the wedges are moved toward one another, the first wedge slides onto the second, thereby raising the tactor. Conversely, as the wedges move away from each other, the tactor can be lowered. Similar to the cam 270 ( Figures 3A-3B), the first wedge can be coupled to the tactor, thereby pulling the tactor as the first wedge slides down along the second wedge.
  • the tactor can be spring-loaded; hence, the spring can force the tactor downward as the first wedge slides down the second wedge.
  • relational information can be displayed to the user by providing changes in pressure on the user's skin.
  • the change (increase/decrease) in pressure can correspond to a correlating change in upward/downward direction of movement of the controlled object.
  • the increase/decrease in pressure can correspond with increase/decrease in force experienced by the controlled object from its environment.
  • the shear display device can convey a sensation of increased upward pressure by reducing the area of the tactor that contacts the user's skin.
  • the shape of the tactor can be changed, thereby reducing the area of the tactor that contacts the user's skin.
  • the tactor can have a substantially curved or spherical outer surface. By reducing the radius (or contact area) of the curved or spherical surface (i.e., outer surface) that defines the tactor, the user can experience greater pressure applied to the portion of the skin that contacts the tactor. Conversely, by increasing the radius of the sphere defining the tactor, a greater area will contact the user's skin, thereby decreasing the pressure felt by the user.
  • the tactor includes a flexible outer shell that is connected to at least two connectors (e.g., two tendons).
  • the tendons may be connected to a shortening device.
  • the shortening device can be a pulley coupled to an actuator. As the pulley reduces the length of the tendons, the flexible outer shell is reduced in radius by the tactor reducing the contact area of the outer surface.
  • the tactor may include an inflatable balloon or membrane, which can be inflated and deflated, thereby changing the area of the tactor and varying the pressure sensed by the user.
  • the tactor can comprise a domed shell actuated by piezo-electrical elements.
  • the shape of the shell or membrane of the tactor can change in a manner that may provide the user with a sensation of increased or decreased pressure on the user's skin.
  • the shear display device can incorporate a tactor comprising multiple concentric rings.
  • Such concentric rings can move up or down, thereby increasing and decreasing the area that is in contact with user's skin. Accordingly, whether by moving the tactor outward/inward (toward and away from the user's skin) and/or by decreasing/increasing the area of the tactor, the user can sense a change in pressure on the skin that is in contact with the tactor.
  • the tactor may be in a base configuration where the tactor is z-axis neutral. In some embodiments, the concentric rings move outward in the z-direction.
  • the concentric rings can cooperate to form a contoured surface.
  • a contoured surface may create a sense of surface curvature or changing contact area, which may be used to portray differing forces, direction cues, pressure, etc.
  • the concentric rings also can cooperate to provide the same outward movement in the z- direction, but only the centermost concentric ring remains in the outward position (the other rings move away from the finger pad). This provides a smaller surface area of the tactor which can be used to create a sense of surface curvature or changing contact area which may also be used to create a sense of surface curvature or changing contact area, which may be used to portray differing forces, direction cues, pressure, etc.
  • the shear display device can include multiple tactors, which can be located in the same or in one or more different planes.
  • a shear display device 100c can incorporate the first tactor 110c' as well as a second tactor 110c".
  • the shear display device 100c and its components and elements can be similar to or the same as any one of the shear display devices 100a, 100b ( Figures 1A-2D) and their respective components and elements (e.g., actuator assemblies). More specifically, the second tactor 110c"can be positioned substantially orthogonally relative to the first tactor 110c'.
  • the control system can incorporate the shear display devices that have multiple tactors.
  • a control system can include the force feedback device and the shear display device 100c connected to the force feedback device.
  • a shear display device lOOd can include multiple opposing tactors, such as a first tactor l lOd' and a second tactor l lOd". Except as otherwise described herein, the shear display device lOOd and its components and elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c ( Figures 1A-2A and4) and their respective components and elements (e.g., actuator assemblies). More specifically, the first and second tactors l lOd', l lOd" can be directly opposite each other.
  • the tactors l lOd', l lOd" are opposite to each other and centered approximately about the same axis (e.g., about Z axis).
  • Such a configuration of the shear display device lOOd can allow the shear display device lOOd to display various movement information to the user, which can be presented in a more intuitive manner.
  • the user can perceive relative movement of the first and second tactors l lOd', l lOd", which can provide various information to the user.
  • moving the first tactor l lOd' and the second tactor l lOd" in opposite directions within their respective X-Y planes can inform the user about rotational motion of the object (e.g., controlled object, shear display device 1 lOd, etc.).
  • movement of the first tactor 110d' and the second tactor 1 lOd" in opposite directions along the Y axis can display rotational motion about the X axis.
  • first and second tactors l lOd', l lOd are spaced apart along the X-axis, the user can experience a torque-like sensation, produced by the relative movement of the first and second tactors 110d', 1 lOd" in opposite directions.
  • the first and second tactors 110d', 1 lOd" may move in opposite directions in three-dimensional motion.
  • first tactor l lOd' may move in a first direction along the Y axis and the second tactor 1 lOd" may move in a second, opposite direction along the Y-axis, while the first tactor l lOd' and the second tactor l lOd” both move inward (i.e., while the first tactor l lOd' moves along the Z-axis, away from the user's skin and the second tactor 1 lOd" moves along the Z-axis away from the user's skin) in an arc-like motion.
  • the first and second tactors l lOd', l lOd" of the shear display device lOOd can move in opposite directions to display rotation about another axis (e.g., about an axis that is concentric with the shear display device lOOd).
  • the first tactor l lOd' and the second tactor l lOd" can move in opposite directions the X axis, thereby displaying rotational motion of the shear display device l lOd and/or of the object, such as the control object.
  • rotational motions can be display to the user by moving the first and second tactors 110d', 1 lOd" in a similar manner (i.e., by relative motion of the first and second tactors 1 lOd, 1 lOd").
  • the shear display device lOOd can provide the user with information about rotation of an object about X and/or Y axes. For example, counterclockwise rotation of the shear display device l lOd about the Y-axis can be displayed to the user by moving the first tactor 100d' in a first direction along the X-axis, while moving the second tactor l lOd" in a second, opposite direction along the X-axis. Such movement of the first and second tactors l lOd', l lOd" can create a sensation of torque and can convey relevant rotational information to the user.
  • the first and second tactors l lOd', l lOd" of the shear display device lOOd can move in the same or similar manner as the tactors of the shear display devices 100a, 100b, 100c ( Figures 1A-3). Accordingly, the shear display device lOOd also can convey the same information as a single tactor shear display device. For instance, to display rotation about the Z axis, the first and/or second tactors l lOd', l lOd" can move in a spiral or circular path and/or can rotate about various axes, as described herein.
  • first tactor l lOd' can be moved in a clockwise spiral or circular motion about the center or another point in the well.
  • the opposing, second tactor l lOd" can move in an opposite direction, namely counterclockwise relative to a view looking at that tactor (to indicate the same motion) about the same or another point.
  • circular and/or rotational movements of the first and second tactors l lOd', l lOd” can be synchronized, such as to display the same rotation on both sides of the shear display device l lOd.
  • the first tactor l lOd' and the second tactor l lOd" may both rotate and/or move clockwise, counterclockwise, or otherwise simultaneously.
  • the shear display device l lOd also can display translational or linear motion of the shear display device l lOd and/or of the controlled object or user's hand.
  • both the first tactor l lOd' and the second tactor l lOd" can move in the same direction and at the same speed or acceleration to indicate translational motion.
  • both the first tactor 110d' and the second tactor 1 lOd" can move toward the user (i.e., upward on the page) indicating corresponding linear movement of the controlled object.
  • both the first tactor l lOd' and the second tactor l lOd" can move to toward the user (i.e., toward the left of the page) indicating corresponding linear movement of the controlled object.
  • both the first tactor l lOd' and the second tactor l lOd" can move to the user's left, thereby displaying corresponding movement of the controlled object.
  • both the first tactor 110d' and the second tactor l lOd" can move to the user's right, thereby displaying corresponding movement of the controlled object.
  • first and second tactors l lOd', l lOd may be coupled (hard- or soft- coupled, i.e., physically or via a controller) such that the first and second tactors l lOd', l lOd" move together.
  • first and second tactors l lOd', 110d" may both move in the same direction (i.e., to the right, but with the first tactor l lOd' moving outward in the z direction and the second tactor 1 lOd" moving inward in the z-direction).
  • the shear display device lOOd can include any number of suitable actuator assemblies and/or actuator mechanisms.
  • the first and second tactors l lOd', l lOd" can move in opposite directions along the Z-axis in a synchronized and/or in an independent manner. Such movement, for example, can convey force or pressure experienced by the controlled object.
  • location and movement of the first and second tactors 110d', 1 lOd" can indicate to the user the magnitude of force or pressure experienced by and/or the location or movement or size of an object.
  • the actuator assembly of the shear display device lOOd can include a sliding wedge (or multiple sliding wedges).
  • the shear display device lOOd may include a wedge actuator that moves the wedges with respect to each other.
  • the wedge actuator may move a left wedge away from the wedge actuator.
  • a right wedge may stay in its relative longitudinal location.
  • the right wedge also may be actuated by the wedge actuator or another actuator in the opposite direction than the left wedge.
  • both first and second tactors l lOd', HOd" can move outward (e.g., the first tactor l lOd' moves in a first direction and the second tactor 1 lOd" moves in a second, opposite direction).
  • first and second tactors l lOd', l lOd may be operatively associated with actuator mechanisms and/or actuator assemblies that can move the first and second tactors l lOd', l lOd" in a plane substantially parallel with the user's skin (e.g., actuation mechanism 150b ( Figures 2B-2D)). Accordingly, the first and second tactors 110d', 1 lOd” can move in three-dimensional space— i.e., in a plane parallel to the user's skin as well as in a plane perpendicular to the user's skin.
  • first and second tactors 110d', 1 lOd" may be coupled together, but may be able to move both in the same direction and in opposite directions along the Z-Axis. Any number of mechanisms may be used that can provide coupled motion in both the opposite and the same directions.
  • the shear display device lOOd may use a tapered eccentric cam that is both eccentric about and tapered along its longitudinal axis.
  • the shear display device lOOd may include an actuation mechanism (not shown) that may both rotate and longitudinally move the tapered eccentric cam.
  • both the first and second tactors l lOd', l lOd can move outward along the Z-axis.
  • both the first and second tactors l lOd', l lOd can move inward along the Z-axis.
  • the first and second tactors 110d', 1 lOd can both move in the same direction. In other words, the first tactor 110d' can move outward and the second tactor 1 lOd” can move inward.
  • the first and second tactors 110d', 1 lOd may move in the same direction (i.e., outward or inward) but at a different rate and/or a different amount.
  • the tapered eccentric cam is rotated to a neutral position (e.g., where each of the first and second tactors l lOd', l lOd" may be the same distance from the center of the tapered eccentric cam) the tactor to which the larger diameter portion of the tapered eccentric cam is approaching will move farther outward as the tapered eccentric cam is longitudinally advanced and rotated such that the larger diameter portion of the tapered eccentric cam approaches that tactor.
  • first and second tactors 110d', 1 lOd may be operatively associated with actuator assemblies that can move the tactors in a plane substantially parallel with the user's skin. Accordingly, the tactors can move in three- dimensional space— i.e., in a plane parallel to the user's skin as well as in a plane perpendicular to the user's skin.
  • the shear display device lOOd also can display information about force experienced by an object about the Z-axis.
  • the first and second tactors 110d', 1 lOd" can move outward, thereby displaying to the user increased pressure along the Z-axis.
  • the first and second tactors 110d', 1 lOd” can move inward, which may signal to the user a decrease in force experienced by an object.
  • outward movement of the first and second tactors l lOd', l lOd” may signal to the user growing, zooming, or size increase of an object.
  • first and second tactors l lOd', l lOd can signal to the user shrinking, zooming out, or size decrease of an object.
  • shape of the body of the shear display device can vary from one embodiment to another. For instance, as illustrated in Figure 6, a flat or rectangular shear display device lOOe also can have opposing first and second tactors 110e', 1 lOe". Except as otherwise described herein, the shear display device lOOe and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd ( Figures 1A-5) and their respective components and elements.
  • first and second tactors l lOe', HOe" can exhibit relative motion (e.g., similar to the tactor movements described in connection with the shear display device lOOd ( Figure 5)) to display rotation of an object about X and/or Y axes.
  • the shear display device also can provide the same or similar information as any one of the shear display devices 100a, 100b, 100c, lOOd ( Figures 1A- 4).
  • the first and/or second tactors 1 10e', 1 lOe" can move in a linear manner to indicate linear directional or force cues.
  • the first and/or second tactors 110e', 1 lOe can move along spiral, circular or semicircular paths to provide rotational information about the Z axis.
  • the shear display device lOOe can provide the same information as can be provided by a single tactor shear display device as well as additional information that can be presented by differential relative movement of the first and second tactors 110e', 1 lOe.
  • the shear display device lOOe can be coupled to or integrated with a tool or an instrument.
  • the shear display device lOOe can be coupled to a stylus, which can be used to send instructions to a controller.
  • the shear display device lOOe can be coupled to a controller and/or force feedback device.
  • the shear display device lOOe, together with the controller and/or force feedback device can be used in robotic procedures (e.g., surgery, catheter insertion, etc.).
  • the body of the shear display device can have any number of suitable configurations, shapes, and sizes. Furthermore, among other things, locations of the tactors and wells on the body of the shear display device can vary from one embodiment to another. In addition, the shape and size of the tactors can vary from one embodiment to the next and may depend on location of the tactor on the shear display device, portion of the skin intended to be contacted by the tactors, etc. For instance, as illustrated in Figure 7A, a shear display device lOOf can have a first and second tactors 11 Of, 11 Of located on a joystick- like body 120f.
  • shear display device lOOf and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe ( Figures 1A- 6) and their respective components and elements.
  • the shear display device lOOf can have the first tactor
  • the first tactor 11 Of can be surrounded by a first well 13 Of, which can restrain the skin on the user's thumb, in a manner that isolates skin movement and produces skin stretch in response to the movement of the first tactor 11 Of. Moreover, the first tactor
  • I I Of can be offset from a center axis of the body 120f.
  • a right-handed shear display device lOOf can have the first tactor 1 lOf offset to the left of the center axis of the body 120f.
  • the location of the first tactor 1 lOf can allow the user's thumb to remain in a more natural position.
  • the user's thumb can remain in a relaxed or un-flexed state when in contact with the first tactor 1 lOf .
  • Such a configuration can provide better isolation of the user's skin relative to the first tactor 1 lOf , which can lead to more accurate and/or sensitive feedback for the user.
  • such configuration also can provide improved angular accuracy of user's interpretation of the skin stretch cues from the first tactor 11 Of (as compared with the configuration where the tactor is aligned with the center axis of the body).
  • the first tactor 1 lOf can be offset to the right of the center axis of the body 120f, such that the user's left thumb can remain in a relaxed state while in contact with the first tactor 1 lOf .
  • other embodiments can include the first tactor 11 Of located at other suitable locations on the upper end of the shear display device lOOf (e.g., aligned with the center axis of the body 120f).
  • the first tactor 11 Of can be located on the body 120f such that the user is capable of placing a left or a right thumb on the first tactor 1 lOf (and in the well 130f) while holding the body 120f of the shear display device lOOf with the corresponding hand.
  • the first tactor 11 Of can be capable of movements similar to or the same as the movements of any one of the tactors described herein. Hence, the first tactor 1 lOf can display the same or similar information as any one of the tactors mentioned herein.
  • the shear display device lOOf can include the second tactor 11 Of".
  • the second tactor 11 Of can move substantially along the length of the body 120f.
  • the tactor 11 Of can be in contact with the user's palm and can provide any number of tactile cues, represented by a substantially vertical movement of the second tactor 1 lO '.
  • the second tactor 1 lO ' can have a plate- or bar-like shape. As such, the second tactor 1 lO ' can engage a larger portion of the user's skin.
  • Such a configuration can be particularly advantageous on portions of the user's skin that have a relatively low density of mechanoreceptors. For example, skin on the user's palm may have a lower density of mechanoreceptors or nerve endings than the skin on the user's fingertips.
  • the second tactor 1 lO ' can recruit or engage substantially the same or similar number of mechanoreceptors on the user's palm as recruited or engaged by the first tactor 1 lOf on the user's fingertip.
  • Such a balanced engagement can allow the shear display device lOOf to provide tactile cues to the user's fingertip(s), fmgerpads, and palm, which appear with similar or substantially the same intensity to the user.
  • the second tactor 11 Of can be sufficiently larger than the first tactor 11 Of, to allow the second tactor 11 Of to contact sufficiently larger portion of the user's palm.
  • the second tactor 11 Of can be approximately a .75" x 1.5" rectangle (i.e., can have an approximate area of 1.125 in ), while the first tactor 11 Of can have an approximately 0.25" diameter (i.e., can have an approximate area of 0.05 in ).
  • the second tactor 11 Of may have a substantially rectangular shape with sides greater than 1.5" and/or smaller than .75".
  • the second tactor 11 Of can have any number of other shapes, which may vary from one embodiment to the next (e.g., square, triangular, trapezoid, irregular, etc.).
  • the greater area of the second tactor 11 Of, as compared with the area of the first tactor 11 Of, can allow the second tactor 11 Of to recruit sufficient number of mechanoreceptors to compensate for lower density of the mechanoreceptors on the portion of the skin in contact with the second tactor 1 lOf .
  • the second tactor 11 Of can include ridges or texture, which can create sufficient friction between the user's palm and the second tactor 11 Of while also increasing the experienced sensations.
  • the second tactor 11 Of can comprise a material that has sufficient coefficient of friction to prevent or reduce slippage between the tactor 1 lOf and the user's palm.
  • the second tactor 11 Of can comprise rubber, neoprene, silicone, and the like. In any event, the second tactor 11 Of can stretch the user's skin, thereby activating mechanoreceptors thereon.
  • portions of the body 120f surrounding or adjacent to the second tactor 1 lO ' can at least partially restrain the user's skin surrounding the skin that is in contact with the second tactor 1 lO '.
  • the portions of the body 120f that surround the second tactor 11 Of can comprise material that exhibits a relatively high coefficient of friction (e.g., roughened surface) or may be tacky, such as to prevent or limit movement of the skin adjacent to the portion of the user's skin that is in contact with the second tactor 11 Of .
  • the skin in contact with the second tactor 11 Of can be at least partially isolated in a manner that allow the second tactor 11 Of to stretch the skin.
  • the shear display device lOOf can produce skin stretch without incorporating a well into the body 120f thereof.
  • the act of grasping the body 120f can sufficiently isolate the user's skin to allow the tactor 11 Of to stretch an isolated portion of the user's skin.
  • the body 120f can incorporate a well about the tactor 1 lOf , and such well also can isolate the user's skin.
  • the first and/or second tactors 1 lO , 1 lOf can be actuated in any number of ways, which can include any one or more of the actuation mechanisms and actuator assemblies described herein.
  • the second tactor 11 Of can be coupled to a motor 160f, as illustrated in Figure 6B.
  • a resilient flexible spine 170f can be coupled to a crank 180f secured to the shaft of the motor 160f.
  • the crank 180f can push the spine 170f upward, thereby moving the second tactor 11 Of upward.
  • the crank 180f can pull the spine 170f downward, thereby moving the second tactor 1 lOf downward.
  • the body 120f can have a channel 190f, which can restrain and/or guide the spine 170f therein.
  • the body 120f can have a curvilinear configuration, which may provide an ergonomic fit with the user's hand.
  • the channel 190f also can have a curvilinear configuration, as the path of the channel 190f may generally follow the outside geometry of the body 120f.
  • the spine 170f can have sufficient flexibility to move up and down within the channel 190f, thereby moving the second tactor 11 Of in corresponding directions.
  • the tactor 1 lOf can move in any number of directions and patterns, linear and nonlinear, as described herein in connection with other tactors.
  • the shear display device lOOf can allow the user to "power grip" the body 120f, while the second tactor 11 Of may continue moving and transmitting tactile information to the user (i.e., stretching user's skin).
  • the user's hand can apply a relatively large amount of compressive force onto the body as well as onto the second tactor 11 Of without impeding or interfering with the operation of the second tactor 11 Of and of the shear display device lOOf.
  • the shear display device lOOf can continue providing tactile cues to the user when the user applies relatively large force onto the body 120f.
  • the shear display device lOOf can be incorporated into a control system, as described herein in further detail.
  • the shear display device lOOf can include a connecting portion 200f, which can couple the shear display device lOOf to a controller and/or force feedback device. Accordingly, the user can receive tactile cues from the first and second tactors 11 Of, 11 Of and also can receive force feedback.
  • the shear display device can include multiple large tactors.
  • large tactors can provide enhanced sensation to the user when in contact with the user's skin that has a relatively low density of mechanoreceptors.
  • a shear display device lOOg can incorporate two plate-like tactors HOg', HOg".
  • the shear display device lOOf and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf ( Figures 1A-7B) and their respective components and elements.
  • a body 120g of the shear display device lOOg can have any number of suitable configurations.
  • the body 120g can be substantially rectangular or bar-like.
  • Other embodiments can include a cylindrical, spherical, or any other number of configurations for the body 120g.
  • the tactors HOg', HOg" can be located essentially anywhere on the body 120g, as may be more suitable or desirable for a particular application.
  • the tactors 110g', 1 lOg" are located near one edge of the body 120g.
  • the tactors 110g', 1 lOg" can be similar to or the same as the second tactor
  • the tactors 110g', 110g" can exhibit that same or similar movements as any one of the tactors described herein and may be actuated in the same or similar manner as any one of such tactors (i.e., the shear display device lOOg can incorporate any one of the actuator assemblies described herein). Accordingly, the tactors l lOg', HOg" can move up and down along one edge of the body 120g (along Y axis). Furthermore, the tactors HOg', HOg” can move toward and away from the edge of the body 120g (along X Axis). Hence, the tactors HOg', HOg” can exhibit any number of movement path, patterns, and sequences described herein.
  • the tactors HOg', HOg" can display the same or similar information as described above in connection with multiple opposing tactors. To mention a few: relative movement of the tactors HOg', HOg" can display rotational information such as rotary motions and torques; likewise, the tactors HOg', HOg” can display linear motion and/or force information.
  • the shear display device lOOg can allow the user to apply sufficiently large grasping force onto the body 120g.
  • the tactors HOg', HOg" can continue moving relative to the body 120g and displaying tactile information to the user. Consequently, such configuration can be employed in applications where the body 120g of the shear display device lOOg may be subjected to relatively large grasping forces from the user.
  • the shear display device lOOg can be incorporated into controller used in virtual games of tennis, baseball, golf, sword fighting, etc.
  • the shear display device lOOg can be incorporate into any number of controller or devices, which require or allow the user to apply relatively large grasping force onto the body 120g of the shear display device lOOg.
  • the tactors HOg', HOg" can provide sufficient stimulation of user's mechanoreceptors at locations of low or lower sensitivity (e.g., lower than fingertips).
  • the shear display device lOOf can also be used in applications where user's gripping force may not be relatively large and may provide skin stretch cues to portion of the skin with lower sensitivity (e.g., user's palm).
  • the body of the shear display device can have any number of configurations.
  • a shear display device may incorporate any number of tactors (one, two, three, four, etc.), which may be positioned and/or oriented on the body of the shear display device in any number of suitable configurations.
  • a shear display device lOOh can have a joystick- like body 120f and may include first, second, and third tactors 110h', HOh", HOj.
  • the shear display device lOOh and its components or elements can be similar to or the same as any of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg ( Figures 1A-8).
  • the joystick-like body 120h may be similar to or the same as the joystick-like body 120f ( Figures 7A-7B).
  • the first and second tactors 11 Oh', 11 Oh" may move upward and downward along a length of the body 120h. Such movement may approximately follow the lengthwise curvature or surface of the body 120h. In additional or alternative embodiments, the first and second tactors 11 Oh', 11 Oh” may move approximately perpendicular to the length of the body 120h. In still further embodiments, the first and second tactors 110h', 1 lOh" may move about the body 120h in a manner that the path of the first and second tactors 11 Oh', 11 Oh" approximately follows the curvature of the perimeter or surface of the body 120f.
  • the third tactor 1 lOj can move in a similar or the same manner and can provide similar or the same tactile cues as the first tactor 1 lOf ( Figures 7A-7B).
  • the first and second tactors 11 Oh', 11 Oh” can move in a similar or the same manner and can provide similar or the same tactile cues as the first and second tactors HOg', HOg".
  • the shear display device lOOh can incorporate cues provided by the shear display device lOOf, lOOg ( Figures 7A-8) in a single device.
  • the shear display device lOOh can display torque or rotation experienced by an object by moving the first and second tactors 110h', 11 Oh" in opposite directions (e.g., along the length of the body 120, about the perimeter of the body 120, etc.).
  • linear movement or force experienced by an object can be displayed by the shear display device lOOh as movement of the first and second tactors 110h', 1 lOh" in the same direction.
  • the shear display device lOOh can be used in any number of applications.
  • the shear display device lOOh may be used as an element or component of a control stick of an airplane (e.g., the body 120h of the shear display device can operate as the control stick).
  • the first and second tactors 11 Oh', 11 Oh" may provide instruction to a pilot (e.g., to a student pilot) regarding how to move the control stick and/or navigate the airplane.
  • the first tactor 11 Oh' can move downward along the body 120h, while the second tactor 1 lOh" moves upward.
  • the shear display device can include four tactors.
  • a shear display device 100k can include a partially cylindrical body 120k and four tactors positioned about the body 120k. Except as otherwise described herein, the shear display device 100k and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh ( Figures 1A-9).
  • the shear display device can include first and second tactors 110k', 110k" located along the X-axis of the shear display device 100k and opposite to one another.
  • the shear display device can include third and fourth tactors 110m', 110m" positioned along the Y-axis of the shear display device and opposite to each other.
  • the first and second tactors 110k', 110k" may have an approximately orthogonal orientation relative to the third and fourth tactors 110m', 110m".
  • the first and second tactors 110k', 110k” can move along the length of the body 120k (i.e., in a direction along or parallel the Z-axis of the shear display device 100k).
  • the third and fourth tactors 110m', 110m" can move along the length of the body 120k. In at least one embodiment, such movement of the first, second, third, and fourth tactors 110k', 110k", 110m', 110m” can approximately follow the contour or the surface of the body 120k.
  • embodiments of the present invention can include the first and second tactors 110k', 110k” that may move about the body 120k.
  • the first and second tactors 110k', 110k” can rotate about the body 120k (e.g., about the Z- axis of the shear display device 100k).
  • Such rotation of the first and second tactors 110k', 110k” can approximately follow the contour or the surface of the body 120k.
  • such rotation may be synchronized in a manner that the first and second tactors 110k', 110k” rotate together, as a single unit.
  • the third and fourth tactors 110m', 110m” can rotate about the body 120k in the same or similar manner as the first and second tactors 110k', 110k".
  • the first and second tactors 110k', 110k” and the third and fourth tactors 110m', 110m” can rotate about the body 120k together.
  • the body 120k may be sized and configured to allow the user to grasp the body 120k together with the first, second, third, and fourth tactors 110k', 110k", 110m', 1 10m".
  • the user can grasp the body 120k in a manner that the user's hand contacts the first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m", which may transmit tactile information to the user.
  • the body 120k may isolate portions of the user's skin that are in contact with the respective first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m", such that the first, second, third, and fourth tactors 110k', 110k", 110m', 110m" may produce skin stretch on the skin portions in contact therewith.
  • various linear and/or nonlinear movements of the first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m” may provide tactile cues or information to the user.
  • the first and second tactors 110k', 110k” can move in opposite directions along or parallel to the Z-axis to signal rotation of an object about the Y-axis.
  • such movement can signal to the user to rotate the shear display device lOOh (e.g., in order to move an object), as described below in further detail.
  • Linear movements of the first and second tactors 110k', 110k” can represent linear movements and/or forces experienced by an object or cues to move an object, as described herein.
  • the third and fourth tactors 110m', 110m” can move in opposite directions along the Z-axis to signal to the user rotation or torque about the X-axis. Likewise, such movement also can signal to the user to rotate the shear display device about the Y-axis.
  • the first, second, third, and/or fourth tactors 110k', 110k", 110m', 110m” may rotate about the body 120k (e.g., about the Z-axis). Among other things, such rotation can signal to the user rotation and/or torque about the Z-axis experienced by an object.
  • such rotation may signal to the user to rotate the shear display device 100k about the Z-axis.
  • first and second tactors 110k' and 110k can be moved upwards, while the third and fourth tactors 1 10m' 110m” are moved downwards to provide a sensation of torque for the user, which would rotate the top of the body 120k about an axis that is approximately 45 degrees between the X- and Y-Axes.
  • An exemplary application of the shear display device 100k may include controlling an object and receiving cues regarding desired or necessary movements for such object and/or for the shear display device 100k.
  • the shear display device 100k may incorporate or may be integrated with a wireless remote or controller, such a Wii remote.
  • moving the shear display device 100k may send instructions (e.g., movement instructions) to a controlled object.
  • the user may receive cues or suggestions about where to move the shear display device 100k (and, thus, the controlled object), as described above.
  • the shear display device 100k can provide correctional or training cues regarding an optimal or improved movement in a particular application.
  • the shear display device 100k can provide correctional or training cues for a tennis swing (e.g., the shear display device 100k can represent or can be incorporated into a handle of a tennis racket and may provide tactile cue regarding where and/or how to move the tennis racket).
  • the shear display device 100k can provide any number of corrective or training cues, which may improve user's movements in connection with any number of activities (e.g., golf, baseball, fishing, etc.).
  • the rotational and translational degrees of freedom communicated by the shear display device 100k could also include assisting in orienting or pointing an object such as a satellite dish or camera. That is, the shear display device 100k can supply the degrees of freedom for pointing the camera with rotations about the X- and Y-Axes, it could suggest how to zoom the lens with translational cues along the Z-axis, and could suggest changes in focus with cues to rotate the camera lens, with rotational cues about the Z-Axis. Such cues could be especially advantageous for individuals with vision impairments.
  • the shear display device can have any number of tactors located on any portion thereof.
  • Figure 11 illustrates another embodiment of a shear display device, which incorporates multiple tactors on the same side thereof. Particularly, Figure 11 illustrates a shear display device 100 ⁇ that incorporates first and second tactors 11 On', 11 On" on the top of a body 120n. Except as otherwise described herein, the shear display device 100 ⁇ and its components or elements can be similar to or the same as any one of the shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, 100k ( Figures lA-10) and their respective components and elements.
  • the first and second tactors 11 On', 11 On can move in opposing directions. More specifically, the first and second tactors 1 ⁇ ', 110 ⁇ " can move in opposing directions along their respective Y' and Y" axes. This movement can generate torque about a center point between the first and second tactors 11 On', 11 On". Such torque can indicate to the user that the shear display device 100 ⁇ and/or the controlled object is rotating or that they should rotate their hand about a Z-axis (not shown), which can be perpendicular to the X'-Y' plane.
  • the displacement or movement of the first and second tactors 11 On', 11 On also can indicate the location, distance, displacement, or motion of the controlled object.
  • moving the first and second tactors 11 On', 11 On" with opposing motions also can indicate torque being applied to the controlled object or that the user should rotate their hand about that respective axis.
  • location or movement of the first and second tactors 1 lOn', 110 ⁇ " can indicate the amount of torque being applied to the controlled object.
  • movement of the first and second tactors 11 On', 11 On" in the same direction can indicate translational/linear movement of the shear display device 100 ⁇ and/or of an object, such as the controlled object.
  • the first and second tactors 11 On', 11 On" can move in opposite directions on an axis that lies along the direction between the center point of each of the tactors to display various tactile cues to the user.
  • the first and second tactors 11 On', 11 On" can move radially away from each other to display zooming out or an increase in distance (virtual or otherwise) between two objects (e.g., between an object and the controlled object; between the user and an object, etc.) or increase in the size of an object.
  • Such movement also can display to the user tensile forces experienced by an object or provide similar force/pressure cues.
  • first and second tactors 11 On', 11 On can move radially toward each other, thereby signaling to the user zooming in or a reduction in distance (virtual or otherwise) between two objects or the decreasing size of an object.
  • movement of the first and second tactors 11 On', 11 On” can indicate compressive forces experienced by an object or provide similar force/pressure cues. For instance, as a tool passes through a narrowing and is compressed, the first and second tactors 11 On', 110 ⁇ " can move toward each other, thereby indicating compression of the tool.
  • the first and second tactors 11 On', 11 On" are offset from each other in both the x- and y-direction. Such a configuration can provide tactile feedback to the user's index and middle fingertips, which are commonly offset in a similar manner as the first and second tactors 11 On', 11 On". This offset configuration can be advantageous if the shear display device 100 ⁇ was a computer mouse or a similar device.
  • the first and second tactors 11 On', 11 On" can be substantially aligned with each other.
  • the shear display device 100 ⁇ can have a continuous edge, and the first and second tactors 1 lOn', 110 ⁇ " can be positioned at approximately the same distance from the edge.
  • the user can position or orient the shear display device 100 ⁇ in any manner relative to the user's hand and/or fingertips, such as to accommodate a particular placement of the user's fingertips on the first and second tactors 1 ⁇ ', 110 ⁇ ".
  • Additional or alternative embodiments include the shear display device
  • the shear display device 100 ⁇ that comprises multiple unconnected bodies, each of which can incorporate one of the first and second tactors 11 On', 11 On".
  • the shear display device 100 ⁇ can comprise multiple unconnected shear display devices, which can be place near each other, such that the user can place fingertips on the desired tactors.
  • Such otherwise unconnected shear display devices can function together (e.g., by receiving commands from the same controller) and can produce the same movement of the first and second tactors 1 ⁇ ', 110 ⁇ " as a single body shear display device 100 ⁇ .
  • the shear display device 100 ⁇ can include first and second tactors 11 On', 11 On" that have any number suitable shapes and sizes, which may vary from one implementation to another.
  • the shear display device 100 ⁇ can include bar- or plate-like first and second tactors (e.g., similar to the first and second tactors HOg', HOg" ( Figure 8).
  • the plate-like first and second tactors can move in any number of ways described herein and can provide the same or similar information as described herein.
  • the plate-like first and second tactors can allow the user to apply a relatively large compressive force to the shear display device 100 ⁇ , while continuing to display information to the user.
  • the plate-like tactors can recruit more mechanoreceptors and, thus, can provide additional sensation in the areas of relatively low density of mechanoreceptors (e.g., palm, wrist, etc.), which can facilitate greater accuracy of the user's perception of the displayed information.
  • mechanoreceptors e.g., palm, wrist, etc.
  • movements of the tactor can display one-, two-, three-dimensional, and generally multi-dimensional information about movement of the controlled object.
  • the shear display device can be connected to or form a part of a controller, which can control the controlled object.
  • the controller also can provide force feedback to the user, as further described herein.
  • a force feedback device which can incorporate both the control functionality and can provide force feedback.
  • the force feedback device can provide force feedback on any one or more of three axes in a three- dimensional space as well as torque feedback about any one of the three axes.
  • the feedback can be such as to provide a certain amount of resistance to movement of a shear display device.
  • shear display device can be coupled to or integrated with a controller that can provide either of the control functionality or force feedback.
  • Figure 12 illustrates a control system 300 that includes the shear display device 100a connected to a force feedback device 310.
  • the force feedback device 310 may include a commercially available force feedback device, such as the Phantom Robot Arm.
  • the force feedback device 310 can provide resistive force in response to the movement of the shear display device 100a.
  • Such resistive force for example, can signal a resistance (as well as magnitude thereof) encountered by the controlled object (where this object could be a physical object or a simulated/virtual object).
  • the force feedback device 310 may provide greater (or in this example infinite) resistance to further advancement in the direction of the bone.
  • the force feedback device 310 can sense movements of the shear display device 100a in one-, two-, three-, and up to six-dimensional space (i.e., linear movements in along any one or more axes in a three-dimensional space plus rotational movements about such axes in the three-dimensional space).
  • the force feedback device 310 can provide instructions to a controller (or to the controlled object), and the instructions can correspond to the user's movements of the shear display device 100a.
  • the force feedback device 310 can provide information to the user about position and/or movement of the controlled object as well as provide instruction to the controlled object related to the movement thereof (e.g., as sensed by the force feedback device 310 from the movement of the shear display device 100a).
  • control system In addition to or in lieu of force feedback, the control system
  • control system 300 can provide the user with information about the movement of an object, such as the controlled object. More specifically, the control system 300 can display position of the controlled object, forces experienced by of the controlled object, etc., via the shear display device 100a. For example, the force feedback device 310 can detect rotation of the shear display device 100a about the Z-axis.
  • the force feedback device 310 can detect rotation of the shear display device 100a about the X-axis. As noted above, as the force feedback device 310 detects rotation of the shear display device 100a about and/or linear movement of the shear display device 100a in the X-, Y-, and/or Z-axes (or any other axes), the force feedback device 310 can translate such linear movements and/or rotations, and direct the controlled object in a corresponding manner. [00157] Similarly, the tactor 110a and its motion can display a representative force experienced by the controlled object.
  • a multi-tactor shear display device e.g., shear display device 100k, Figures 10A-10B
  • skin shear i.e., stretching the skin
  • force may be displayed by displacing the tactor 110a from a center or default position to a second position.
  • the shear display device 100a can display torque by rotating the tactor 110a about its axis and/or moving the tactor 110a along a circular path.
  • direction and distance of movement (or speed or acceleration) of the tactor 110a from its center position can indicate the forces experienced by the controlled object.
  • relatively larger movements of the tactor 110a can represent a relatively greater amount of force experienced by the controlled object.
  • a relatively smaller movement of the tactor 110a can represent a relatively lower amount of force experienced by the controlled object.
  • a shear display device provides localized sensations, which do not affect or move the user's hand.
  • providing haptic feedback via skin shear instead of force feedback can reduce the risk associated with unintentional hand movement, which can occur in response to receiving force feedback from the force feedback device 310.
  • unintentional hand movements during a procedure where a physician controls a medical device with the help of the control system e.g., during a surgery
  • the shear display device 100a can reduce potential accidents that may occur during various procedures, which can be especially relevant to high risk procedures.
  • the control system 300 can provide a combined force feedback and shear feedback.
  • the force feedback device 310 can prevent movement of the shear feedback device 100a in a certain direction (e.g., along X-axis beyond a predetermined point or a limit), and the shear feedback device 100a can provide shear feedback (e.g., by moving in the tactor 110a in a direction away from the predetermined limit).
  • the control system 300 can reduce the amount of force applied in the force feedback, while the user can experience sufficient tactile feedback to for accurate responses to such feedback.
  • Combined force feedback and skin stretch can enhance user's sensation and can lead to more accurate interpretation of the cues as compared with providing only the force feedback or skin stretch cues.
  • control system 300 can be used in surgical procedures, where the tactile feedback can represent the controlled object's interaction with the surrounding environment (e.g., resistance or forces experienced by the controlled object). Accordingly, increased accuracy of movement of the controlled device as well as reduced or eliminated unintentional movements (e.g., movements that can result from excessive force feedback) of the shear display device 110a, which can control the controlled object, can lead to safer surgical procedures.
  • the control system 300 can include any of the shear display devices described herein.
  • the control system 300 may incorporate a multi-tactor shear display device (e.g., the shear display device lOOd ( Figure 5)).
  • the user can receive cues from the multi-tactor shear display device (as described herein) and can move the shear display device in a direction and/or to a position indicated by such cues.
  • the user may rotate the shear display device about the Z-axis (e.g., which would result from the shear feedback provided to a user).
  • the shear display device can move only about the Z-axis, while remaining substantially stationary otherwise (e.g., via force feedback that prevents linear movement and rotation about the X and Y axes.
  • rotation of the shear display device 100a about the Z- axis can direct the controlled object to perform certain function.
  • the controlled object can be directed to rotate.
  • the controlled object can experience torque applied by its environment.
  • providing shear feedback can inform the user about the amount of torque experienced by or the orientation or relative movement of the controlled object.
  • the information about the torque experienced by the controlled object also can be displayed by a combined skin shear and force feedback.
  • the control system 300 can be configured to partially restrict the user's ability to move the shear display device in three- or six- dimensional space.
  • the force feedback device 310 can apply force to prevent the user's movements of the shear display device in one or more directions and/or prevent rotations about one or more axes.
  • the control system 200a may restrict all linear movement of the shear display device and can restrict rotation about X-, Y- or Z- axes.
  • the shear display device can be allowed to rotate only about the Z-axis.
  • the force/torque feedback device 310 can provide resistance to movement of the shear display device in a manner that would effectively constrain the movements of the shear display device to a predetermined plane, surface, or three-dimensional surface or area (e.g., allowing movements only on the surface or on one side of the plane or surface).
  • Such restrictions also can confine movement of the shear display device to a particular confinement plane or surface.
  • the user may be able to move the shear display device only in a two-dimensional plane.
  • the control system 300 can restrict the user's movements of the shear display device to a surface, which can have a three-dimensional profile.
  • the control system 300 can prevent the user from moving the shear display device past a predetermined safety plane or surface in the three- dimensional space. More specifically, the control system 300 can allow the user to move the shear display device to any location on the one side of the safety plane or surface and prohibit movement of the shear display device beyond that side of the safety plane or surface.
  • a surface of an object can be obtained (e.g., using three-dimensional scanning techniques) and can be used as a safety surface for the control system 300. Such safety surface can protect the object from inadvertent contact or impact by the controlled object.
  • the center of the shear display device is the center of the shear display device
  • providing torque feedback at the force feedback device 310 can provide a means of controlling the orientation of the shear display device 100a such that it always lies in the predetermined or desired orientation. All three axes of rotation can be similarly controlled such that the orientation of the shear display device can be suggested to or controlled for the user. Specifically, rotation of the shear display device can be controlled about any one or more of the X, Y, and Z axes.
  • torque feedback can provide an effective means to guide the orientation of the shear display device to be placed in a desired or preferred orientation within a three-dimensional space with three translations and three rotations, for conveying task specific information to the shear display device.
  • torque feedback has a lower potential to cause safety issues.
  • Figures 1-12 and the corresponding text provide a number of different components and mechanisms for displaying movement, direction, force, and torque information to a user via tactile cues provided by the shear display device and/or by the control system.
  • some embodiments of the present invention also can be described in terms of flowcharts comprising acts and steps in a method for accomplishing a particular result.
  • Figure 13 illustrates a flowchart of one exemplary method of providing above-described information with tactile cues using principles of the present invention. The acts of Figure 13 are described herein with reference to the components and diagrams of Figures 1 through 12.
  • the method can involve an act 400 of receiving information about an object.
  • an object can be a real (or physical) object or can be a virtual (i.e., computer-generated) object.
  • an object can be an object being controlled by the user, such as the controlled object, an object used to control one or more other objects (e.g., shear display device in combination with a controller), or any other object being observed by the user.
  • the particular information received can vary from one embodiment to another.
  • such information can include information about the object's movements.
  • such information can include information about the object's direction, velocity, and acceleration.
  • such information can include information about the forces, torques, or pressure experienced by or applied to the object.
  • the method can further involve an act 410 of displaying the information to the user via one or more tactile cues. More specifically, the above-described information about the object can be displayed to the user via predetermined one or more skin stretch cues, which can be provided by the shear display device (e.g. shear display device 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, 100k, 100 ⁇ ). In other words, the shear display device can stretch one or more portions of the user's skin to provide tactile sensations that can correspond with the information about the object and can be interpreted by the user as such. For example, the shear display device can move one or more tactors (e.g.
  • the shear display device can move one or more tactors in a first direction and at a first speed and/or by a first displacement along a linear path to indicate a force experienced by the object (i.e., magnitude of the force and/or direction thereof).
  • the shear display device can move one or more tactors along a curved, circular, or semicircular path to indicate rotation of or a torque experienced by the object.
  • the shear display device can include multiple tactors (e.g. 110c * , 110c", HOd * , 110d", HOe * , HOe", HOf, l lOf", 110g', 1 lOg", 110h', 1 lOh", etc.).
  • the shear display device can move the tactors in a coordinated manner to display linear movement, force, rotation, torque, and combinations thereof.
  • opposing tactors can move in opposite directions (e.g., along X- or Y-axis), as described herein, to indicate rotation of and/or torque experienced by the object.
  • directions of the movement of the tactors and the direction of a perceivable torque created about a center point therebetween can provide information about the object's rotation.
  • speed and/or acceleration as well as displacement of the tactors i.e., the manner in which the user's skin is stretched
  • tactors facing in the same or similar direction also can provide rotational information.
  • one or more of the tactors can rotate about the respective axes thereof or may move in a circular or semicircular paths to display information about the object's rotation and/or torque experienced thereby.
  • the tactors can move in opposite directions along a linear or substantially linear paths (e.g., along X- or Y-axis), thereby producing a perception of torque about a point therebetween. Accordingly, such movements can provide information about the object's rotational movement and/or torque experienced thereby.
  • the tactors also can move toward and away from the user's skin.
  • the tactors can provide three-dimensional information about linear movement or forces experienced by the object.
  • the tactors can provide information about rotational movement and/or torques experienced by the object about any one or more of the three-dimensional axes (i.e., the shear display device 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, etc., can provide four-, five, and six-dimensional information to the user).
  • Embodiments of the present invention also can include and/or can be implemented in performing medical procedures, such as upper extremity rehabilitation, surgery, catheter insertion, etc.
  • a shear display device e.g. shear display devices 100a, 100b, 100c, lOOd, lOOe, lOOf, lOOg, lOOh, etc.
  • the shear display device can inform the user about the forces and/or torques experienced by the catheter or other controlled tool or object. Accordingly, the shear display device can improve the safety of medical procedures, such as catheter insertion and guidance.
  • the shear display device can display impact between two objects (e.g., between a controlled object and another object) in a different manner than a force applied onto the controlled object. Specifically, upon impact, the shear display device can produce a relatively large initial stretch of the user's skin (i.e., a large displacement of the tactors (e.g.
  • tactors 110c', 110c", 110d', 110d", 110e', 110e", HOf, 110f", HOg * , 110g", HOh * , 110h”, etc.) which can be proportional to the impact (e.g., to the speed of the controlled object at the time of impact and/or mass thereof), followed by displaying further tactor motion in proportion to the penetration of the virtual object that is contacted.
  • a larger initial stretch of the user's skin can produce an enhanced or more accurate perception of impact.
  • a single or multiple tactors (as applicable) can move in unison to produce skin stretch of multiple portions of the user's skin.
  • the displacement of the tactor and the corresponding amount of skin stretch can be proportional to the force or torque experienced by the object. For instance, a force of 2 N can be displayed by displacing the tactor by 1 mm, while a force of 4 N can be displayed by displacing the tactor by 2 mm.
  • relationships between the force experienced by the object and the displacement of the tactor can be non-linear (e.g., logarithmic, quadratic, etc.).
  • small amounts of force can be displayed to the user by sufficiently large displacement of the tactor, such that the user can perceive such forces.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Des modes de réalisation de la présente invention concernent des dispositifs, des systèmes et des méthodes d'affichage d'informations à propos de la direction et de l'amplitude de la position, du mouvement et/ou de la force de résistance éprouvées pour un objet. La présente invention concerne aussi un dispositif d'affichage de cisaillement qui peut produire un cisaillement de peau avec un ou plusieurs stimulateurs tactiles, chacun se déplaçant dans un espace bi- ou tridimensionnel. Le mouvement des stimulateurs tactiles peut représenter différentes informations à propos d'un objet à un utilisateur.
EP13803464.0A 2012-06-13 2013-03-15 Dispositifs, systèmes et méthodes de rétroaction d'étirement de peau Ceased EP2862151A4 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201261659421P 2012-06-13 2012-06-13
US201261659452P 2012-06-14 2012-06-14
US201261660162P 2012-06-15 2012-06-15
PCT/US2013/032053 WO2013187977A1 (fr) 2012-06-13 2013-03-15 Dispositifs, systèmes et méthodes de rétroaction d'étirement de peau

Publications (2)

Publication Number Publication Date
EP2862151A1 true EP2862151A1 (fr) 2015-04-22
EP2862151A4 EP2862151A4 (fr) 2016-01-13

Family

ID=49758596

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13803464.0A Ceased EP2862151A4 (fr) 2012-06-13 2013-03-15 Dispositifs, systèmes et méthodes de rétroaction d'étirement de peau

Country Status (2)

Country Link
EP (1) EP2862151A4 (fr)
WO (1) WO2013187977A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9317123B2 (en) 2012-06-13 2016-04-19 University Of Utah Research Foundation Skin stretch feedback devices, systems, and methods
GB201420931D0 (en) 2014-11-25 2015-01-07 Nokia Corp Apparatuses, methods and computer programs for providing vehicle feedback
US9946350B2 (en) 2014-12-01 2018-04-17 Qatar University Cutaneous haptic feedback system and methods of use

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009018330A2 (fr) * 2007-07-30 2009-02-05 University Of Utah Research Foundation Système d'affichage tactile par cisaillement pour communiquer une direction et d'autres repères tactiles
US20090139359A1 (en) * 2007-12-04 2009-06-04 Cybernet Systems Corporation Cable driven tactor
US20100161079A1 (en) * 2008-12-22 2010-06-24 The Board Of Trustees Of The Leland Stanford Junior University Skin stretch tactile feedback device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4591868A (en) * 1982-04-09 1986-05-27 National Industries, Inc. Collapsible motor operated antenna
GB2410316B (en) * 2002-10-20 2007-03-21 Immersion Corp System and method for providing rotational haptic feedback
US7554167B2 (en) * 2003-12-29 2009-06-30 Vladimir Vaganov Three-dimensional analog input control device
US9268401B2 (en) * 2007-07-30 2016-02-23 University Of Utah Research Foundation Multidirectional controller with shear feedback
WO2011062910A1 (fr) * 2009-11-17 2011-05-26 Immersion Corporation Systèmes et procédés pour dispositif rotatif à friction servant au retour de sensations haptiques

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009018330A2 (fr) * 2007-07-30 2009-02-05 University Of Utah Research Foundation Système d'affichage tactile par cisaillement pour communiquer une direction et d'autres repères tactiles
US20090139359A1 (en) * 2007-12-04 2009-06-04 Cybernet Systems Corporation Cable driven tactor
US20100161079A1 (en) * 2008-12-22 2010-06-24 The Board Of Trustees Of The Leland Stanford Junior University Skin stretch tactile feedback device

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
None *
See also references of WO2013187977A1 *

Also Published As

Publication number Publication date
EP2862151A4 (fr) 2016-01-13
WO2013187977A1 (fr) 2013-12-19

Similar Documents

Publication Publication Date Title
US10488934B2 (en) Skin stretch feedback devices, systems, and methods
US20230301738A1 (en) Master control device and methods therefor
Schorr et al. Three-dimensional skin deformation as force substitution: Wearable device design and performance during haptic exploration of virtual environments
US11135029B2 (en) User-interface control using master controller
US11191596B2 (en) Foot controller
US6024576A (en) Hemispherical, high bandwidth mechanical interface for computer systems
US20200275985A1 (en) Master control device with multi-finger grip and methods therefor
EP3217910B1 (fr) Interaction entre une interface utilisateur et une unité de commande maître
US20080248871A1 (en) Interface device
WO2019217882A1 (fr) Dispositif de commande maître à fonction de détection de préhension par les doigts et procédés associés
US20040041828A1 (en) Adaptive non-contact computer user-interface system and method
KR102162922B1 (ko) 촉각 피드백이 가능한 가상현실 기반의 손 재활 시스템
EP2862151A1 (fr) Dispositifs, systèmes et méthodes de rétroaction d'étirement de peau
Ma et al. Sensing and force-feedback exoskeleton robotic (SAFER) glove mechanism for hand rehabilitation
WO2007030947A1 (fr) Mappage de capteurs de mouvement sur des dispositifs d'entree standard
US20240111357A1 (en) Interaction between user-interface and master controller
Avizzano et al. Design guidelines for generating force feedback on fingertips using haptic interfaces
CN116761568A (zh) 用户接口和主控制器之间的交互

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150105

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIN1 Information on inventor provided before grant (corrected)

Inventor name: GWILLIAM, LANDON T.

Inventor name: PROVANCHER, WILLIAM, R.

Inventor name: MONTANDON, MARKUS N.

DAX Request for extension of the european patent (deleted)
RA4 Supplementary search report drawn up and despatched (corrected)

Effective date: 20151214

RIC1 Information provided on ipc code assigned before grant

Ipc: G08B 6/00 20060101AFI20151208BHEP

Ipc: G06F 3/033 20060101ALI20151208BHEP

Ipc: G06F 3/041 20060101ALI20151208BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20170328

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

REG Reference to a national code

Ref country code: DE

Ref legal event code: R003

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20210228