WO2009124209A1 - Mesure fiable des vibrations induites par le frottement de glissement pour la détection tactile biomimétique - Google Patents

Mesure fiable des vibrations induites par le frottement de glissement pour la détection tactile biomimétique Download PDF

Info

Publication number
WO2009124209A1
WO2009124209A1 PCT/US2009/039354 US2009039354W WO2009124209A1 WO 2009124209 A1 WO2009124209 A1 WO 2009124209A1 US 2009039354 W US2009039354 W US 2009039354W WO 2009124209 A1 WO2009124209 A1 WO 2009124209A1
Authority
WO
WIPO (PCT)
Prior art keywords
skin
fluid
core
sensor
pressure
Prior art date
Application number
PCT/US2009/039354
Other languages
English (en)
Inventor
Gerald E. Loeb
Jeremy Fishel
Nicholas Wettels
Veronica J. Santos
Raymond Peck
Original Assignee
University Of Southern California
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University Of Southern California filed Critical University Of Southern California
Publication of WO2009124209A1 publication Critical patent/WO2009124209A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B25J13/081—Touching devices, e.g. pressure-sensitive
    • B25J13/084—Tactile sensors
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2/54—Artificial arms or hands or parts thereof
    • A61F2/58—Elbows; Wrists ; Other joints; Hands
    • A61F2/583—Hands; Wrist joints
    • A61F2/586—Fingers
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2/68—Operating or control means
    • A61F2/74—Operating or control means fluid, i.e. hydraulic or pneumatic
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2/76—Means for assembling, fitting or testing prostheses, e.g. for measuring or balancing, e.g. alignment means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B25J13/081—Touching devices, e.g. pressure-sensitive
    • B25J13/082—Grasping-force detectors
    • B25J13/083—Grasping-force detectors fitted with slippage detectors
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2002/5001—Cosmetic coverings
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2002/5058—Prostheses not implantable in the body having means for restoring the perception of senses
    • A61F2002/5061—Prostheses not implantable in the body having means for restoring the perception of senses the sense of touch
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2/68—Operating or control means
    • A61F2002/6827—Feedback system for providing user sensation, e.g. by force, contact or position
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2/68—Operating or control means
    • A61F2/70—Operating or control means electrical
    • A61F2002/704—Operating or control means electrical computer-controlled, e.g. robotic control
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F2/00—Filters implantable into blood vessels; Prostheses, i.e. artificial substitutes or replacements for parts of the body; Appliances for connecting them with the body; Devices providing patency to, or preventing collapsing of, tubular structures of the body, e.g. stents
    • A61F2/50—Prostheses not implantable in the body
    • A61F2/76—Means for assembling, fitting or testing prostheses, e.g. for measuring or balancing, e.g. alignment means
    • A61F2002/7615—Measuring means
    • A61F2002/7635—Measuring means for measuring force, pressure or mechanical tension

Definitions

  • This application relates generally to devices and methods to provide tactile sensory information from robotic or prosthetic finger tips comparable to the tactile sensing provided by human skin.
  • Tactile recognition the ability to recognize objects by manipulation
  • tactile sensors can be put in contact with the object to extract information about and, even more, such contact should be competently organized in order to extract the maximum degree of information from manipulative acts.
  • the most common groups are piezoresistive, piezoelectric, capacitive and elastoresistive structures.
  • the common feature of all of these devices is the transduction of mechanical strains or deformations into electrical signals.
  • Tactile sensors are commonly used in the field of robotics and in particular with those robotic devices that pick up and place objects in accordance with programmed instructions; the so- called "pick and place” class of robot.
  • Unfortunately while it would be desirable for the above-listed groups of tactile sensors to respond in much the same way that the human finger does, many of them can provide only limited information about contact with an object whose position, orientation and mechanical properties are highly predictable. More generalized sensing requires a multiplicity of sensors and extensive electrical connections and signal processing circuitry.
  • MEMS micro-electromechanical system
  • the input-output properties of these biological transducers differ generally from engineered transducers.
  • Engineered transducers are usually designed to produce a linear response to a single mechanical variable such as normal or tangential force at a single point.
  • the signals from arrays of such transducers can be combined according to simple, analytical algorithms to extract orthogonal physical parameters of touch such as total force, center of force, directional force vector and two-point resolution.
  • Biological touch receptors are highly nonlinear and non-orthogonal. Their signals are combined by adaptive neural networks to provide subconscious adjustment of motor output as well as high level conscious perception associated with haptic identification of objects.
  • Engineered sensors and their signal processing systems use linear, orthogonal representations because the downstream control systems generally have been based on such inputs.
  • This strategy may work well for engineered systems such as industrial robots that can perform accurately for highly constrained and predictable tasks. It is difficult to apply to anthropomorphic robots and prosthetic limbs that can perform a broad and unpredictable range of tasks associated with activities of daily living. The problem may further be complicated by environmental factors in such environments (e.g. temperature, moisture, sharp edges etc.), which tend to damage or bias sensitive and/or physically exposed transducers.
  • U.S. Patent No. 4,980,646, to Zemel (“Zemel”), is incorporated in its entirety herein by reference and teaches a tactile sensor based on changes in the local electrical resistance presented by a layer of weakly conductive fluid whose shape is deformed by external forces applied to a deformable membrane.
  • Zemel describes the application of a voltage gradient across the entire extent of the fluid by means of electrodes arranged on either side of the array of sensing strips, and the measurement of the local strength of that gradient by differential voltage measurements between adjacent pairs of electrode strips.
  • U.S. Patent No. 4,555,953 to Dario et al. which is incorporated herein by reference in its entirety, teaches different techniques and materials that have been utilized for the construction of artificial skin-like sensors.
  • Johansson R., et al. Somatosensory control of precision grip during unpredictable pulling loads, Changes in load force amplitude, Experimental Brain Research 89: 181 -191 , 1992 . Birznieks I., et al, Encoding of direction of fingertip forces by human tactile afferents, Journal of Neuroscience. 21 :8222-8237, 2001. Flanagan J. R., et al. Control of fingertip forces in multi-digit manipulation, Journal of Neurophysiology. 81 :1706-1717, 1999. Johansson R.S., Westling G., Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects, Experimental Brain Research.
  • Embodiments of the present disclosure are directed to biomimetic sensors, and related structures and processes.
  • Exemplary embodiments of the present disclosure include sensory devices that have features comparable to features found in biological systems.
  • they may use biomimetic mechanical structures similar to those found in the finger tip to endow a set of simple, robust electronic sensors with a wide range of modalities and sensitivities similar to those found in biological mechanoreceptors.
  • Exemplary sensory devices include a sensor assembly whose basic form and function are similar to that of a human finger tip.
  • the sensory device may have a biomimetic shape of a core with covering skin and pulp (fluid reservoir) that results in distinctive and readily detectable patterns of impedance changes across an array of electrodes disposed on the core, to take advantage of the various distortions of the pulp produced by the contact parameters to be detected and discriminated.
  • High detection sensitivity and wide dynamic range can be achieved for monitoring and/controlling the forces between a manipulator and objects being manipulated.
  • the biomimetic designs of such sensor assemblies can allow for detection of stimulus features, e.g., by feature extraction circuitry, including those features that may be most useful for automatic adjustment of contact force to achieve and maintain stable and efficient grasp of an object.
  • An exemplary embodiment comprises a device through which a set of information is generated concerning tactile interaction between a manipulator and an object to be manipulated and recognized.
  • a device can be incorporated into autonomous robots, telerobots or prosthetic limbs.
  • the tactile information may be generated either by robot or prosthetic finger tips.
  • Biomimetic tactile sensors taught herein may possess softness, elasticity, and some mechanical resistance that mimics natural human skin. Such sensors can detect and discriminate various aspects of contact with external objects, including the direction and magnitude of force, the location, extent and shape of the contacting object, and small movements associated with impending slip. Furthermore, such sensors may discriminate thermal properties of contacted objects through heat-flow sensing.
  • Exemplary embodiments may employ a number of small, local electrodes, e.g., deployed in a curved array, as part of a sensing modality having a shape and mechanical properties to mimic those of a biological finger tip. Such electrodes can be used to detect changes in impedance of a fluid within the sensor. Each sensing electrode may be energized to provide an independent measure of the local mechanical deformations of the overlying membrane based on its impedance with respect to a remote common electrode. Further improvements are described to enhance the sensitivity and dynamic range of each sensing electrode by contouring the inner surface of the overlying membrane.
  • neural networks may compute directly the actuator adjustments required to maintain stable grip of objects with a variety of shapes and force vectors in a manner similar to that employed by neural control of the human hand.
  • Exemplary embodiments can include one or more temperature sensors, e.g., thermocouple or thermistor, mounted to the surface of the core.
  • the surrounding core is heated by the supporting electronics above ambient; when objects are contacted, they will cause temperature changes in the temperature sensor (e.g., thermistor) consummate with their thermal properties.
  • These detected voltage changes can be exported to logic for analysis.
  • Exemplary embodiments of biomimetic sensors and related techniques may employ a fluid pressure sensor either stand-alone or in addition to other types of sensor modalities, e.g., a number of small, local electrodes deployed in a curved array to detect fluid impedance changes.
  • a fluid pressure sensor to detect the static and dynamic pressure of a fluid trapped between an elastomeric skin and a rigid core allows for the detection of dynamic vibrations that arise as a device of this design is slid across a textured surface mimicking the function of known biological transducers.
  • the sensing of static pressure levels of the fluid can be used to enhance the ability to extract the normal forces as detected by impedance electrodes.
  • the pressure sensor may be located inside a rigid core where it is displaced from the contact region of the finger. From this location it can be coupled with the fluid via a small pathway. A fluid with low compressibility and low viscosity can be used to reduce the attenuation as the pressure waves travel to the transducer in this remote location.
  • Embodiments can include a biomimetic tactile sensor that is sensitive to the wide range of normal and shear forces encountered in robotic and prosthetic applications. Spatially resolved force distributions can be detected through electrode impedance fluctuations as the conductive fluid profile deforms. A useful force range for biomimetic impedance sensors can be extended by internally texturing their elastomeric skin.
  • One or more temperature sensors e.g., a thermistor
  • the temperature sensors (thermistor) or the core material around it can be heated above ambient to detect heat flow to extract thermal features of contacted objects. Heat produced of dissipated by a microprocessor within control electronics can, for exemplary embodiments, be controlled (e.g., by appropriate software/firmware instructions), which is used to affect thermal characterization of contacted objects.
  • One embodiment of the present device may consist of a set of sensors that work by measuring static and dynamic pressure in a fluid or gel that is trapped by the elastomeric skin.
  • the pressure transducer may be deployed inside a substantially rigid core that is protected from direct contact with external objects.
  • a feature of this design may be the location of mechanically vulnerable pressure transducer and signal processing circuitry, which can be wholly contained within the substantially rigid core.
  • a related feature may be that this design enables methods of manufacture and repair that are simple and efficient.
  • the plurality of sensors and their associated mechanical structures have similarities to the biological relationships among the cutaneous neural receptors, the distal phalanx, overlying finger pulp and covering skin and nail.
  • Information may be extracted from such a plurality of sensors whereby such information can be related to canonical physical representations used to describe stimuli to be sensed, and/or used to control automatic adjustments of grip forces similar to the neural reflexes whereby humans maintain stable grip on complex objects.
  • One embodiment of present device may consist of a biomimetic tactile sensor that includes a dynamic pressure sensor to measure fluid pressure changes due to sliding motion of the device skin over a surface encountered in robotic and prosthetic applications.
  • the fluid can have a low viscosity and low compressibility such that these dynamic pressures are transmitted with little attenuation through the fluid allowing for the pressure detection circuitry to be housed far away from the contact surface where it can be protected from damage while still maintaining sensitivity to dynamic tactile events.
  • Another embodiment of the present device may consist of a biomimetic tactile sensor capable of detecting vibrations relating to the onset of slip.
  • Another embodiment may consist of a biomimetic tactile sensor with signal processing electronics for rapid extraction of spectral information of these vibrations for realtime slip detection and automated grip control.
  • Another embodiment may consist of a biomimetic tactile sensor with signal processing electronics which also adds normal and tangential force sensing to determine the true coefficient of friction between the interface between the finger and the object.
  • Another embodiment may consist of a method of controlling grip force based on the accurate measure of grip force as detected by the biomimetic tactile sensor and signal processing electronics.
  • Another embodiment may consist of a method of controlling grip force that reduces power by relaxing grip force until slip is detected and then increasing grip force to maintain grasp.
  • Another embodiment of the present device may consist of a biomimetic tactile sensor capable of detecting vibrations characteristic of surface texture.
  • Another embodiment may consist of a biomimetic tactile sensor with signal processing electronics for extraction of spectral information related to texture for texture identification.
  • Another preferred embodiment may consist of a biomimetic tactile sensor capable of sensing normal and shear forces that are used in conjunction with an actuator to control the exploration of a surface in an optimal pattern to identify texture.
  • Another embodiment may consist of a biomimetic tactile sensor capable of sensing friction-induced vibrations in conjunction with a haptic display to replay this information on human skin to take advantage of biological mechanisms of slip detection and texture identification.
  • Another embodiment may consist of a biomimetic tactile sensor with external texturing similar to fingerprints or bumps which further enhances the response of the sensor to friction induced vibrations.
  • biomimetic tactile sensor systems and methods according to the present disclosure will become readily apparent to those skilled in the art from the following detailed description, wherein exemplary embodiments are shown and described by way of illustration.
  • the biomimetic tactile sensor systems and methods are capable of other and different embodiments, and details of such are capable of modification in various other respects. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not as restrictive.
  • FIG. 1 depicts a longitudinal cross-section of a tactile sensor in the form of a finger pad, in accordance with exemplary embodiments of the present disclosure
  • FIG. 2 depicts several views including a detail of a fill-port disassembled (A) and assembled (B), and detail of resealing forces present on the fill-port (C), in accordance with exemplary embodiments of the present disclosure;
  • FIG. 3 depicts a schematic of the electronic system for signal detection and processing, in accordance with exemplary embodiments of the present disclosure
  • FIG 4 depicts forces diagrams of a tactile sensor, showing measurement of both normal and shear forces, at a cross section of the tactile sensor subjected to low shear force (A), and a cross section of the tactile sensor subjected to high shear force (B), in accordance with exemplary embodiments of the present disclosure
  • FIG. 5 depicts an internally textured skin with asperities; a perspective view is shown (A), a side view depicts an uncompressed state, (B) and a side view is shown depicting an applied force that compresses the textured rubber and narrows the flow path of the fluid to the electrode surface (C), in accordance with exemplary embodiments of the present disclosure;
  • FIG. 6 illustrates a five-times-repeatability log-log plot of force versus impedance for a skin incorporating inner-surface asperities with dimensions 0.25 tall x 0.25 mm diameter, in accordance with exemplary embodiments of the present disclosure
  • FIG. 7 demonstrates pressure signals associated with contact and sliding for a variety of contact events: tapping (A), pushing (B), sliding (C), and poking (D), along with an AC pressure signal (E) on the same time scale as normal and tangential forces as recorded by a force plate (F), in accordance with exemplary embodiments of the present disclosure;
  • FIG. 8 shows spectrograms of vibration signals used to discriminate textured surfaces, in accordance with exemplary embodiments of the present disclosure
  • FIG. 9 illustrates a cross section of synthetic skin with ridges performing the function of fingerprints to enhance vibrations, in accordance with exemplary embodiments of the present disclosure
  • FIG. 10 depicts a block diagram of a system with a manipulator, display, control features, and signal processing features, in accordance with exemplary embodiments of the present disclosure
  • FIG. 1 1 depicts a flow chart for detecting friction coefficient, in accordance with exemplary embodiments of the present disclosure
  • FIG. 12 depicts a flow chart for controlling grip force with a known friction coefficient while also checking for slip, in accordance with exemplary embodiments of the present disclosure
  • FIG. 13 depicts a flow chart for controlling grip force using only slip detection, in accordance with exemplary embodiments of the present disclosure.
  • FIG. 14 depicts a flow chart for controlling exploratory movements for discriminating textures using vibration sensing, in accordance with exemplary embodiments of the present disclosure.
  • FIG.15 depicts a flow chart for controlling exploratory movements and signal processing related to characterizing thermal properties of contacted objects, in accordance with exemplary embodiments of the present disclosure.
  • a prosthetic hand or anthropomorphic robotic manipulator in accordance with the present disclosure can combine several sensor assemblies at the ends of appendages controlled by actuators, similar to the individual finger tips of a biological hand or foot.
  • Pad-like structures with sensors can be deployed on grip contact surfaces akin to the palmar eminences over the heads of the metacarpal bones etc.
  • One or more such sensor assemblies could be built with various sizes and shapes and mounted in varying numbers and positions on a variety of hand- like or foot-like locomotor supports to interact with external objects while utilizing information derived from a plurality of contact sensors having one or more types of sensing modalities.
  • the plurality of sensors and their associated mechanical structures can have similarities to the biological relationships among the cutaneous neural receptors, the distal phalanx, overlying finger pulp and covering skin and nail.
  • Information may be extracted from such a plurality of sensors whereby such information can be related to canonical physical representations used to describe stimuli to be sensed, and/or used to control automatic adjustments of grip forces similar to the neural reflexes whereby humans maintain stable grip on complex objects.
  • a sensor assembly 100 can consist of mechanical elements corresponding to a biological fingertip, namely a molded, rigid core 1 corresponding to the bone, a molded, elastomeric skin 16 corresponding to the biological skin, a fluid-filled space 2 corresponding to the deformable pulp of a biological finger, and a plate 22 affixing the skin 16 to the top surface of the fingertip corresponding to a fingernail.
  • One embodiment of the present device may consist of a set of sensors that work by measuring electrical impedances through a weakly conductive fluid in contact with a plurality of electrodes.
  • the electrodes may be deployed on a substantially rigid core that is protected from direct contact with external objects by overlying deformable structures.
  • a feature of this design may be the location of mechanically vulnerable connections between the electrodes and the signal processing circuitry, which are wholly contained within the substantially rigid core.
  • a related feature may be that this design enables methods of manufacture and repair that are simple and efficient.
  • sensors may be incorporated alone or in various combinations: [0050] i) Impedance Sensing - electrodes 6 on the surface of the core 1 that are used to detect changes in the impedance through the weakly conductive fluid that arise as a result of deformation of the overlying skin 16;
  • Temperature Sensing - a temperature sensor e.g., thermistor 15 capable of detecting temperature and temperature fluctuations.
  • one or more sensor assemblies 100 and their associated signal conditioning circuitry 9 can be incorporated into actuated manipulators 500 that perform exploratory movements that both rely on the signals from the sensors 100 to control their movements and interactions with external objects and surfaces and utilize signals from the sensors 100 to identify various properties of external objects and surfaces.
  • actuated manipulators 500 that perform exploratory movements that both rely on the signals from the sensors 100 to control their movements and interactions with external objects and surfaces and utilize signals from the sensors 100 to identify various properties of external objects and surfaces.
  • a sensor assembly 100 consists of a substantially rigid central core 1 covered by an elastomeric skin 16.
  • the surface of core 1 contains a multiplicity of electrodes 6.
  • a weakly conductive fluid is introduced into space 2 thereby separating skin 16 from that region of the core 1 where electrodes 6 are located.
  • the edge of skin 16 is shaped and dimensioned into collar 20 which provides a seal to core 1 to prevent leakage of the fluid; the force affecting this seal may be augmented by shrinking band 21.
  • the portion of skin 16 that lies on the top portion of sensor assembly 100 is compressed against core 1 by plate 22, which is held in place by screws 23 tightened into threaded inserts 24 in core 1.
  • the overall structure is similar to a human fingertip with a fingernail.
  • the weakly conductive fluid can be introduced into space 2 via hypodermic needle 26 during manufacture or servicing. Plates 19 molded into skin 16 create pressure seals around screws 23 and hole 25 where the pressurized region of skin 16 forms a resealable injection port when punctured by hypodermic needle 26. This process can be aided by monitoring the fluidic pressure as sensed by a pressure sensor 10 to inflate to a prescribed hydraulic pressure level.
  • One way to avoid air bubbles is to prevent their introduction in the first place. In an exemplary embodiment, this can be accomplished through the following design features illustrated in FIG. 1 and fabrication steps:
  • Core 1 is designed to be injection molded around all of the internal components illustrated in FIG. 1.
  • tube 4 and inlet channel 12 will have already been filled with inert fluid and plugged with soluble keeper 3 made from a material that dissolved readily upon contact with the weakly conductive fluid but not the inert fluid.
  • Soluble keeper 3 contacts the walls of the mold so that it forms a continuous path from the location of the resealable injection port to the space 2 to be inflated with weakly conductive fluid.
  • Core 1 can be inserted into skin 16 without introducing air pockets by performing this step while submerged in a fluid.
  • Soluble keeper 3 can be made from a material such as polyvinyl alcohol that can be softened by heating and then displaced and dissolved by the weakly conductive fluid when it is introduced via hypodermic needle 26.
  • the choice of the weakly conductive fluid is a solution of 0.4M NaI in propylene glycol.
  • One choice of the inert fluid in contact with the pressure sensor is mineral oil. It is desirable to protect the transductive electronic elements of pressure sensor 10 from contact with the salt ions that provide the electrical conductivity of the weakly conductive fluid, hence the requirement that these two fluids be immiscible in each other. It is also desirable the weakly conductive fluid not lose substantial volume by outward diffusion through skin 16 or gain substantial volume by inward diffusion of water if the sensor is used in a humid or wet environment. This can be achieved by the selection of propylene glycol as the solvent base of the weakly conductive fluid and silicone elastomer for the skin 16. These examples are illustrative only; other combinations of fluids and elastomers meeting these general requirements can be practiced within the scope of the present disclosure.
  • core 1 The components that are contained within core 1 can be preassembled mechanically and electrically via flex circuit 5 and loaded into the injection mold used to form core 1. These mechanical and electrical connections can be made by conventional techniques well-known to practitioners of the art, including soldering and conductive epoxy. These components include electrodes 6, which can be made from disks of conductive, inert metals such as gold or platinum, thermistor 15, pressure sensor 10 in housing 11 , and electronic components 9 as required for signal conditioning and processing as described later.
  • the sensor assembly 100 has been designed with many features such that the sensing mechanisms are robust and protected by the rigid core 1. This permits for the sensor assembly 100 to have a prolonged useful life before needing replacement.
  • the elastic skin 16 however through normal operation will wear down at a much faster rate and will need to be replaced in more frequent service intervals.
  • the design of this tactile sensor consisting of a homogeneous elastic skin which contains no electronics provides a unique advantage of low repair costs as the materials for casting skins are comparatively cheap than skins used in competing technologies.
  • injecting fluid into the device may be made possible by inserting a hypodermic needle 26 through hole 25 in plate 22 which was designed to reseal after the hypodermic needle 26 is removed.
  • Novel aspects of this design include forming the elastic skin 16, rigid plate 22 and rigid core in such a way that once compressed and held together with screws 23 the resulting pressure produces adequate sealing properties, this is illustrated in FIG. 2 (A) and (B).
  • the skin 16 is thickened to permit for the development of compression forces from the rigid surfaces it contacts once assembled.
  • the core 1 and plate 22 are made of a material of higher stiffness than the skin to ensure that the skin absorbs the deformation. As shown in FIG.
  • the rigid surfaces on the core 1 and the place 22 that come into contact with the deformable skin 16 are angled such that the resultant forces produce compression perpendicular to the axis in which the hypodermic needle 26 will be inserted. Once the syringe is removed these forces act to reseal the tiny hole that was created when the hypodermic needle 26 was inserted. This allows for the skin 16 to have a longer lifetime over more inflation cycles before needing to be replaced.
  • Impedance Sensing may be accomplished by measuring changes in the electrical impedance among electrodes 6 whose distribution and location on the contoured surface of the core 1 may be a key factor in the sensing properties of the sensor assembly 100.
  • detection circuitry 9 is illustrated schematically in FIG. 3 and described in more detail below.
  • the electrical impedances Z 0- i5 so measured can be dominated by the dimensions and electrical properties of the weakly conductive fluid in space 2. These dimensions are changed by forces applied to skin 16, which result in deformation of the skin 16 and displacement of the weakly conductive fluid.
  • Multiplexer 215 selects each electrode in turn for connection to the measurement circuitry under the control of microcontroller 230.
  • the impedance Z of a selected electrode 6 can be measured by applying a voltage to a circuit consisting of one or more excitation electrodes 6 located elsewhere on the surface of core 1 in series with selected electrode 6 and the intervening weakly conductive fluid path between them plus a fixed reference resistor 220 labeled R
  • the applied voltage is an alternating or pulsatile voltage that does not result in net direct current flow through the electrodes 6 which would tend to damage them by inducing electrolysis and corrosion.
  • this is achieved by using a clock signal CLK in series with a DC blocking capacitor, but other configurations would be obvious to someone normally skilled in the art.
  • Oa d At the peak of each pulse in clock signal CLK, the voltage across R
  • Oa d plus the selected one of Z 0- i5 constitute a voltage divider such that the voltage measured across R
  • Pressure Sensing may be accomplished by one or more pressure sensors
  • Pressure Sensing can be usefully divided into the relatively large, quasistatic pressures P DC in the fluids, which are measured by pressure sensing subsystem 205, and the much smaller, audio frequency vibrations P A c arising as the skin 16 slides over objects in contact with sensor assembly 100, which are measured by vibration sensing subsystem 205.
  • the sensitivity and dynamic range of a pressure sensor 10 depends on the nature of the material contained within its associated reference channel 13 within its housing
  • pressure sensing subsystem 205 includes a wide dynamic range pressure sensor 10 and its reference channel 13 is filled with an air pocket that is trapped there by cap 14.
  • the vibration sensing subsystem 205 includes a narrow dynamic range pressure sensor 10 that needs only to be sensitive to alternating fluctuations in pressure. This is achieved by trapping an air pocket in its reference channel 13 but connecting the other side of the reference channel via a small gage tube (not illustrated) back to its inlet channel 12.
  • the combination of an incompressible fluid path in the inlet channel 12 and a compressible air bubble in series with the high fluidic resistance of the small gage tube constitutes a mechanical high pass filter, protecting pressure sensor 10 from large hydrostatic pressures that may arise in the fluid during the initial inflation of space 2 or firm contact with external objects. This allows for use of a more sensitive transducer with smaller dynamic range but larger signal-to-noise ratio. Alternatively, if a single pressure sensor 10 is available with sufficient dynamic range and signal-to-noise values, both pressure and vibration information may be extracted from it using analog or digital signal conditioning means as would be obvious to one normally skilled in the art.
  • the signals from the one or more pressure sensors 10 are suitably amplified by X100 and digitized by ADC channels in microcontroller 230.
  • Temperature Sensing may be accomplished via thermistor 15 and associated circuitry described in FIG. 3. Temperature Sensing may be usefully divided into i) quasistatic sensing of ambient temperature T D c by thermistor 15 in DC temperature subsystem 240; and ii) dynamic sensing of local fluctuations in temperature T A c by AC temperature subsystem 235. Such fluctuations in temperature that may be induced by contact between the region of the skin 16 overlying thermistor 15 (as illustrated mechanically in FIG. 1 ) and an external object at a temperature different from the ambient temperature. The ambient temperature in sensor assembly 100 depends on the equilibrium between heat energy generated through the operation of all of its electronic components 9 and the heat energy conducted away by various conductive, convective, and radiative losses.
  • the heat energy generated will tend to be dominated by the contribution from microcontroller 230, whose power dissipation can be controlled dynamically by its software, for example by changing its clock rate or sleep intervals.
  • the ambient temperature in the environment around sensor assembly 100 can thus be inferred by the amount of heat energy that can be generated to keep thermistor 15 at a desired temperature, such as the 37C core temperature of the human body.
  • T D c is sensed by a voltage divider and amplifier, one of many measurement circuits that would be well-known to practitioners of the art.
  • the fluctuations T A c will depend on the difference between the temperature of the core 1 and the temperature of a contacting object, the thermal conductivity of that contacting object, and the location, extent and force of contact between sensor assembly 100 and the contacting object. These fluctuations can be high-pass filtered and amplified to generate T A c-
  • the signals from DC temperature subsystem 240 and AC temperature subsystem 235 are digitized by ADC channels in microcontroller 230. Signal Processing and Transmission
  • a simple circuit can be built from off-the-shelf components, including integrated circuits that could be procured as surface-mount packages or bare dies and incorporated onto flex circuit 5.
  • FIG. 4 illustrates a cross-sectional view of sensor assembly 100 that is orthogonal to the view in FIG. 1.
  • the positioning of the electrodes 6 with respect to the contours of the core 1 and overlying fluid-filled space 2 and skin 16 may cause distinct patterns of change in the various impedances Z 0- i5 measured by electronic components 9 as the sensor assembly 100 contacts various objects and surfaces with various force vectors. It may be useful to identify how different aspects of any particular stimulus parameter to be sensed will influence the array of electrodes comprising the sensor assembly 100.
  • a single point of contact may experience various combinations of normal and tangential force components (labeled Fnorm and Ftang, respectively), which result in distributed changes in the impedances measured at each of the electrodes 6 as a result of sliding and deformation of skin 16 with respect to core 1 and plate 22.
  • Fnorm and Ftang normal and tangential force components
  • Sensor assembly 100 can have properties similar to the biological fingertip, however, so it may likely require non-analytical signal processing methods similar to those employed by the biological nervous system.
  • the temporospatial distribution of activity in the biological touch sensors depends complexly on the inherent sensitivity of the sensors, their distributions throughout the tissues of the fingertip and the forces that the fingers apply to external object, as well as on the nature of the external object itself.
  • force magnitude and location interact with each other. For example, the same force vector applied close to the nail bed may create a different amount of net impedance change than if applied to the fingertip; the total change in impedance may not be used as a measure of the applied force unless corrected for the position.
  • the information about position may be blurred because of nonlinear changes in electrode impedance as the inside surface of the skin makes contact with the electrodes. This is similar to the saturation of light touch receptors and the need to incorporate information from deep touch and nociceptors in biological skin.
  • the characterization experiments described above may produce a rich data set consisting of pairs of input vectors (describing location and components of applied force) and output vectors (voltages related to impedances of the electrode array). These may be used to train neural networks for various tasks. This approach can be used to determine the discriminability of various input conditions or, conversely, to determine the ability to generalize a single parameter such as magnitude of forces applied to different portions of the finger tip.
  • a multi-layer perceptron (MLP) and radial-basis neural network may be used initially because both have proven to be able to approximate any given non-linear relation when a sufficient number of neurons are provided in the hidden layer.
  • Two-point discrimination may likely be possible but may depend critically on the thickness and viscoelastic properties of the skin. It may be feasible to employ algorithms known as neural networks that may function similar to those embodied in the nervous system in order to identify the nature of the contact state in terms of feature of contacted objects and spatiotemporal distribution of contact forces. That is, neural networks can be trained by learning to respond in a useful manner to those features of any stimulus that can be discriminated, as would be obvious to one normally skilled in the art. Active feature extraction is described in more detail in FIGS. 10-15 and related text. Enhancement of Dynamic Range of Force Sensing
  • the dynamic range of forces that can be measured usefully by Impedance Sensing depends on the rate at which space 2 over a given electrode 6 tends to be occluded by increasing force applied to the overlying skin 16.
  • both the core and the skin are smooth, there may be a tendency for the measured electrical impedance of the sensing electrode to rise abruptly and to saturate when the skin is pressed against it, forming a tight seal.
  • the useful dynamic range of sensing can be greatly extended.
  • the inner surface of skin 16 can include asperities 18 molded into the contour of the skin, for example, by forming skin 16 in an injection mold whose corresponding surface is a negative of the desired pattern of asperities 18.
  • Other methods of fabrication of textures would be obvious to one normally skilled in the art, including photolithography, incorporation of soluble particles, plasma etching, etc.
  • Related U.S. Patent Application Serial No. 1 1/692,718 teaches that it may be desirable to have the inside surface of the skin patterned with "bumps and/or ridges".
  • FIG. 5A illustrates one advantageous embodiment of a useful surface textural feature consisting of a repeated pattern of closely spaced asperities 18, each of which is a pyramidal or columnar protrusions from the inner surface of skin 16. Such a pattern tends to leave channels of conductive fluid on the surface of core 1 and electrodes 6, which channels may be gradually compressed and narrowed with increasing compressive force applied to the skin, as illustrated in cross-sectional views in FIGS. 5B and 5C.
  • FIG. 5A illustrates one advantageous embodiment of a useful surface textural feature consisting of a repeated pattern of closely spaced asperities 18, each of which is a pyramidal or columnar protrusions from the inner surface of skin 16.
  • Such a pattern tends to leave channels of conductive fluid on the surface of core 1 and electrodes 6, which channels may be gradually compressed and narrowed with increasing compressive force applied to the skin, as illustrated in cross-sectional views in FIGS. 5B and 5C.
  • FIG. 6 illustrates a wide dynamic range of input normal forces (log scale on abscissa) that result in a wide dynamic range of output electrode impedance (log scale on ordinate) for a pattern of cylindrical asperities 0.25mm tall, 0.25mm in diameter and 0.5mm apart in a regular, square grid.
  • the dynamic range can be extended by texturing the inner surface of the elastomeric skin.
  • a suitable sensor core can be fabricated by creating a negative mold of the desired core shape, advantageously using a relatively soft material such as machinist's wax.
  • Components that need to be present on the surface of the mold can be affixed in the desired locations by pressing them onto the surface of the mold. Any desired mechanical or electrical connections from those components can be made to electronic circuits or connector pins in the open mold. All of the components and their interconnections are then embedded in the core material that is poured into the mold around the components and cured in place.
  • High density polyurethane can be used to form the core. This method lends itself well to rescaling the tactile array for different applications, changing the curvature of its surface, and/or changing the number and distribution of electrode contacts.
  • FIG. 7 illustrates the signals produced by a pressure sensor 10 connected to the incompressible fluid occupying space 2 of a sensor assembly 31 as it contacts a flat, hard surface with different patterns of force and motion.
  • the output of pressure sensing subsystem 210 reflecting P D c is provided as a function of time during various contact events.
  • the output of vibration sensing subsystem 205 reflecting P A c is provided as a function of time, along with the normal and tangential reaction forces (FIG. 7F) recorded by a commercial force-plate under the surface being contacted. Friction-induced vibrations in skin 16 are sensed as fluctuations in fluid-pressure by pressure sensor 10 having a bandwidth of approximately 0-1000Hz.
  • An exemplary embodiment included a commercially available pressure transducer (Honeywell Model #40PC015G1 A) and based on the vibration sensitivity of the human finger an analog first-order low-pass filter with center frequency at 1000 Hz and a digital sampling rate of 2500 samples/sec were used.
  • Fidelity of transmission of acoustic signals is a concern when measuring dynamic fluctuations in pressure from a signal source at a remote location.
  • Important parameters to consider are the geometries of pathways involved between the signal source and recording site as well as the wavelengths of signals being recorded.
  • the wavelength of a signal ( ⁇ ) can be calculated from the speed of sound in a media (c) and the frequency of the signal (f) from the following formula: 1 « ⁇ . Given the inherently long wavelengths of these frequencies in candidate
  • pressure sensor 10 provides easily detected audio frequency signals when skin 16 starts to slip over a contact object.
  • Dynamic information correlated with slip has been shown to lie in frequencies between 50- 700 Hz using this approach. This correlates well with what is known of biological slip detection mechanisms. Presence of frequency content within this band is not unique to slip and is also common to other dynamic events such as shock or contact. However, slip events are unique in that they have significantly less spectral information in low frequency bands (0-50Hz) with respect to these other dynamic events, as shown in FIG. 7 (A), (B) and (D). In experiments conducted by the inventors, the absence of low-frequency vibrations combined with the presence of high-frequency vibrations tended to occur only during slip when contacting passive objects. Texture Discrimination from Friction-Induced Vibrations
  • the sensor detects different spectral patterns as it is slid across surfaces of different texture.
  • the sensor is capable of performing texture discrimination tasks when used in conjunction with mechanical means to slide the sensor over textures to be discriminated.
  • Textures have been shown to provide characteristic, repeatable, and identifiable frequency patterns when observed visually on short-time Fourier transform spectrograms or when listened to acoustically.
  • One such signal processing method for extracting this in an automated fashion is to use a short-time Fourier transform with a wide time window. This wide time window is desired to improve the frequency resolution of this transform and improve the ability to discriminate between different textures.
  • a regular pattern of ridges 17 similar to those found on human fingers or bumps similar to those found on the fingerpads of raccoons can be incorporated onto the exterior of the elastic skin as depicted in FIG. 9. This would permit for the enhancement of slip detection as well as improve the spectral content used for the determination of texture. This is accomplished due to the inherent structure of elastic structures in this shape. As illustrated in FIG. 9 (A) and (B), the cantilever structure of the elastic ridges 17 allows for energy to be stored in the bending of these structures as tangential movements are applied. This stored elastic potential produces a force at the contact surface which is maintained by the friction between the ridges 17 and the object being contacted.
  • Impedance Sensing depends on the conductivity of the weakly conductive fluid, which tends to vary directly with the temperature of the fluid.
  • the ambient core temperature detected as T D c can be measured and stabilized via a feedback control algorithm that adjusts the power dissipation of electronic components 9.
  • Pressure Sensing of vibrations produced by slippage between sensor assembly 100 and the contacting object depend on the location, extent and force of contact between them. These mechanical factors can all be extracted from the Impedance Sensing and used in a feedback control scheme to adjust the nature of the contact to standardized values, much as humans tend to standardize their exploratory movements when trying to identify the texture of an unknown surface.
  • Temperature Sensing of fluctuations produced by contact with an external object depend on the location, extent and force of contact between sensor assembly 100 and the contacting object. These mechanical factors can all be extracted from the Impedance Sensing and used in a feedback control scheme to adjust the nature of the contact to standardize values, much as humans tend to standardize their exploratory movements when trying to identify an unknown object.
  • FIG. 10 provides a block diagram of a complete system 1000 for control and sensing in a multi-articulated manipulator, in accordance with an exemplary embodiment of the present disclosure.
  • a manipulator can include one or more sensor assemblies 100 and one or more powered actuators 600 responsive to a controller 400.
  • Sensor signals from conditioning circuitry 9 in each sensor assembly 100 is transmitted to feature extraction means 300 and may be used to provide sensory feedback to controller 400 according to the requirements of various applications described below.
  • the functions performed by controller 400 are partially or wholly under the command of an external operator 900 which may be a human operator or other source of commands such as would be generated in an autonomous robot.
  • sensory information from feature extractor 300 may be advantageously provided to operator 900 by one or more of the many haptic display interfaces now available or under development, including tactors for generating force, vibration and temperature stimuli on an innervated skin surface of the operator 900.
  • sensor assembly 100 produces broadband responses to slip and texture that are similar to the signals available to the nervous system from biological tactile receptors, it enables a particularly realistic haptic display as follows.
  • the signals produced by vibration sensing subsystem 205 and pressure sensing subsystem 210 can be combined and applied more or less directly onto the skin of a human operator 800 with a haptic display 700 in order to produce realistic illusions of contact, slip and texture. This would be an advance over vibrotactile information currently delivered from tactors in other applications which typically stimulate at a single frequency with modulated amplitude.
  • FIG. 7 and 8 illustrated that slip and texture vibrations are a combination of multiple frequency components and amplitudes. Presenting some or all of the complete set of force, vibration and temperature information sensed by biomimetic sensor assembly 100 to the skin of an operator 900 via a multimodal haptic display 700 would produce more realistic illusions of contact, slip and texture. For human user interaction such as telerobotics and prosthetics, this would be expected to improve haptic perception and dexterous manipulation of objects.
  • Stabilizing a grip may be a function whose requirements and natural strategies are starting to be well understood.
  • the grip stability may be affected by an object's size and shape, its mass and weight distribution, and by the coefficient of friction between the fingertips and surface of the object.
  • the central nervous system usually may adjust the grip force so that the friction force developed between the fingertips and the object surface may have a small margin over the external forces that would otherwise cause the object to slip.
  • This strategy may energetically be efficient and suitable for manipulating delicate objects that might be crushed, but it demands continuous tactile sensing and adjustment of grip forces according to the perceived properties of the gripped object.
  • Each finger's grip force may be adjusted independently based on the sensory information from that finger only and on the local conditions in terms of weight distribution and friction. At least some of this adjustment may occur so rapidly that it appears to be mediated reflexively in the spinal cord rather than via the brain. This is important for prosthetic limbs because it suggests that tactile information can serve a useful function even if communication channels to provide conscious perception of touch to the operator remain nonexistent or primitive, as they are now. Algorithms for the automatic adjustment of grip using biomimetic strategies are likely to be valuable also in telerobotic and purely robotic manipulators.
  • the assumed friction coefficient can be combined with an unnecessarily high safety factor to maintain grasp on objects with a wide range of friction properties, which is undesirable for the handling of fragile objects.
  • a secondary method for controlling grip can be implemented by using the slip-detection derived from feature extraction 300 circuitry to constantly check for slip and adjust this static coefficient when slip is detected as depicted in the flowchart in FIG. 12.
  • This active method of reevaluating the static coefficient can reduce errors that may have occurred due to noise in sensing components and errors in evaluating the static friction coefficient. Because this strategy is more effective at maintaining grip the safety factor used in grip force from the equation above can be reduced which is advantageous for reducing the power required in the gripping actuator.
  • An additional method of controlling grip would be to slowly reduce contact force while holding an object and make small increases in force when a slip is detected as depicted in the flow-chart in FIG. 13. This would allow for further improvements in power savings of the mechanical actuator, as the applied grip force would not deviate far from the needed grip force. This is particularly valuable if the actuator is producing more grip force than is needed to maintain grasp.
  • the grip controller is able to continuously survey if it is possible to maintain grasp of an object with a lower grip force yet maintain grasp when grip force becomes critical, thus reducing power required in the gripping actuator.
  • This also offers the additional advantage of not needing to determine the static friction coefficient as well as not requiring any sensors to detect normal and shear forces as described in the methods presented above.
  • this 60-80ms delay as found in the biological system can be used to determine the confidence in slip as detected using the aforementioned methods with no loss in performance when compared to the gold standard of biological reaction times for grip adjustments. Additional methods to accomplish this digitally would to be to use short-time Fourier transforms to analyze the frequency content. When converting to the frequency domain with a time window of this size the frequency resolution is also sufficient to distinguish between low-frequency content and high-frequency content as describe above and required for accurate determination of slip. The ability to automate the detection of slip would allow for advanced biomimetic grip control mechanisms.
  • an object can be located using robotic vision, proximity sensing, or through other means and a mechanical actuator 500 with the sensor assembly 100 attached can be moved towards the object until a contact is detected.
  • Utilizing force and contact information extracted from impedance data using feature extraction 300 commands can be delivered to the controller 400 such that the desired contact force and location are optimized in the sensor assembly 100.
  • the manipulator 500 and sensor assembly 100 can be stroked along the surface while maintaining desired contacting forces and locations while recording spectral content from the pressure sensor 10.
  • spectral analysis can be performed on the recorded content and it can be compared to a library of materials for texture identification.
  • the sensor assembly 100 If the sensor assembly 100 is heated, and contacted with an external object, its temperature will change appropriately with the mass, temperature, contact surface area, thermal conductivity and heat capacity of the object.
  • One method of heating the finger would be through the use of a heater. These tend to be bulky and would require a controller for proper operation. Instead of using a heater, the power dissipation energy from the electronics of the control board, 9, could be used to heat the finger.
  • the heat dissipated by the microcontroller 230 depends on its clock frequency and duty cycle of active use as opposed to "sleep states". By utilizing a feedback signal from the thermistor 15, controller 400 can adjust the clock frequency or operation duty cycle of microcontroller 230 to heat the finger to the desired temperature. By keeping track of the amount of energy so applied, it is possible to estimate the ambient temperature around sensor assembly 100 according to principles of thermodynamics that would be obvious to one ordinarily skilled in the art.
  • FIG. 15 illustrates an algorithm for determining thermal properties of objects based on the combination of temperature and force sensing capabilities of exemplary embodiments of the present disclosure.
  • a sensor assembly e.g., sensor assembly 100 of FIG. 1
  • controller 400 adjusts the amount of power dissipated as heat by microcontroller 230 by changing its clock frequency or duty cycle. In the event that the measured temperature deviates from the desired value, a negative feedback loop within the heat control software will compensate for this to adjust the temperature back to the desired value.
  • controller 400 initiates an exploratory movement by sending commands to actuators 600.
  • Information about the timing and nature of the contact with the object is obtained from impedance sensing electrodes 6 in order to adjust actuators 600 so that the contact conforms to a standardized exploratory behavior for determining thermal properties of objects. This behavior will generally require that the contact be centered over thermistor 15 and that normal force of contact be sufficient to displace fluid in space 2 allowing skin 16 to make firm contact directly with thermistor 15.
  • embodiments and/or portions of embodiments of the present disclosure can be implemented in/with computer- readable storage media (e.g., hardware, software, firmware, or any combinations of such), and can be distributed and/or practiced over one or more networks. Steps or operations (or portions of such) as described herein, including processing functions to derive, learn, or calculate formula and/or mathematical models utilized and/or produced by the embodiments of the present disclosure, can be processed by one or more suitable processors, e.g., central processing units ("CPUs) implementing suitable code/instructions in any suitable language (machine dependent on machine independent).
  • CPUs central processing units

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Transplantation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Vascular Medicine (AREA)
  • Cardiology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Human Computer Interaction (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Orthopedic Medicine & Surgery (AREA)
  • Manipulator (AREA)

Abstract

L'invention concerne des capteurs tactiles qui imitent le bout d'un doigt humain et ses récepteurs de contact. Les composants mécaniques sont similaires à un bout de doigt, avec un noyau rigide entouré d'un fluide faiblement conducteur contenu à l'intérieur d'une peau élastomérique. Les propriétés déformables du doigtier peuvent être utilisées comme élément d'un processus de transduction. De multiples électrodes peuvent être montées sur la surface du noyau rigide et connectées à un ensemble de circuits de mesure d'impédance à l'intérieur du noyau. Des forces externes déforment le trajet du fluide autour des électrodes, ce qui donne un modèle distribué de variations d'impédance contenant des informations relatives à ces forces et aux objets qui les ont appliquées. La présente invention concerne également des stratégies permettant d'extraire des caractéristiques relatives aux entrées mécaniques et d'utiliser ces informations pour une commande de préhension réflexe. L'invention concerne la commande de la force de préhension dans un élément prosthétique qui reçoit des informations en retour sensorielles. Des capteurs de pression peuvent fournir un retour sensoriel en mesurant les microvibrations dues au frottement de glissement.
PCT/US2009/039354 2008-04-02 2009-04-02 Mesure fiable des vibrations induites par le frottement de glissement pour la détection tactile biomimétique WO2009124209A1 (fr)

Applications Claiming Priority (10)

Application Number Priority Date Filing Date Title
US4186808P 2008-04-02 2008-04-02
US4186508P 2008-04-02 2008-04-02
US4186708P 2008-04-02 2008-04-02
US4186108P 2008-04-02 2008-04-02
US61/041,865 2008-04-02
US61/041,868 2008-04-02
US61/041,867 2008-04-02
US61/041,861 2008-04-02
US4218408P 2008-04-03 2008-04-03
US61/042,184 2008-04-03

Publications (1)

Publication Number Publication Date
WO2009124209A1 true WO2009124209A1 (fr) 2009-10-08

Family

ID=41135942

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2009/039354 WO2009124209A1 (fr) 2008-04-02 2009-04-02 Mesure fiable des vibrations induites par le frottement de glissement pour la détection tactile biomimétique
PCT/US2009/039357 WO2009124211A1 (fr) 2008-04-02 2009-04-02 Améliorations de la fonction d'un capteur tactile biomimétique

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/US2009/039357 WO2009124211A1 (fr) 2008-04-02 2009-04-02 Améliorations de la fonction d'un capteur tactile biomimétique

Country Status (1)

Country Link
WO (2) WO2009124209A1 (fr)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8033189B2 (en) * 2005-12-28 2011-10-11 Honda Motor Co., Ltd. Robot skin
US8181540B2 (en) 2006-03-28 2012-05-22 University Of Southern California Measurement of sliding friction-induced vibrations for biomimetic tactile sensing
US8272278B2 (en) 2007-03-28 2012-09-25 University Of Southern California Enhancements to improve the function of a biomimetic tactile sensor
WO2020128509A1 (fr) * 2018-12-21 2020-06-25 Imperial College Of Science, Technology And Medicine Capteur
CN114585479A (zh) * 2019-10-25 2022-06-03 聪慧公司 检测从机器人紧抓的滑移
US11969880B2 (en) 2019-10-25 2024-04-30 Dexterity, Inc. Detecting robot grasp of very thin object or feature

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10444887B2 (en) 2016-09-20 2019-10-15 Cypress Semiconductor Corporation Force sensing
US20210293643A1 (en) * 2018-07-05 2021-09-23 The Regents Of The University Of Colorado, A Body Corporate Multi-Modal Fingertip Sensor With Proximity, Contact, And Force Localization Capabilities
CN110108751B (zh) * 2019-06-11 2024-05-14 清华大学深圳研究生院 一种可测量热导率和热扩散率的触觉传感器及测量方法
US11408788B2 (en) 2020-03-31 2022-08-09 Toyota Research Institute, Inc. Variable geometry and stiffness control for fluid filled sensor
CN116749217A (zh) * 2023-05-12 2023-09-15 苏州大学 一种仿生触觉机构、机器人灵巧手及机器人

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030051561A1 (en) * 2001-09-14 2003-03-20 Weiss Roger E. Flexible tactile sensor
US20050234292A1 (en) * 2003-08-25 2005-10-20 Faulkner Roger W Hydraulically driven vibrating massagers
US20050239191A1 (en) * 2002-06-19 2005-10-27 Koninklijke Philips Electronics N.V. Tactile device
US20060175770A1 (en) * 2000-04-20 2006-08-10 Ball Burnishing Machine Tools Ltd. Mechanical coupling devices
US20070227267A1 (en) * 2006-03-28 2007-10-04 Alfred E. Mann Institute For Biomedical Engineering At The Univ. Of Southern California Biomimetic Tactile Sensor
US20070265515A1 (en) * 2004-07-13 2007-11-15 Mark Brister Transcutaneous analyte sensor

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5631861A (en) * 1990-02-02 1997-05-20 Virtual Technologies, Inc. Force feedback and texture simulating interface device
US5604314A (en) * 1994-10-26 1997-02-18 Bonneville Scientific Incorporated Triaxial normal and shear force sensor

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060175770A1 (en) * 2000-04-20 2006-08-10 Ball Burnishing Machine Tools Ltd. Mechanical coupling devices
US20030051561A1 (en) * 2001-09-14 2003-03-20 Weiss Roger E. Flexible tactile sensor
US20050239191A1 (en) * 2002-06-19 2005-10-27 Koninklijke Philips Electronics N.V. Tactile device
US20050234292A1 (en) * 2003-08-25 2005-10-20 Faulkner Roger W Hydraulically driven vibrating massagers
US20070265515A1 (en) * 2004-07-13 2007-11-15 Mark Brister Transcutaneous analyte sensor
US20070227267A1 (en) * 2006-03-28 2007-10-04 Alfred E. Mann Institute For Biomedical Engineering At The Univ. Of Southern California Biomimetic Tactile Sensor

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8033189B2 (en) * 2005-12-28 2011-10-11 Honda Motor Co., Ltd. Robot skin
US8181540B2 (en) 2006-03-28 2012-05-22 University Of Southern California Measurement of sliding friction-induced vibrations for biomimetic tactile sensing
US8272278B2 (en) 2007-03-28 2012-09-25 University Of Southern California Enhancements to improve the function of a biomimetic tactile sensor
WO2020128509A1 (fr) * 2018-12-21 2020-06-25 Imperial College Of Science, Technology And Medicine Capteur
CN114585479A (zh) * 2019-10-25 2022-06-03 聪慧公司 检测从机器人紧抓的滑移
EP4048480A4 (fr) * 2019-10-25 2023-11-15 Dexterity, Inc. Détection de glissement à partir d'une préhension de robot
US11969880B2 (en) 2019-10-25 2024-04-30 Dexterity, Inc. Detecting robot grasp of very thin object or feature

Also Published As

Publication number Publication date
WO2009124211A1 (fr) 2009-10-08

Similar Documents

Publication Publication Date Title
US8181540B2 (en) Measurement of sliding friction-induced vibrations for biomimetic tactile sensing
US8272278B2 (en) Enhancements to improve the function of a biomimetic tactile sensor
WO2009124209A1 (fr) Mesure fiable des vibrations induites par le frottement de glissement pour la détection tactile biomimétique
US7878075B2 (en) Biomimetic tactile sensor for control of grip
Chen et al. Tactile sensors for friction estimation and incipient slip detection—Toward dexterous robotic manipulation: A review
Wettels et al. Biomimetic tactile sensor array
EP2010868B1 (fr) Capteur tactile biomimetique
Fishel et al. A robust micro-vibration sensor for biomimetic fingertips
Wettels et al. Multimodal tactile sensor
Su et al. Use of tactile feedback to control exploratory movements to characterize object compliance
da Silva et al. A strain gauge tactile sensor for finger-mounted applications
Edin et al. Bio-inspired sensorization of a biomechatronic robot hand for the grasp-and-lift task
JP5187856B2 (ja) 触覚センサ
Dahiya et al. Tactile sensing for robotic applications
Ke et al. Fingertip tactile sensor with single sensing element based on FSR and PVDF
Fishel Design and use of a biomimetic tactile microvibration sensor with human-like sensitivity and its application in texture discrimination using Bayesian exploration
WO2009127071A1 (fr) Système pour détecter et afficher une souplesse et une force
Edin et al. Bio-inspired approach for the design and characterization of a tactile sensory system for a cybernetic prosthetic hand
Goethals Tactile feedback for robot assisted minimally invasive surgery: an overview
Wettels et al. Biomimetic tactile sensor for control of grip
Wettels Biomimetic tactile sensor for object identification and grasp control
Atieh Design, modeling, fabrication and testing of a piezoresistive-based tactile sensor for minimally invasive surgery applications
Dahiya et al. Biomimetic tactile sensing
Jamali et al. Texture recognition by tactile sensing
Kuehn et al. A fingertip-shaped tactile sensor with machine-learning-based sensor-to-information processing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09726839

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09726839

Country of ref document: EP

Kind code of ref document: A1