USRE45578E1 - Tactile sensor using elastomeric imaging - Google Patents

Tactile sensor using elastomeric imaging Download PDF

Info

Publication number
USRE45578E1
USRE45578E1 US14/045,594 US201314045594A USRE45578E US RE45578 E1 USRE45578 E1 US RE45578E1 US 201314045594 A US201314045594 A US 201314045594A US RE45578 E USRE45578 E US RE45578E
Authority
US
United States
Prior art keywords
skin
light
fluorescent
elastomer
reflective
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/045,594
Inventor
Edward Adelson
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Massachusetts Institute of Technology
Original Assignee
Massachusetts Institute of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Massachusetts Institute of Technology filed Critical Massachusetts Institute of Technology
Priority to US14/045,594 priority Critical patent/USRE45578E1/en
Application granted granted Critical
Publication of USRE45578E1 publication Critical patent/USRE45578E1/en
Assigned to NATIONAL SCIENCE FOUNDATION reassignment NATIONAL SCIENCE FOUNDATION CONFIRMATORY LICENSE (SEE DOCUMENT FOR DETAILS). Assignors: MASSACHUSETTS INSTITUTE OF TECHNOLOGY
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12Fingerprints or palmprints
    • G06V40/13Sensors therefor
    • G06V40/1318Sensors therefor using electro-optical elements or layers, e.g. electroluminescent sensing
    • G06K9/00
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01LMEASURING FORCE, STRESS, TORQUE, WORK, MECHANICAL POWER, MECHANICAL EFFICIENCY, OR FLUID PRESSURE
    • G01L1/00Measuring force or stress, in general
    • G01L1/24Measuring force or stress, in general by measuring variations of optical properties of material when it is stressed, e.g. by photoelastic stress analysis using infrared, visible light, ultraviolet
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/117Identification of persons
    • A61B5/1171Identification of persons based on the shapes or appearances of their bodies or parts thereof
    • A61B5/1172Identification of persons based on the shapes or appearances of their bodies or parts thereof using fingerprinting
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01PMEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
    • G01P5/00Measuring speed of fluids, e.g. of air stream; Measuring speed of bodies relative to fluids, e.g. of ship, of aircraft
    • G01P5/02Measuring speed of fluids, e.g. of air stream; Measuring speed of bodies relative to fluids, e.g. of ship, of aircraft by measuring forces exerted by the fluid on solid bodies, e.g. anemometer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03547Touch pads, in which fingers can move on a surface
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12Fingerprints or palmprints
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/14Coupling media or elements to improve sensor contact with skin or tissue
    • A61B2562/146Coupling media or elements to improve sensor contact with skin or tissue for optical coupling
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/0179Display position adjusting means not related to the information to be displayed
    • G02B2027/0187Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04104Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
    • G06K9/00026
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12Fingerprints or palmprints
    • G06V40/1335Combining adjacent partial images (e.g. slices) to create a composite input or reference pattern; Tracking a sweeping finger movement
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/141Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/142Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/15Conference systems

Definitions

  • the invention is related to the field of sensors, and in particular to tactile sensors.
  • 2-D tactile sensors have been described in the art.
  • an array of individual elements change some electrical property, such as resistance or capacitance, in response to pressure.
  • the electrical changes are sensed and conveyed via wires or other electronic means to the controller or user.
  • Another type of tactile sensor is optical. Some optical properly such as luminance or reflectance changes as a result of pressure, and a light sensing system detects and conveys the signal to the controller or user.
  • a tactile sensor For an application such as a robot fingerpad, there are a number of properties that are desired in a tactile sensor. It should have high resolution (be able to make fine spatial discriminations), have high sensitivity (be able to detect small variations in pressure), and be compliant (able to elastically deform in response to pressure).
  • the tactile sensor should be manufacturable with reasonably large areas. It should be easily manufactured using inexpensive materials. It has been impossible to achieve all of these goals in a single sensor.
  • a tactile sensor including a photosensing structure, a volume of elastomer that is capable of transmitting an image, and a reflective skin covering the volume of elastomer.
  • the reflective skin is illuminated through the volume of elastomer by one or more light sources, and has particles that reflect light incident on the reflective skin from within the volume of elastomer.
  • the reflective skin is geometrically altered in response to pressure applied by an entity touching the reflective skin, the geometrical alteration causing localized changes in the surface normal of the skin and associated localized changes in the amount of light reflected from the reflective skin in the direction of the photosensing structure.
  • the photosensing structure is positioned to receive a portion of the reflected light in the form of an image, the image indicating one or more features of the entity producing the pressure.
  • FIG. 1 is a schematic diagram illustrating the elements of a sensor comprising a clear elastomer, a reflective skin, a light source, and a camera in accordance with the invention
  • FIG. 2 is a photograph illustrating a slab of a clear elastomer covered with a skin containing fine gold-colored flakes, pressed against a twenty dollar bill;
  • FIG. 3 is a photograph illustrating a slab of a clear elastomer covered with a skin containing fine gold-colored flakes, pressed against a sea urchin shell;
  • FIGS. 5A and 5B are schematic diagrams illustrating the elements of an imaging system using diffused light and edge illumination in accordance with the invention.
  • FIGS. 6A-6B are schematic diagrams illustrating the technique used in accordance with the invention to measure deformation and shear;
  • FIGS. 7A and 7B are schematic diagrams illustrating the various tactile sensor arrangements used in accordance with the invention.
  • FIG. 8 is a schematic diagram illustrating a structure to reconstruct 3D shape of an object
  • FIG. 10 is a schematic diagram illustrating a contact image sensor (CIS) linear array being used directly in a tactile sensor structure in accordance with the invention
  • FIG. 11 is a schematic diagram illustrating a fabric covering on the reflective skin.
  • FIG. 12 is a schematic diagram illustrating the use hair or whiskers on the inventive tactile sensor structure.
  • FIG. 13 is a schematic diagram illustrating the elements of a rolling scanner.
  • the invention provides a new approach to making tactile sensors that attain high sensitivity, high spatial resolution, and low cost.
  • it can be built in a compliant form, so that a robot finger incorporating this sensor can deform elastically in depth, following the profile of the object being manipulated, thereby allowing good control.
  • An exemplary embodiment of the invention is made from slab of clear elastomer, 3 , supported by a rigid sheet 7 made of glass or other rigid clear material.
  • the surface of the elastomer is coated with a reflective layer 2 , referred to as the “skin,” which is made, for example, from an elastomeric paint comprising metallic powder embedded in an elastomeric material.
  • the skin has an inner surface (facing the elastomer) and an outer surface (facing the outside world).
  • Light from an illuminator 5 passes through the rigid support 7 and the clear elastomer 3 and strikes the reflective skin 2 .
  • an object such as a finger 1 applies pressure to the outer surface of the skin, it causes a distortion of the skin.
  • a change in the surface normal leads to a change in the amount of light reflected in a given direction.
  • a camera 4 views the inner skin and records the reflected light as an image.
  • the image pattern is the result of the pressure pattern, and thus conveys information about the pressure pattern. Note that the image pixel values do not directly encode pressure. They encode the angle of surface normal, which is indicative of the spatial variation of the pressure.
  • the clear elastomer can be composed of a wide range of materials including but not limited to silicone rubber, polyurethane, thermoplastic elastomer, plastisol, natural rubber, polyisoprene, polyvinyl chloride, or a mixture thereof.
  • the hardness of the elastomer as measured on the Shore A scale, will range between 5 and 90.
  • the reflective skin is also elastomeric, and will typically have a hardness that is equal to or greater than that of the clear elastomer body.
  • the reflective skin may be comprised of the same material as the body, or of a different material.
  • the sensor skin can be made by adding reflective particles to the elastomer when it is in a liquid state, via solvent or heat, or before curing. This makes a reflective paint that can be attached to the surface by standard coating techniques such as spraying or dipping.
  • the skin may be coated directly on the surface of the bulk elastomer, or it may be first painted on a smooth medium such as glass and then transferred to the surface of the bulk material and bound there.
  • the particles (without binder) can be rubbed into the surface of the bulk elastomer, and then bound to the elastomer by heat or with a thin coat of material overlaid on the surface. Also, it may be possible to evaporate, precipitate, sputter, other otherwise attach thin films to the surface.
  • the reflective particles in the skin may reflect light directionally or non-directionally. If the particles reflect light uniformly in all directions regardless of the light's angle of incidence, the resulting skin will behave like a Lambertian surface, which is entirely non-directional. Titanium dioxide powder, as is used in white paint, leads to a largely Lambertian reflectance. If the reflective particles are comprised of tine metal flakes, and if these flakes tend to be aligned with each other, then the skin will reflect light directionally, meaning that, for a given angle of incident light, there will be a non-uniform distribution of reflected light. If the metal flakes are flat and mirror-like, and if they are well aligned with each other, the distribution of reflected light will be highly directional.
  • the distribution of reflected light will be moderately directional, with an appearance resembling sandblasted metal.
  • Directional reflectance can also be obtained with flakes of other materials such as mica.
  • pigments comprising flakes covered with multilayer interference coatings that can have different directionality for different wavelengths of light.
  • Skin with highly directional reflectance illuminated by a highly directional light source yields a device that is sensitive to small variations in pressure. This sensitivity can be increased by recording the skin's image in its resting state, and using this as a baseline image that is subtracted from images recorded when pressure is applied to the skin. Softer elastomers lead to devices that are more sensitive to low amplitude pressure patterns.
  • FIG. 2 shows an image obtained with a skin containing gold-colored bronze flakes that are directionally selective.
  • the slab was coated with a thin skin containing fine gold-colored flakes.
  • the slab was pressed against a twenty dollar bill, and the skin was viewed through the glass and the elastomer. Due to the way that bills are printed, the printing on a twenty dollar bill has a raised relief.
  • the skin is pressed against the bill, its surface deforms in accordance with the bill's relief. The deformation causes a variation in the amount of light reflected toward the camera, revealing the fine details of the bill's surface in the form of an image.
  • FIG. 3 Another example is shown in FIG. 3 .
  • a slab of clear elastomer was coated with a reflective skin made with bronze flakes and was placed, skin side up, on the platen of a flatbed scanner.
  • a sea urchin shell was pressed against the reflective skin, causing 3D deformation in the contact region 6 .
  • the untouched region 8 remained smooth and the image of this region retained its original smooth appearance.
  • the scanner's internal light source was reflected differently depending on the reflective skin's surface normal, resulting in an image that is recognizable as a shaded relief of the sea urchin shell.
  • the image pixel values do not directly encode pressure. If spatially uniform pressure is applied to the entire skin surface, there will be no change in surface normal and thus no observable variation in the image.
  • the image pixel values depend on surface normal, which in turn depends on the spatial derivative of pressure. Thus, it is the pattern of pressure variation across the surface that is encoded in the image.
  • Pressure can be applied to the skin by a rigid object or a non-rigid object.
  • a non-rigid object such as a fingertip
  • both the object and the skin will deform, and the skin's shape will depend on the balance of pressures that the skin and the object exert on each other.
  • Pressure can also be applied by a liquid or gas. For example, a stream of water striking the skin causes it to deform, and the pattern of deformation is visible in the image. If the skin and the elastomer are made of very soft gel-like materials, and if a froth of soap bubbles is placed in contact with the skin, one can visualize the forces exerted by the soap bubble walls.
  • FIGS. 4A-4D show the elements of the imaging system that can be used in a compact structure such as a robot fingertip.
  • a compact structure such as a robot fingertip.
  • FIG. 4A shows a pair of LEDs 12 and a small camera 14 , which are looking through the rounded piece of elastomer 16 positioned on a rigid support 17 .
  • the skin 18 of the elastomer is reflective, and the camera forms an image of the inner side of the skin.
  • FIG. 4B shows an example wherein the camera is a pinhole camera 20 .
  • FIG. 4C shows a folded path optical system that utilizes a curved mirror 22 which reflects light from the skin to the camera 24 .
  • FIG. 4D shows the case Where the skin is imaged with an endoscope 26 (or the related videoscope, borescope, fiberscope, or the like). This allows the camera 28 to be placed at a distance from the sensor.
  • FIG. 5A shows an exemplary embodiment of a sensor 30 using an extended diffuse source of light 31 .
  • the diffuse light source 31 and camera 36 are positioned on a rigid support 32 .
  • a volume of clear elastomer 34 is positioned on the rigid support 32 .
  • FIG. 5B shows another exemplary embodiment of the invention where one or more light sources 33 illuminates the sensor from a side or edge of the support 32 , preferably made of glass or other clear material.
  • Light will bounce off the back face of the support by total internal reflection, and will also be reflected by the reflective skin surface 38 by ordinary reflection. This will cause many of the light rays to remain within the glass+elastomer volume; these rays will illuminate the reflective skin surface 38 , and the surface can be viewed by the camera 36 .
  • FIGS. 6A-6B show two exemplary embodiments of the invention wherein deformation and shear can be sensed.
  • Deformation in particular, changes in surface normal
  • FIG. 6B shows a case in which the surface 48 exerts shear forces on the elastomer and skin 50 , causing no change in surface normal, but causing a lateral displacement.
  • the skin typically contains a visible microtexture due to the random pattern of reflective particles, and shear causes a displacement of this microtexture. Motion analysis methods can then be used to estimate the shear.
  • FIG. 7A shows a slab of elastomer 136 covered with reflective skin 134 and mounted on lens 146 .
  • An object 138 applies pressure to skin 134 .
  • the focal length of the lens is such that the light rays from light source 140 are refracted to be parallel when striking skin 134 .
  • Camera 144 views the skin through the same lens. The optical properties of the skin 134 as observed by camera 144 will be fairly uniform across the image.
  • FIG. 7B shows an arrangement in which a slab of elastomer 160 is covered by a reflective skin 152 .
  • the elastomer is mounted on a right angle prism 162 .
  • Light source 156 passes through lens 164 , emerging as parallel rays that enter prism 162 and strike skin 152 .
  • An object 154 presses on the skin 152 , causing local variation in surface normal.
  • Camera 158 views the skin through a lens 166 that places the skin at optical infinity for that camera. The optical properties of the skin as observed by the camera will be fairly uniform across the image.
  • the two light sources there are two light sources, 80 and 82 , which illuminate the skin 84 thought the elastomer 79 .
  • the two light sources illuminate the surface from substantially different azimuths, for example one aligned with the x-axis and the other with the y-axis of the slab.
  • the light sources are turned on one after the other and two images are recorded by camera 78 . These two images can then be analyzed in accord with the known methods of photometric stereo to estimate the surface normal and surface height at every position.
  • the two lights 80 and 82 are of different colors, for example red and blue, and if the camera 78 is a color camera, then it is possible to record the two images at the same time in separate color channels.
  • Photometric stereo benefits from the use of additional images. With a color camera it is straightforward to use three light sources and to separate the channels into three images. Alternately, one can use an arbitrary number of light sources if they are turned on one at a time.
  • each light brings out surface normal variation along one axis, but not along the orthogonal axis.
  • the lights can be arranged so that one light reveals the relief that is missed by another light. This makes it possible to distinguish a wide range of surface normals in different directions.
  • the preferred method of using two or more lights is to have them be different colors, so that a color camera will separate the information about the different lights into different color channels.
  • a sensor surface that covers a large area. For example one may require a touchpad that covers an entire desktop. If the device is simply scaled up, then the camera must be placed at a large distance from the surface, making the device undesirably large.
  • One way to ameliorate this problem is to use the methods of folded optics that are used, for example, in many rear projection televisions.
  • Another way is to use a tiled array of cameras, as shown in FIG. 9 .
  • the reflective skin 180 covers the clear elastomer 182 which is mounted on a rigid transparent support 184 .
  • An array of cameras 186 is placed a short distance from the elastomer and skin. The cameras can be arranged so that their image slightly overlap, and these images can be combined into a single large image by standard stitching techniques.
  • a sensor is formed into a cylinder, which can be rolled over the surface of the object.
  • the sensor would look like a brayer or paint roller.
  • a video camera inside the roller is aimed continuously at the portion that is in contact with the surface of the object.
  • the series of images so obtained can be combined into a single image by the same methods that are used to obtain panoramic photographs from a series of smaller photographs.
  • the skin it is advantageous for the skin to have a texture rather than being smooth.
  • the reflective skin has texture and elasticity like the upper layer of human skin, and the clear elastomer beneath the skin has mechanical properties like the deeper layers of human skin. Multiple layers of clear elastomer with different mechanical properties are required to mimic the complex properties of human skin.
  • the reflective skin distorts in response to the mechanical forces applied to it, and this distortion is viewed by a camera looking through the clear elastomer layers.
  • a specific object such as a tire or the sole of a shoe. It is possible to form the tactile sensor into the same shape as this specific object, and with the same hardness or other mechanical properties as this specific object.
  • FIG. 10 shows a strip of elastomer 198 covered with reflective skin 192 and mounted on CIS 194 .
  • object 196 presses on the skin it modifies the surface normal, which modifies the amount of light that will be reflected toward the photosensing elements in that neighborhood.
  • the result is a 1D image that encodes information about the location and amplitude of the pressure variation on the skin.
  • a multitouch touchscreen device is made in conjunction with a flat panel LED display.
  • the LED photosensing responses comprise an image that is indicative of the pattern of pressure on the skin. This image indicates where the user is touching the screen. In addition, because this is an inherently high resolution image, it is possible to detect the fingerprint of the user. This allows each finger of each user to be distinguished.
  • FIG. 11 shows another exemplary embodiment of a tactile sensor 202 in accordance with the invention.
  • a stretchy fabric is placed over the reflective skin.
  • this fabric will produce a surface with desirable mechanical qualities, including the frictional qualities and the ability to withstand the forces of industrial usage.
  • FIG. 11 shows a cross section of a fabric covering 204 that is attached to the reflective skin 206 on the clear elastomer 208 .
  • An object 214 presses on the fabric causing the skin to take on the texture 210 of the fabric.
  • this pattern appears as a gridwork of fibers corresponding to the fabric's construction. Greater pressure leads to a higher contrast fabric pattern, as seen by camera 212 . Local properties of this pattern, including the mean value and the contrast, are indicative of the pressure applied at that location.
  • the device is used to measure fluid flow.
  • FIG. 12 shows hairs or whiskers 220 that are attached to the reflective skin 224 at attachment points 222 .
  • the attachment points are small pads that are rigidly attached to the whiskers.
  • the skin is viewed by a camera, and the variation in surface normal causes a variation in image radiance from point to point. The image indicates the speed and direction of fluid flow across each whisker.
  • An extra layer of elastomer on top of the skin acts as a mechanical lowpass filter.
  • a 1 mm thick layer reduces the resolution to be on the order of 1 mm.
  • a fluorescent pigment can be used in the skin, illuminated by Ultraviolet (UV) light or blacklight. If the blacklight comes at a grazing angle, it can readily reveal variations in surface normal. The material will be fairly close to Lambertian. To reduce interreflections, one would select a surface that appears dark to emitted wavelengths. This principle is true with ordinary light as well. If one is using a Lambertian pigment in the skin, it is better for it to be gray than white, to reduce interreflections.
  • UV Ultraviolet
  • Blacklight or UV can be used to illuminate a fluorescent surface, which would then serve as a diffuse source.
  • a single short flash for instance, recording the instantaneous deformation of an object against the surface
  • multiple periodic (strobed) flashes to capture rapid periodic events or to modulate one frequency down to another frequency.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Force Measurement Appropriate To Specific Purposes (AREA)
  • Position Input By Displaying (AREA)
  • Push-Button Switches (AREA)

Abstract

A tactile sensor includes a photosensing structure, a volume of elastomer capable of transmitting an image, and a reflective skin covering the volume of elastomer. The reflective skin is illuminated through the volume of elastomer by one or more light sources, and has particles that reflect light incident on the reflective skin from within the volume of elastomer. The reflective skin is geometrically altered in response to pressure applied by an entity touching the reflective skin, the geometrical alteration causing localized changes in the surface normal of the skin and associated localized changes in the amount of light reflected from the reflective skin in the direction of the photosensing structure. The photosensing structure receives a portion of the reflected light in the form of an image, the image indicating one or more features of the entity producing the pressure.

Description

PRIORITY INFORMATION
More than one reissue patent application has been filed for the reissue of U.S. Pat. No. 8,411,140, which include U.S. Reissue patent application Ser. No. 13/971,456, filed Aug. 20, 2013, now U.S. Pat. No. Re. 44,856, U.S. Reissue patent application Ser. No. 14/045,620, filed Oct. 3, 2013, now U.S. Pat. No. Re. 45,176, U.S. Reissue patent application Ser. No. 14/045,647, filed Oct. 3, 2013, U.S. Reissue patent application Ser. No. 14/045,668, filed Oct. 3, 2013, and U.S. Reissue patent application Ser. No. 14/045,594, filed Oct. 3, 2013. This application is a continuation reissue of application Ser. No. 13/971,456, which is an application for reissue of U.S. Pat. No. 8,411,140, and thereby claims priority from provisional application Ser. No. 61/073,904 filed Jun. 19, 2008, which is incorporated herein by reference in its entirety.
SPONSORSHIP INFORMATION
This invention was made with government support under grant number BCS-0345805 awarded by the National Science Foundation. The government has certain rights in this invention.
BACKGROUND OF THE INVENTION
The invention is related to the field of sensors, and in particular to tactile sensors.
A variety of 2-D tactile sensors have been described in the art. In a typical sensor, an array of individual elements change some electrical property, such as resistance or capacitance, in response to pressure. The electrical changes are sensed and conveyed via wires or other electronic means to the controller or user. Another type of tactile sensor is optical. Some optical properly such as luminance or reflectance changes as a result of pressure, and a light sensing system detects and conveys the signal to the controller or user.
For an application such as a robot fingerpad, there are a number of properties that are desired in a tactile sensor. It should have high resolution (be able to make fine spatial discriminations), have high sensitivity (be able to detect small variations in pressure), and be compliant (able to elastically deform in response to pressure). The tactile sensor should be manufacturable with reasonably large areas. It should be easily manufactured using inexpensive materials. It has been impossible to achieve all of these goals in a single sensor.
SUMMARY OF THE INVENTION
According to one aspect of the invention, there is provided a tactile sensor including a photosensing structure, a volume of elastomer that is capable of transmitting an image, and a reflective skin covering the volume of elastomer. The reflective skin is illuminated through the volume of elastomer by one or more light sources, and has particles that reflect light incident on the reflective skin from within the volume of elastomer. The reflective skin is geometrically altered in response to pressure applied by an entity touching the reflective skin, the geometrical alteration causing localized changes in the surface normal of the skin and associated localized changes in the amount of light reflected from the reflective skin in the direction of the photosensing structure. The photosensing structure is positioned to receive a portion of the reflected light in the form of an image, the image indicating one or more features of the entity producing the pressure.
According to another aspect of the invention, there is a method of performing tactile sensing. The method includes providing a volume of elastomer capable of transmitting an image, and covering the volume of elastomer with a reflective skin. The reflective skin is illuminated through the volume of elastomer by one or more light sources, and has particles that reflect light incident on the reflective skin from within the volume of elastomer. The method also includes geometrically altering the reflective skin in response to pressure applied by an entity touching the reflective skin, the alteration causing localized changes in the surface normal of the skin and associated localized changes in the amount of light reflected from the reflective skin in the direction of a photosensing structure. The photosensing structure is positioned to receive a portion of the reflected light in the form an image, the image indicating one or more features of the entity producing the pressure.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic diagram illustrating the elements of a sensor comprising a clear elastomer, a reflective skin, a light source, and a camera in accordance with the invention;
FIG. 2 is a photograph illustrating a slab of a clear elastomer covered with a skin containing fine gold-colored flakes, pressed against a twenty dollar bill;
FIG. 3 is a photograph illustrating a slab of a clear elastomer covered with a skin containing fine gold-colored flakes, pressed against a sea urchin shell;
FIGS. 4A-4D are schematic diagrams illustrating the elements of an imaging system that can be used in a compact structure;
FIGS. 5A and 5B are schematic diagrams illustrating the elements of an imaging system using diffused light and edge illumination in accordance with the invention;
FIGS. 6A-6B are schematic diagrams illustrating the technique used in accordance with the invention to measure deformation and shear;
FIGS. 7A and 7B are schematic diagrams illustrating the various tactile sensor arrangements used in accordance with the invention;
FIG. 8 is a schematic diagram illustrating a structure to reconstruct 3D shape of an object;
FIG. 9 is a schematic diagram illustrating a large area high resolution sensor formed by tiling a set of smaller sensors;
FIG. 10 is a schematic diagram illustrating a contact image sensor (CIS) linear array being used directly in a tactile sensor structure in accordance With the invention;
FIG. 11 is a schematic diagram illustrating a fabric covering on the reflective skin; and
FIG. 12 is a schematic diagram illustrating the use hair or whiskers on the inventive tactile sensor structure.
FIG. 13 is a schematic diagram illustrating the elements of a rolling scanner.
DETAILED DESCRIPTION OF THE INVENTION
The invention provides a new approach to making tactile sensors that attain high sensitivity, high spatial resolution, and low cost. In addition, it can be built in a compliant form, so that a robot finger incorporating this sensor can deform elastically in depth, following the profile of the object being manipulated, thereby allowing good control.
An exemplary embodiment of the invention, as shown in FIG. 1, is made from slab of clear elastomer, 3, supported by a rigid sheet 7 made of glass or other rigid clear material. The surface of the elastomer is coated with a reflective layer 2, referred to as the “skin,” which is made, for example, from an elastomeric paint comprising metallic powder embedded in an elastomeric material. The skin has an inner surface (facing the elastomer) and an outer surface (facing the outside world). Light from an illuminator 5 passes through the rigid support 7 and the clear elastomer 3 and strikes the reflective skin 2. When an object such as a finger 1 applies pressure to the outer surface of the skin, it causes a distortion of the skin. Local variations in pressure lead to local variations in the skin's surface normal. A change in the surface normal leads to a change in the amount of light reflected in a given direction. A camera 4 views the inner skin and records the reflected light as an image. The image pattern is the result of the pressure pattern, and thus conveys information about the pressure pattern. Note that the image pixel values do not directly encode pressure. They encode the angle of surface normal, which is indicative of the spatial variation of the pressure.
The clear elastomer can be composed of a wide range of materials including but not limited to silicone rubber, polyurethane, thermoplastic elastomer, plastisol, natural rubber, polyisoprene, polyvinyl chloride, or a mixture thereof. Typically, the hardness of the elastomer, as measured on the Shore A scale, will range between 5 and 90. The reflective skin is also elastomeric, and will typically have a hardness that is equal to or greater than that of the clear elastomer body. The reflective skin may be comprised of the same material as the body, or of a different material.
The sensor skin can be made by adding reflective particles to the elastomer when it is in a liquid state, via solvent or heat, or before curing. This makes a reflective paint that can be attached to the surface by standard coating techniques such as spraying or dipping. The skin may be coated directly on the surface of the bulk elastomer, or it may be first painted on a smooth medium such as glass and then transferred to the surface of the bulk material and bound there. Also, the particles (without binder) can be rubbed into the surface of the bulk elastomer, and then bound to the elastomer by heat or with a thin coat of material overlaid on the surface. Also, it may be possible to evaporate, precipitate, sputter, other otherwise attach thin films to the surface.
The reflective particles in the skin may reflect light directionally or non-directionally. If the particles reflect light uniformly in all directions regardless of the light's angle of incidence, the resulting skin will behave like a Lambertian surface, which is entirely non-directional. Titanium dioxide powder, as is used in white paint, leads to a largely Lambertian reflectance. If the reflective particles are comprised of tine metal flakes, and if these flakes tend to be aligned with each other, then the skin will reflect light directionally, meaning that, for a given angle of incident light, there will be a non-uniform distribution of reflected light. If the metal flakes are flat and mirror-like, and if they are well aligned with each other, the distribution of reflected light will be highly directional. If the metal flakes are rough or irregular, or if there is randomness in their alignment, then the distribution of reflected light will be moderately directional, with an appearance resembling sandblasted metal. Directional reflectance can also be obtained with flakes of other materials such as mica. In addition there are pigments comprising flakes covered with multilayer interference coatings that can have different directionality for different wavelengths of light.
Further still, metal flake powder of aluminum can be used.
Skin with highly directional reflectance illuminated by a highly directional light source yields a device that is sensitive to small variations in pressure. This sensitivity can be increased by recording the skin's image in its resting state, and using this as a baseline image that is subtracted from images recorded when pressure is applied to the skin. Softer elastomers lead to devices that are more sensitive to low amplitude pressure patterns.
FIG. 2 shows an image obtained with a skin containing gold-colored bronze flakes that are directionally selective. A slab of a clear elastomer about 1 cm thick, was mounted on a sheet of glass. The slab was coated with a thin skin containing fine gold-colored flakes. The slab was pressed against a twenty dollar bill, and the skin was viewed through the glass and the elastomer. Due to the way that bills are printed, the printing on a twenty dollar bill has a raised relief. When the skin is pressed against the bill, its surface deforms in accordance with the bill's relief. The deformation causes a variation in the amount of light reflected toward the camera, revealing the fine details of the bill's surface in the form of an image.
Another example is shown in FIG. 3. A slab of clear elastomer was coated with a reflective skin made with bronze flakes and was placed, skin side up, on the platen of a flatbed scanner. A sea urchin shell was pressed against the reflective skin, causing 3D deformation in the contact region 6. The untouched region 8 remained smooth and the image of this region retained its original smooth appearance. The scanner's internal light source was reflected differently depending on the reflective skin's surface normal, resulting in an image that is recognizable as a shaded relief of the sea urchin shell.
The image pixel values do not directly encode pressure. If spatially uniform pressure is applied to the entire skin surface, there will be no change in surface normal and thus no observable variation in the image. The image pixel values depend on surface normal, which in turn depends on the spatial derivative of pressure. Thus, it is the pattern of pressure variation across the surface that is encoded in the image.
Pressure can be applied to the skin by a rigid object or a non-rigid object. In the case of a non-rigid object, such as a fingertip, both the object and the skin will deform, and the skin's shape will depend on the balance of pressures that the skin and the object exert on each other. Pressure can also be applied by a liquid or gas. For example, a stream of water striking the skin causes it to deform, and the pattern of deformation is visible in the image. If the skin and the elastomer are made of very soft gel-like materials, and if a froth of soap bubbles is placed in contact with the skin, one can visualize the forces exerted by the soap bubble walls.
FIGS. 4A-4D show the elements of the imaging system that can be used in a compact structure such as a robot fingertip. In the illustrated embodiment, there is a rounded piece of elastomer, which is mounted on a rigid member.
FIG. 4A shows a pair of LEDs 12 and a small camera 14, which are looking through the rounded piece of elastomer 16 positioned on a rigid support 17. The skin 18 of the elastomer is reflective, and the camera forms an image of the inner side of the skin. FIG. 4B shows an example wherein the camera is a pinhole camera 20. FIG. 4C shows a folded path optical system that utilizes a curved mirror 22 which reflects light from the skin to the camera 24. FIG. 4D shows the case Where the skin is imaged with an endoscope 26 (or the related videoscope, borescope, fiberscope, or the like). This allows the camera 28 to be placed at a distance from the sensor.
FIG. 5A shows an exemplary embodiment of a sensor 30 using an extended diffuse source of light 31. The diffuse light source 31 and camera 36 are positioned on a rigid support 32. A volume of clear elastomer 34 is positioned on the rigid support 32.
To make the structure compact, it may be preferable to introduce the light at the edge of the support. FIG. 5B shows another exemplary embodiment of the invention where one or more light sources 33 illuminates the sensor from a side or edge of the support 32, preferably made of glass or other clear material. Light will bounce off the back face of the support by total internal reflection, and will also be reflected by the reflective skin surface 38 by ordinary reflection. This will cause many of the light rays to remain within the glass+elastomer volume; these rays will illuminate the reflective skin surface 38, and the surface can be viewed by the camera 36. It may be advantageous to use a glass wedge rather than an ordinary sheet of glass. With a wedge (for example, the “Light Wedge” book light) the light reflects of the front and back faces successively, making a larger angle with each bounce. This causes a greater amount of light to exit the wedge at larger distance from the light source.
FIGS. 6A-6B show two exemplary embodiments of the invention wherein deformation and shear can be sensed. Deformation (in particular, changes in surface normal) can be sensed by measuring the change in intensity at each point on the skin 44 produced by a work piece 46, as shown in FIG. 6A. FIG. 6B shows a case in which the surface 48 exerts shear forces on the elastomer and skin 50, causing no change in surface normal, but causing a lateral displacement. The skin typically contains a visible microtexture due to the random pattern of reflective particles, and shear causes a displacement of this microtexture. Motion analysis methods can then be used to estimate the shear.
In some applications it is desirable that the light source and the camera be placed at optical infinity so that the angle of incidence and reflectance are parallel when the device is in its resting state. This causes the devices optical properties to be spatially uniform across the recorded image. FIG. 7A shows a slab of elastomer 136 covered with reflective skin 134 and mounted on lens 146. An object 138 applies pressure to skin 134. The focal length of the lens is such that the light rays from light source 140 are refracted to be parallel when striking skin 134. Camera 144 views the skin through the same lens. The optical properties of the skin 134 as observed by camera 144 will be fairly uniform across the image.
FIG. 7B shows an arrangement in which a slab of elastomer 160 is covered by a reflective skin 152. The elastomer is mounted on a right angle prism 162. Light source 156 passes through lens 164, emerging as parallel rays that enter prism 162 and strike skin 152. An object 154 presses on the skin 152, causing local variation in surface normal. Camera 158 views the skin through a lens 166 that places the skin at optical infinity for that camera. The optical properties of the skin as observed by the camera will be fairly uniform across the image.
In accordance with another exemplary embodiment of the invention, it is desirable to reconstruct the 3-D shape of the deformed surface. In FIG. 8, there are two light sources, 80 and 82, which illuminate the skin 84 thought the elastomer 79. Preferably, the two light sources illuminate the surface from substantially different azimuths, for example one aligned with the x-axis and the other with the y-axis of the slab. The light sources are turned on one after the other and two images are recorded by camera 78. These two images can then be analyzed in accord with the known methods of photometric stereo to estimate the surface normal and surface height at every position. If the two lights 80 and 82 are of different colors, for example red and blue, and if the camera 78 is a color camera, then it is possible to record the two images at the same time in separate color channels. Photometric stereo benefits from the use of additional images. With a color camera it is straightforward to use three light sources and to separate the channels into three images. Alternately, one can use an arbitrary number of light sources if they are turned on one at a time.
The use of multiple lights to get multiple images is useful even when 3D reconstruction is not being performed. Each light brings out surface normal variation along one axis, but not along the orthogonal axis. By using two or more lights, the lights can be arranged so that one light reveals the relief that is missed by another light. This makes it possible to distinguish a wide range of surface normals in different directions. The preferred method of using two or more lights is to have them be different colors, so that a color camera will separate the information about the different lights into different color channels.
It is also possible to use standard stereoscopic techniques as well, in which multiple cameras are placed in different azimuths relative to the skin of the sensor. The techniques set forth herein of sensing changes in luminance are believed to yield better sensitivity and resolution in many applications relative to known techniques. Furthermore, stereopsis can be used in combination with the techniques disclosed herein. The microtexture of the skin and/or the deformation image can be used to establish correspondence.
In some applications it is desirable to make a sensor surface that covers a large area. For example one may require a touchpad that covers an entire desktop. If the device is simply scaled up, then the camera must be placed at a large distance from the surface, making the device undesirably large. One way to ameliorate this problem is to use the methods of folded optics that are used, for example, in many rear projection televisions. Another way is to use a tiled array of cameras, as shown in FIG. 9. The reflective skin 180 covers the clear elastomer 182 which is mounted on a rigid transparent support 184. An array of cameras 186 is placed a short distance from the elastomer and skin. The cameras can be arranged so that their image slightly overlap, and these images can be combined into a single large image by standard stitching techniques.
In another embodiment, a sensor is formed into a cylinder, which can be rolled over the surface of the object. In one illustrative configuration, the sensor would look like a brayer or paint roller. As the roller rolls over the surface, a video camera inside the roller is aimed continuously at the portion that is in contact with the surface of the object. The series of images so obtained can be combined into a single image by the same methods that are used to obtain panoramic photographs from a series of smaller photographs.
Alternatively, the roller could be in a fixed position while the surface of interest was pulled over it. For example, as shown in FIG. 13, an illustrative embodiment of a rolling scanner is a counterfeit banknote detector 300 which has a clear elastomer 310 formed into a cylinder, and a reflective skin 320. The detector 300 also has a slot 330 for feeding a banknote 340. A roller 350, driven by a motor, pulls in the banknote 340 and forms an image of the banknote's surface shape based on an embossed surface of the banknote's paper and/or the raised level of printed ink. A genuine banknote will have a known profile, and a mismatch would indicate a counterfeit banknote.
In some applications it is advantageous for the skin to have a texture rather than being smooth. In some situations one wishes to study the distribution of pressures across a region of human skin. For example, when a skin care product is applied to the skin, the application process produces a certain distribution of pressure on the skin which changes over time. In order to estimate this changing distribution, a sensor can be made that mimics the texture, elasticity, and other properties of human skin. When a skin care product is, for example, wiped across the artificial skin, it causes the skin to distort in a manner similar to that of human skin. The pattern of distortion can be assessed by making a tactile sensor with mechanical properties emulating human skin. This means that the reflective skin has texture and elasticity like the upper layer of human skin, and the clear elastomer beneath the skin has mechanical properties like the deeper layers of human skin. Multiple layers of clear elastomer with different mechanical properties are required to mimic the complex properties of human skin. When a skin care product is applied to this device, the reflective skin distorts in response to the mechanical forces applied to it, and this distortion is viewed by a camera looking through the clear elastomer layers.
In some applications it may be desired to study the distribution of pressure over the surface of a specific object, such as a tire or the sole of a shoe. It is possible to form the tactile sensor into the same shape as this specific object, and with the same hardness or other mechanical properties as this specific object.
In other embodiments, it is not necessary that the image be formed by a camera. Many flatbed scanners use a Contact Image Sensor, or CIS, which is a linear array of lenses and photosensors placed in close contact with the object being scanned. No image forming lens is required. FIG. 10 shows a strip of elastomer 198 covered with reflective skin 192 and mounted on CIS 194. When object 196 presses on the skin, it modifies the surface normal, which modifies the amount of light that will be reflected toward the photosensing elements in that neighborhood. The result is a 1D image that encodes information about the location and amplitude of the pressure variation on the skin.
In another embodiment, a multitouch touchscreen device is made in conjunction with a flat panel LED display. A thin sheet of clear elastomer, covered with a semi-reflective skin, covers the front surface of the display. Most of the light that is emitted by the LED's passes through the skin and is seen by a viewer. A portion of the light is reflected by the skin back toward the LEDs. LEDs have the ability to act as photosensors, and thus can be used to measure the amount of reflected light. Pressure variation on the skin causes local changes in the surface normal, which changes the amount of light reflected toward any given LED in the array. The LED photosensing responses comprise an image that is indicative of the pattern of pressure on the skin. This image indicates where the user is touching the screen. In addition, because this is an inherently high resolution image, it is possible to detect the fingerprint of the user. This allows each finger of each user to be distinguished.
FIG. 11 shows another exemplary embodiment of a tactile sensor 202 in accordance with the invention. A stretchy fabric is placed over the reflective skin. For some applications such a robot manipulator this fabric will produce a surface with desirable mechanical qualities, including the frictional qualities and the ability to withstand the forces of industrial usage. FIG. 11 shows a cross section of a fabric covering 204 that is attached to the reflective skin 206 on the clear elastomer 208. An object 214 presses on the fabric causing the skin to take on the texture 210 of the fabric. In the case of a woven fabric this pattern appears as a gridwork of fibers corresponding to the fabric's construction. Greater pressure leads to a higher contrast fabric pattern, as seen by camera 212. Local properties of this pattern, including the mean value and the contrast, are indicative of the pressure applied at that location.
In another exemplary embodiment, the device is used to measure fluid flow. FIG. 12 shows hairs or whiskers 220 that are attached to the reflective skin 224 at attachment points 222. The attachment points are small pads that are rigidly attached to the whiskers. When fluid flows across the whiskers, it causes the whiskers to tilt, causing the attachment pads to tilt, causing the reflective skin's surface normal to change. The skin is viewed by a camera, and the variation in surface normal causes a variation in image radiance from point to point. The image indicates the speed and direction of fluid flow across each whisker.
There are applications for which high resolution is not needed and not desirable. An extra layer of elastomer on top of the skin acts as a mechanical lowpass filter. For example, a 1 mm thick layer reduces the resolution to be on the order of 1 mm.
A fluorescent pigment can be used in the skin, illuminated by Ultraviolet (UV) light or blacklight. If the blacklight comes at a grazing angle, it can readily reveal variations in surface normal. The material will be fairly close to Lambertian. To reduce interreflections, one would select a surface that appears dark to emitted wavelengths. This principle is true with ordinary light as well. If one is using a Lambertian pigment in the skin, it is better for it to be gray than white, to reduce interreflections.
Blacklight or UV can be used to illuminate a fluorescent surface, which would then serve as a diffuse source. In some cases, it would be useful to use a single short flash (for instance, recording the instantaneous deformation of an object against the surface) or multiple periodic (strobed) flashes (to capture rapid periodic events or to modulate one frequency down to another frequency.)
Although the present invention has been shown and described with respect to several preferred embodiments thereof, various changes, omissions and additions to the form and detail thereof, may be made therein, without departing from the spirit and scope of the invention.

Claims (50)

What is claimed is:
1. A tactile sensor comprising:
a photosensing structure;
a volume of elastomer capable of transmitting an image; and
a reflective skin covering said volume of elastomer, said reflective skin being illuminated through said volume of elastomer by one or more light sources, said reflective skin having particles that non-directionally reflect light incident on the reflective skin from within the volume of elastomer, said reflective skin being geometrically altered in response to pressure applied by an entity touching said reflective skin, said geometrical alteration causing localized changes in the surface normal of said skin and associated localized changes in the amount of light reflected from said reflective skin in the direction of said photosensing structure; wherein
said photosensing structure is positioned to receive a portion of said reflected light in the form of an image, said image indicating one or more features of the entity producing said pressure.
2. The tactile sensor of claim 1, wherein said volume of elastomer comprises silicone rubber, polyurethane, plastisol, thermoplastic elastomer, natural rubber, polyisoprene, polyvinyl chloride or a mixture thereof.
3. The tactile sensor of claim 1, wherein said volume of elastomer comprises a Shore A hardness between 5 and 90.
4. The tactile sensor of claim 1, wherein the volume of elastomer is in the form of a slab.
5. The tactile sensor of claim 1, wherein said photosensing structure comprises a camera.
6. The tactile sensor of claim 1, wherein said photosensing structure comprises an array of sensing elements.
7. The tactile sensor of claim 1, wherein said one or more features comprise roughness of said entity.
8. The tactile sensor of claim 1, wherein said one or more features comprise the location, amplitude, or direction of the applied pressure.
9. The tactile sensor of claim 1, wherein said one or more features comprise the shape, size, or profile of an object producing said pressure.
10. The tactile sensor of claim 1, wherein said one or more features comprise the motion or slip of a surface touching the reflective skin.
11. The tactile sensor of claim 1, wherein the sensor has physical properties that are similar to those of human skin.
12. The tactile sensor of claim 1, wherein the sensor is formed in the shape of a specified object.
13. The tactile sensor of claim 1, wherein the reflective skin is illuminated by two or more light sources of different colors.
14. A method of performing tactile sensing, comprising:
(a) providing a volume of elastomer capable of transmitting an image;
(b) covering the elastomer with a reflective skin having an inner surface facing the elastomer and an outer surface, wherein the reflective skin comprises particles that non-directionally reflect light incident on the inner surface from within the volume of elastomer;
(c) illuminating the reflective skin through the volume of elastomer with one or more light sources, wherein at least a portion of the light is reflected by the inner surface of the reflective skin;
(d) contacting the outer surface of the reflective skin with an entity, wherein the contact produces pressure that geometrically alters the reflective skin, wherein the alteration causes localized changes in the inner surface of the reflective skin, and wherein the localized changes in the inner surface of the reflective skin cause associated localized changes in the light reflected from the inner surface of the reflective skin;
(e) positioning a photosensing structure to receive a portion of the light reflected from the inner surface of the reflective skin in the form of an image indicating one or more features of the entity contacting the outer surface of the reflective skin.
15. The method of claim 14, wherein said volume of elastomer comprises silicone rubber, polyurethane, plastisol, thermoplastic elastomer, natural rubber, polyisoprene, polyvinyl chloride or a mixture thereof.
16. The method of claim 14, wherein said volume of elastomer comprises a Shore A hardness between 5 and 90.
17. The method of claim 14, wherein the volume of elastomer is in the form of a slab.
18. The method of claim 14, wherein said photosensing structure comprises a camera.
19. The method of claim 14, wherein said photosensing structure comprises an array of sensing elements.
20. The method of claim 14, wherein said one or more features comprise roughness of said entity.
21. The method of claim 14, wherein said one or more features comprise the location, amplitude, or direction of the applied pressure.
22. The method of claim 14, wherein said one or more features comprise the shape, size, or profile of an object producing said pressure.
23. The method of claim 14, wherein said one or more features comprise the motion or slip of a surface touching the reflective skin.
24. The method of claim 14, wherein the sensor has physical properties that are similar to those of human skin.
25. The method of claim 14, wherein the sensor is formed in the shape of a specified object.
26. The method of claim 14, wherein the reflective skin is illuminated by two or more light sources of different colors.
27. A tactile sensor comprising:
a volume of elastomer for transmitting an image, the volume of elastomer having a fluorescent skin, the fluorescent skin becoming a source of light emission in response to absorbing energy from exposure to light that excites the fluorescent skin, the fluorescent skin being elastomeric and deformable in response to pressure applied by an object touching the skin, and the deformation causing localized changes in a surface normal of the fluorescent skin;
a photosensing structure, the photosensing structure being positioned to receive at least a portion of the light emitted from the fluorescent skin in the form of an image, the image indicating one or more features of the object producing the pressure; and
a light source, the light source being disposed relative to the volume of elastomer to expose the fluorescent skin to light energy absorbed by the fluorescent skin that excites the fluorescent skin, and the localized changes in the surface normal of the fluorescent skin causing corresponding localized changes in an amount of emitted light received by the photosensing structure.
28. The tactile sensor of claim 27, the localized changes in the surface normal of the fluorescent skin causing corresponding localized changes in an amount of light received by the fluorescent skin from the light source, the localized changes in the amount of light received by the fluorescent skin causing the corresponding localized changes in an amount of emitted light received by the photosensing structure.
29. The tactile sensor of claim 27, the photosensing structure comprising a camera.
30. The tactile sensor of claim 27, the photosensing structure comprising a plurality of cameras positioned at different orientations relative to the fluorescent skin.
31. The tactile sensor of claim 27, comprising a plurality of light sources disposed at different azimuths relative to the fluorescent skin.
32. The tactile sensor of claim 27, the fluorescent skin being substantially nonreflective to the light emitted from the fluorescent skin.
33. The tactile sensor of claim 27, the fluorescent skin being substantially opaque to the light emitted from the fluorescent skin.
34. The tactile sensor of claim 27, the fluorescent skin being substantially opaque to the light that excites the fluorescent skin.
35. The tactile sensor of claim 27, the light source being an ultraviolet light source.
36. The tactile sensor of claim 35, the light source being a black light source.
37. The tactile sensor of claim 27, the light source being a flashed source of light.
38. The tactile sensor of claim 27, the light source being disposed at a grazing angle relative to the fluorescent skin.
39. A method of tactile sensing comprising:
providing a volume of elastomer for transmitting an image, the volume of elastomer having a fluorescent skin;
exposing the fluorescent skin to light from a light source that causes the fluorescent skin to absorb energy from the light source that excites the fluorescent skin and causes the fluorescent skin to become a source of light emission;
contacting the fluorescent skin with an object to cause the fluorescent skin to deform in response to the contact by the object, the deformation causing localized changes in a surface normal of the fluorescent skin and associated localized changes in the amount of emitted light received by a photosensing structure; and
receiving in a photosensing structure at least a portion of the light emitted from the fluorescent skin in the form of an image, the image indicating one or more features of the object producing the pressure.
40. The method of claim 39, the localized changes in the surface normal of the fluorescent skin causing corresponding localized changes in an amount of light received by the fluorescent skin from the light source, the localized changes in the amount of light received by the fluorescent skin causing the corresponding localized changes in an amount of emitted light received by the photosensing structure.
41. The method of claim 39, the photosensing structure comprising a camera.
42. The method of claim 39, the receiving in the photosensing structure at least a portion of the light emitted from the fluorescent skin comprising receiving the at least a portion of the light in a plurality of cameras, each camera of the plurality being positioned at an orientation that differs from any other camera relative to the fluorescent skin.
43. The method of claim 39, the illuminating the fluorescent skin with the light source comprising using a plurality of light sources disposed at different azimuths from each other relative to the fluorescent skin.
44. The method of claim 39, the fluorescent skin being substantially nonreflective to the light emitted from the fluorescent skin.
45. The method of claim 39, the fluorescent skin being substantially opaque to the light emitted from the fluorescent skin.
46. The method of claim 39, the fluorescent skin being substantially opaque to the light that excites the fluorescent skin.
47. The method of claim 39, the illuminating the fluorescent skin with the light source comprising using an ultraviolet light source.
48. The method of claim 47, the illuminating the fluorescent skin with the light source comprising using a black light source.
49. The method of claim 39, the illuminating the fluorescent skin with the light source comprising using a flashed source of light.
50. The method of claim 39, the illuminating the fluorescent skin with the light source comprising disposing the light source at a grazing angle relative to the fluorescent skin.
US14/045,594 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging Active 2031-04-19 USRE45578E1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/045,594 USRE45578E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US7390408P 2008-06-19 2008-06-19
US12/488,008 US8411140B2 (en) 2008-06-19 2009-06-19 Tactile sensor using elastomeric imaging
US13/971,456 USRE44856E1 (en) 2008-06-19 2013-08-20 Tactile sensor using elastomeric imaging
US14/045,594 USRE45578E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US12/488,008 Reissue US8411140B2 (en) 2008-06-19 2009-06-19 Tactile sensor using elastomeric imaging

Publications (1)

Publication Number Publication Date
USRE45578E1 true USRE45578E1 (en) 2015-06-23

Family

ID=41430812

Family Applications (6)

Application Number Title Priority Date Filing Date
US12/488,008 Ceased US8411140B2 (en) 2008-06-19 2009-06-19 Tactile sensor using elastomeric imaging
US13/971,456 Active 2031-04-19 USRE44856E1 (en) 2008-06-19 2013-08-20 Tactile sensor using elastomeric imaging
US14/045,668 Active 2031-04-19 USRE45541E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging
US14/045,620 Active 2031-04-19 USRE45176E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging
US14/045,647 Active 2031-04-19 USRE45266E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging
US14/045,594 Active 2031-04-19 USRE45578E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging

Family Applications Before (5)

Application Number Title Priority Date Filing Date
US12/488,008 Ceased US8411140B2 (en) 2008-06-19 2009-06-19 Tactile sensor using elastomeric imaging
US13/971,456 Active 2031-04-19 USRE44856E1 (en) 2008-06-19 2013-08-20 Tactile sensor using elastomeric imaging
US14/045,668 Active 2031-04-19 USRE45541E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging
US14/045,620 Active 2031-04-19 USRE45176E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging
US14/045,647 Active 2031-04-19 USRE45266E1 (en) 2008-06-19 2013-10-03 Tactile sensor using elastomeric imaging

Country Status (4)

Country Link
US (6) US8411140B2 (en)
EP (2) EP2294375B8 (en)
JP (1) JP5449336B2 (en)
WO (1) WO2009155501A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10837916B2 (en) 2017-03-09 2020-11-17 Spirit Aerosystems, Inc. Optical measurement device for inspection of discontinuities in aerostructures

Families Citing this family (104)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2944901B1 (en) * 2009-04-27 2011-05-20 Sagem Securite DEVICE FOR IDENTIFYING (100) A PERSON BY HIS IMPRESSION.
US9092737B2 (en) * 2009-07-30 2015-07-28 Northwestern University Systems, methods, and apparatus for 3-D surface mapping, compliance mapping, and spatial registration with an array of cantilevered tactile hair or whisker sensors
JP5930166B2 (en) * 2009-12-07 2016-06-08 日本電気株式会社 False finger discrimination device
WO2011069435A1 (en) * 2009-12-07 2011-06-16 北京联想软件有限公司 Method and terminal device for operation control of operation object
WO2011128266A1 (en) * 2010-04-12 2011-10-20 Imec Optical shear sensor and method of producing such an optical shear sensor
US8890016B2 (en) * 2010-12-21 2014-11-18 Stilotech Inc. Touch pad device
CN103429138B (en) 2011-02-16 2016-09-07 通用医疗公司 Photo-coupler for endoscope
WO2012129410A2 (en) * 2011-03-23 2012-09-27 University Of Southern California Elastomeric optical tactile sensor
US9082188B2 (en) * 2011-04-11 2015-07-14 Hid Global Corporation Optical topographic imaging
US8966999B2 (en) * 2011-06-17 2015-03-03 Microsoft Corporation Pressure sensor linearization
US9720525B2 (en) * 2011-06-29 2017-08-01 Wen-Chieh Geoffrey Lee High resolution and high sensitivity optically activated cursor maneuvering device
KR20140041890A (en) * 2011-07-28 2014-04-04 메사추세츠 인스티튜트 오브 테크놀로지 Optical tactile sensor
US8971593B2 (en) * 2011-10-12 2015-03-03 Lumidigm, Inc. Methods and systems for performing biometric functions
US10555697B2 (en) 2011-10-17 2020-02-11 Massachusetts Institute Of Technology Digital image correlation for measuring skin strain and deformation
US9149224B1 (en) * 2011-10-17 2015-10-06 Massachusetts Institute Of Technology System and method for measuring skin movement and strain and related techniques
HUP1100633A2 (en) * 2011-11-17 2013-06-28 Pazmany Peter Katolikus Egyetem Device with optical feedback for measuring force and pressure
US8994694B2 (en) 2011-11-30 2015-03-31 Blackberry Limited Optical interference based user input device
EP2600229A1 (en) * 2011-11-30 2013-06-05 Research In Motion Limited Optical interference based user input device
WO2013096499A1 (en) * 2011-12-19 2013-06-27 The Regents Of The University Of California System for and method of quantifying on-body palpitation for improved medical diagnosis
US9480529B2 (en) 2012-06-22 2016-11-01 S & Y Enterprises Llc Aesthetic treatment device and method
US9364684B2 (en) 2012-06-22 2016-06-14 S & Y Enterprises Llc Aesthetic treatment device and method
US20160157724A1 (en) * 2012-07-10 2016-06-09 Delaval Holding Ab Registering of physiological parameters based on image analysis of light reflection
EP2910009A4 (en) * 2012-10-17 2016-07-27 Gelsight Inc Three-dimensional digital impression and visualization of objects
CN103892841B (en) * 2012-12-26 2016-12-28 大连恒锐科技股份有限公司 A kind of pressure sensitivity light and shadow imaging system and formation method thereof
US9323346B2 (en) * 2012-12-31 2016-04-26 Futurewei Technologies, Inc. Accurate 3D finger tracking with a single camera
US10574944B2 (en) * 2013-03-08 2020-02-25 Gelsight, Inc. Continuous contact-based three-dimensional measurement
US10055562B2 (en) * 2013-10-23 2018-08-21 Intel Corporation Techniques for identifying a change in users
US10114510B2 (en) * 2013-12-23 2018-10-30 Samsung Display Co., Ltd. Apparatus and method for detecting surface shear force on a display device
US9977543B2 (en) * 2014-02-27 2018-05-22 Samsung Display Co., Ltd. Apparatus and method for detecting surface shear force on a display device
JP6321474B2 (en) * 2014-07-04 2018-05-09 京セラ株式会社 Sensor, sensor device and optical operation device
JP6528774B2 (en) * 2014-07-30 2019-06-12 ソニー株式会社 INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM
JP6149825B2 (en) * 2014-08-22 2017-06-21 トヨタ自動車株式会社 Vehicle control device
US9459442B2 (en) 2014-09-23 2016-10-04 Scott Miller Optical coupler for optical imaging visualization device
US9757862B2 (en) * 2014-10-16 2017-09-12 Technische Universität München Tactile sensor
JP6561400B2 (en) * 2015-02-10 2019-08-21 任天堂株式会社 Information processing apparatus, information processing program, information processing system, and information processing method
JP6534011B2 (en) 2015-02-10 2019-06-26 任天堂株式会社 INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING PROGRAM, INFORMATION PROCESSING SYSTEM, AND INFORMATION PROCESSING METHOD
CN104794466A (en) * 2015-05-19 2015-07-22 深圳市恒睿智达科技有限公司 Living body biological characteristic information optimal image acquisition module and mobile terminal
CN104794471A (en) * 2015-05-19 2015-07-22 深圳市恒睿智达科技有限公司 Biological characteristic optical image acquisition module based on glass panel, and mobile terminal
US10548467B2 (en) 2015-06-02 2020-02-04 GI Scientific, LLC Conductive optical element
JP6935100B2 (en) 2015-07-21 2021-09-15 ジーアイ・サイエンティフィック・リミテッド・ライアビリティ・カンパニーGi Scientific, Llc Endoscope accessories with angle-adjustable exit portal
US9870499B2 (en) 2015-08-12 2018-01-16 Nec Corporation Biometric collection device
US10038854B1 (en) * 2015-08-14 2018-07-31 X Development Llc Imaging-based tactile sensor with multi-lens array
US10893814B2 (en) 2015-10-06 2021-01-19 Koninklijke Philips N.V. System and method for obtaining vital sign related information of a living being
DE102016108966B4 (en) * 2016-05-13 2017-11-30 Technische Universität München Visual-haptic sensor for 6D force / torque
WO2017196282A2 (en) * 2016-05-13 2017-11-16 Buyuksahin Utku A practical sensing system
TR201606363A2 (en) * 2016-05-13 2017-11-21 Sensobright Ind Llc A multifunction detection system.
US10663361B2 (en) * 2016-10-13 2020-05-26 The Trustees Of Columbia University In The City Of New York Systems and methods for tactile sensing
CN108534928A (en) * 2017-03-06 2018-09-14 王永华 Pressure distribution sensor
EP4142284A1 (en) * 2017-03-06 2023-03-01 Gelsight, Inc. Surface topography measurement systems
EP3643459A4 (en) * 2017-06-21 2020-12-16 Saito Inventive Corp. Manipulator and robot
US11060214B2 (en) 2017-08-11 2021-07-13 Northwestern University Processing methods and apparatus to manufacture a functional, multi-scale, tapered fiber from polymer filament
CN209460748U (en) * 2017-08-17 2019-10-01 深圳信炜科技有限公司 Display module and electronic equipment
DE102018123546A1 (en) 2017-09-26 2019-03-28 Toyota Research Institute, Inc. DEFORMABLE SENSORS AND METHOD FOR DETECTING POSE AND POWER ON AN OBJECT
US10668627B2 (en) * 2017-09-26 2020-06-02 Toyota Research Institute, Inc. Deformable sensors and methods for detecting pose and force against an object
CN108072464B (en) * 2017-11-30 2019-10-29 东南大学 A kind of imitation human finger end sliding touch sensor
CN108089774B (en) * 2018-02-09 2021-01-26 京东方科技集团股份有限公司 Touch control equipment
EP4434472A2 (en) 2018-04-03 2024-09-25 Trustees of Tufts College Tactile blood pressure imager
JP7005429B2 (en) * 2018-05-31 2022-02-04 株式会社東海理化電機製作所 Display input device
EP3801219A4 (en) * 2018-06-01 2022-07-20 Cardio Ring Technologies, Inc. Optical blood pressure measurement devices and methods
CN112203824B (en) * 2018-06-07 2022-04-19 指纹卡安娜卡敦知识产权有限公司 Smart card comprising a fingerprint sensor and method for manufacturing a smart card
CN108705537A (en) * 2018-08-08 2018-10-26 中国科学技术大学 A kind of raster pattern touch sensor and relevant apparatus
CA3111989A1 (en) * 2018-09-06 2020-03-12 Gelsight, Inc. Retrographic sensors
JP6831038B2 (en) 2018-11-28 2021-02-17 モメンティブ・パフォーマンス・マテリアルズ・ジャパン合同会社 Curable polyorganosiloxane composition and polyorganosiloxane cured product
US11337609B2 (en) * 2019-01-25 2022-05-24 Samsung Electronics Co., Ltd. Texture interface for measuring bio-signal and bio-signal measuring apparatus including the same
NL2022901B1 (en) * 2019-04-09 2020-10-20 Univ Delft Tech Pressure-difference sensitive stack
GB201907744D0 (en) 2019-05-31 2019-07-17 The Shadow Robot Company Ltd Tactile sensor
CN112097675A (en) 2019-06-17 2020-12-18 香港科技大学 Tactile sensor
EP3995268A4 (en) * 2019-07-04 2023-04-12 FingerVision Co., Ltd. Tactile sensor, tactile sensor system, and program
WO2021076697A1 (en) * 2019-10-15 2021-04-22 Massachusetts Institute Of Technology Retrographic sensors with compact illumination
WO2021081084A1 (en) * 2019-10-21 2021-04-29 The Regents Of The University Of California Multi-directional high-resolution optical tactile sensors
JP7520867B2 (en) * 2019-10-30 2024-07-23 ソニーグループ株式会社 Optical sensors and optical sensor modules
US20230294300A1 (en) * 2020-07-30 2023-09-21 Abb Schweiz Ag Diffractive Visual-Tactile Sensing in Robotic Grippers
KR20230104857A (en) * 2020-08-10 2023-07-11 다이노카디아, 인코포레이티드 An optomechanical method for arterial pulse measurement and cardiorespiratory hemodynamics assessment
JP6864401B1 (en) * 2020-08-17 2021-04-28 株式会社SensAI Tactile sensor
EP3957947B1 (en) * 2020-08-18 2023-10-04 Sony Group Corporation Electronic device and method to reconstruct the shape of a deformable object
KR20220028348A (en) * 2020-08-28 2022-03-08 삼성전자주식회사 Stretchable strain sensor and combination sensor and display panel and device
WO2022132300A2 (en) * 2020-12-15 2022-06-23 Massachusetts Institute Of Technology Retrographic sensors with fluorescent illumination
FR3117592B1 (en) 2020-12-16 2022-12-16 Commissariat Energie Atomique Force sensor
CN112595264A (en) * 2020-12-29 2021-04-02 苏州星祥益精密制造有限公司 Visual touch automobile high-reflection surface three-dimensional shape measuring method and system
CN112652060A (en) * 2021-01-06 2021-04-13 上海交通大学 Multi-modal visual and tactile sensing system and method based on particle image velocimetry
US20240060836A1 (en) * 2021-01-08 2024-02-22 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e. V. Sensor arrangement for sensing forces and methods for fabricating a sensor arrangement and parts thereof
US20240153115A1 (en) * 2021-03-08 2024-05-09 Massachusetts Institute Of Technology Enhanced depth estimation using deep learning
GB2605423B (en) * 2021-03-31 2023-09-27 Ocado Innovation Ltd Tactile Sensor
US11472039B1 (en) * 2021-04-23 2022-10-18 Toyota Research Institute, Inc. Deformable sensor with rotatable sensing components for detecting deformation levels
USD989637S1 (en) 2021-06-16 2023-06-20 Gelsight, Inc. Retrographic sensor cartridge
US11686574B2 (en) * 2021-07-20 2023-06-27 Toyota Research Institute, Inc Deformable sensors and methods for modifying run-time membrane stiffness
CN113520617B (en) * 2021-07-21 2023-10-20 天津大学 Passive three-dimensional force sensing head and optical force sensor
WO2023014693A1 (en) 2021-08-02 2023-02-09 Gelsight, Inc. Abrasion-resistant composite
CN113686881A (en) * 2021-09-23 2021-11-23 云智汇(深圳)高新科技服务有限公司 Visual all-angle imaging device in high-reflection mirror surface environment
EP4399481A1 (en) 2021-10-08 2024-07-17 Gelsight, Inc. Retrographic sensing
CN114113008B (en) * 2021-10-22 2023-12-22 清华大学深圳国际研究生院 Structured light-based artificial haptic device and method
WO2023081342A1 (en) * 2021-11-05 2023-05-11 Board Of Regents, The University Of Texas System Four-dimensional tactile sensing system, device, and method
JPWO2023100483A1 (en) * 2021-11-30 2023-06-08
JPWO2023100515A1 (en) * 2021-12-02 2023-06-08
CN114147750B (en) * 2021-12-03 2024-04-09 廊坊晶正光电技术有限公司 Robot touch sensing system and method for sensing pressure in imaging mode
US20230251149A1 (en) * 2022-02-04 2023-08-10 Massachusetts Institute Of Technology Flexible optical tactile sensor
WO2023158840A1 (en) 2022-02-18 2023-08-24 Gelsight, Inc. Fluid tactile sensors
WO2023168463A2 (en) * 2022-03-04 2023-09-07 Gelsight, Inc. Systems and methods for tactile intelligence
CN114659679B (en) * 2022-04-12 2024-09-03 深圳市松果体机器人科技有限公司 Flexible tactile sensor
WO2023210376A1 (en) * 2022-04-28 2023-11-02 富士フイルム株式会社 Pressure sensor and robot
WO2023228527A1 (en) * 2022-05-27 2023-11-30 パナソニックIpマネジメント株式会社 Optical tactile sensor and sensor system
CN115144117A (en) * 2022-06-15 2022-10-04 复旦大学 Flexible high-resolution multi-dimensional force touch sensor based on optics and preparation method thereof
GB2620729B (en) * 2022-06-24 2024-08-07 Stellar Advanced Concepts Ltd Sensor for sensing fluid flow, touch and/or vibration
USD1043990S1 (en) * 2022-08-30 2024-09-24 Gelsight, Inc. Retrographic sensor cartridge

Citations (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4120585A (en) 1976-11-19 1978-10-17 Calspan Corporation Fingerprint identification system using a pliable optical prism
US4340300A (en) 1980-08-11 1982-07-20 Siemens Corporation Input sensor unit for a fingerprint identification system
US4481815A (en) 1982-12-23 1984-11-13 Overton Kenneth J Tactile sensor
US4549093A (en) 1982-09-29 1985-10-22 Emi Limited Tactile array sensor
US4584625A (en) 1984-09-11 1986-04-22 Kellogg Nelson R Capacitive tactile sensor
US4588348A (en) 1983-05-27 1986-05-13 At&T Bell Laboratories Robotic system utilizing a tactile sensor array
US4775961A (en) 1985-06-24 1988-10-04 Vysoka Skola Dopravy A Spojov V Ziline Tactile sensor
US5357799A (en) 1992-07-14 1994-10-25 Bridgestone Corporation Method and apparatus for determining abrasion potential of tire treads
US5448649A (en) 1994-05-24 1995-09-05 Chen; Wang S. Apparatus for imaging fingerprint or topographic relief pattern on the surface of an object
US5616839A (en) 1996-01-29 1997-04-01 Ford Motor Company Apparatus and method for measuring contact force distribution of a tire
US5737071A (en) * 1996-08-16 1998-04-07 Identicator Corporation Method and apparatus for enhancing live-scan fingerprint reader images
US5967940A (en) 1997-09-17 1999-10-19 Toyota Jidosha Kabushiki Kaisha Method and apparatus for reducing backlash sound in gear mechanism
US5967990A (en) 1998-08-13 1999-10-19 President And Fellows Of Harvard College Surgical probe comprising visible markings on an elastic membrane
US5986746A (en) * 1994-02-18 1999-11-16 Imedge Technology Inc. Topographical object detection system
US6144757A (en) * 1997-02-28 2000-11-07 Nec Corporation System and method for identifying an organism
US6234031B1 (en) * 1997-11-27 2001-05-22 Nec Corporaion Fingerprint detection apparatus
US20020150701A1 (en) * 1998-08-28 2002-10-17 Jiro Onishi Fluorescent latent image transfer film, fluorescent latent image transfer method using the same, and security pattern formed matter
US20030169905A1 (en) * 2002-03-07 2003-09-11 Butterworth Mark Melvin Electronic device with digital camera for general-purpose photography and fingerprint sensing
US20030178556A1 (en) 2000-08-31 2003-09-25 Susumu Tachi Optical tactile sensor
US20040237669A1 (en) * 2003-05-29 2004-12-02 Vincent Hayward Method and apparatus to record and reproduce tactile sensations
US20040255128A1 (en) * 2003-04-25 2004-12-16 Nec Corporation Fingerprint authentication system and method
US20040252867A1 (en) * 2000-01-05 2004-12-16 Je-Hsiung Lan Biometric sensor
JP2005121858A (en) 2003-10-16 2005-05-12 Toppan Printing Co Ltd Reflective sheet
WO2005085766A2 (en) 2004-03-01 2005-09-15 University Of Washington Techtransfer Invention Licensing Polymer based distributive waveguide sensor for pressure and shear measurement
US6965685B1 (en) * 2001-09-04 2005-11-15 Hewlett-Packard Development Company, Lp. Biometric sensor
US20060008129A1 (en) * 2004-07-07 2006-01-12 Dong-Jae Lee Electronic device with camera and fingerprint security function
US20060119837A1 (en) * 2004-10-16 2006-06-08 Raguin Daniel H Diffractive imaging system and method for the reading and analysis of skin topology
WO2006098719A1 (en) 2005-03-09 2006-09-21 The Procter & Gamble Company Sensor responsive electric toothbrushes and methods of use
US20070146536A1 (en) 2003-10-24 2007-06-28 Eastman Kodak Company Camera with light detection means
JP2007520040A (en) 2004-01-28 2007-07-19 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ lighting equipment
US7248298B2 (en) * 2000-01-10 2007-07-24 Testech, Inc. Fingerprint recognition sensor and manufacturing method thereof
US20070288186A1 (en) 2006-02-10 2007-12-13 Saswati Datta Methods of use of substrate having properties of keratinous tissue
JP2008008746A (en) 2006-06-29 2008-01-17 Univ Of Tokyo Tactile sensor using reflected image
US20080106258A1 (en) 2006-11-02 2008-05-08 Eduardo Torres-Jara Compliant tactile sensor
US20080219521A1 (en) * 2004-04-16 2008-09-11 Validity Sensors, Inc. Method and Algorithm for Accurate Finger Motion Tracking
US20080284925A1 (en) * 2006-08-03 2008-11-20 Han Jefferson Y Multi-touch sensing through frustrated total internal reflection
US7707001B2 (en) * 2004-03-09 2010-04-27 Nagoya Industrial Science Research Institute Control of object operating force, object gripping force and robot hands
US20100284565A1 (en) * 2006-09-11 2010-11-11 Validity Sensors, Inc. Method and apparatus for fingerprint motion tracking using an in-line array

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4668861A (en) * 1984-12-12 1987-05-26 The Regents Of The University Of California Tactile sensor employing a light conducting element and a resiliently deformable sheet
US4599908A (en) * 1985-03-18 1986-07-15 Sheridan Thomas B Opto-mechanical touch sensor
US4814562A (en) * 1987-11-13 1989-03-21 University Of Arkansas Electro-optic force and pressure transducer and sensor
JPH07260456A (en) * 1994-03-17 1995-10-13 Nissan Motor Co Ltd Method and apparatus for detection of unevenness
JP3011126B2 (en) * 1997-03-27 2000-02-21 日本電気株式会社 Fingerprint detection device
US5879454A (en) * 1998-07-15 1999-03-09 Aetex Biometric Corporation Finger-moistening means for fingerprint recognizing apparatus
US6154285A (en) * 1998-12-21 2000-11-28 Secugen Corporation Surface treatment for optical image capturing system
JP4164968B2 (en) * 1999-11-15 2008-10-15 Nok株式会社 Contact state observation method and contact state observation apparatus
US20030068470A1 (en) * 2001-08-17 2003-04-10 Eastman Kodak Company Combination of imaging member and functional base for new utility
JP2006003137A (en) * 2004-06-16 2006-01-05 Toudai Tlo Ltd Optical tactile sensor and information acquiring method in the same
WO2008017077A2 (en) * 2006-08-03 2008-02-07 Perceptive Pixel, Inc. Multi-touch sensing display through frustrated total internal reflection
JP2008256413A (en) * 2007-04-02 2008-10-23 Nissan Motor Co Ltd Apparatus and method for measuring cross-sectional shape
WO2012006431A2 (en) * 2010-07-09 2012-01-12 Temple University - Of The Commonwealth System Of Higher Education Apparatus and method for surface and subsurface tactile sensation imaging
EP2910009A4 (en) * 2012-10-17 2016-07-27 Gelsight Inc Three-dimensional digital impression and visualization of objects

Patent Citations (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4120585A (en) 1976-11-19 1978-10-17 Calspan Corporation Fingerprint identification system using a pliable optical prism
US4340300A (en) 1980-08-11 1982-07-20 Siemens Corporation Input sensor unit for a fingerprint identification system
US4549093A (en) 1982-09-29 1985-10-22 Emi Limited Tactile array sensor
US4481815A (en) 1982-12-23 1984-11-13 Overton Kenneth J Tactile sensor
US4588348A (en) 1983-05-27 1986-05-13 At&T Bell Laboratories Robotic system utilizing a tactile sensor array
US4584625A (en) 1984-09-11 1986-04-22 Kellogg Nelson R Capacitive tactile sensor
US4775961A (en) 1985-06-24 1988-10-04 Vysoka Skola Dopravy A Spojov V Ziline Tactile sensor
US5357799A (en) 1992-07-14 1994-10-25 Bridgestone Corporation Method and apparatus for determining abrasion potential of tire treads
US5986746A (en) * 1994-02-18 1999-11-16 Imedge Technology Inc. Topographical object detection system
US5448649A (en) 1994-05-24 1995-09-05 Chen; Wang S. Apparatus for imaging fingerprint or topographic relief pattern on the surface of an object
US5616839A (en) 1996-01-29 1997-04-01 Ford Motor Company Apparatus and method for measuring contact force distribution of a tire
US5737071A (en) * 1996-08-16 1998-04-07 Identicator Corporation Method and apparatus for enhancing live-scan fingerprint reader images
US6144757A (en) * 1997-02-28 2000-11-07 Nec Corporation System and method for identifying an organism
US5967940A (en) 1997-09-17 1999-10-19 Toyota Jidosha Kabushiki Kaisha Method and apparatus for reducing backlash sound in gear mechanism
US6234031B1 (en) * 1997-11-27 2001-05-22 Nec Corporaion Fingerprint detection apparatus
US5967990A (en) 1998-08-13 1999-10-19 President And Fellows Of Harvard College Surgical probe comprising visible markings on an elastic membrane
US20020150701A1 (en) * 1998-08-28 2002-10-17 Jiro Onishi Fluorescent latent image transfer film, fluorescent latent image transfer method using the same, and security pattern formed matter
US20040252867A1 (en) * 2000-01-05 2004-12-16 Je-Hsiung Lan Biometric sensor
US7248298B2 (en) * 2000-01-10 2007-07-24 Testech, Inc. Fingerprint recognition sensor and manufacturing method thereof
US20030178556A1 (en) 2000-08-31 2003-09-25 Susumu Tachi Optical tactile sensor
US6909084B2 (en) 2000-08-31 2005-06-21 Toudai Tlo, Ltd Optical tactile sensor having a transparent elastic tactile portion
US6965685B1 (en) * 2001-09-04 2005-11-15 Hewlett-Packard Development Company, Lp. Biometric sensor
US20030169905A1 (en) * 2002-03-07 2003-09-11 Butterworth Mark Melvin Electronic device with digital camera for general-purpose photography and fingerprint sensing
US20040255128A1 (en) * 2003-04-25 2004-12-16 Nec Corporation Fingerprint authentication system and method
US20040237669A1 (en) * 2003-05-29 2004-12-02 Vincent Hayward Method and apparatus to record and reproduce tactile sensations
JP2005121858A (en) 2003-10-16 2005-05-12 Toppan Printing Co Ltd Reflective sheet
US20070146536A1 (en) 2003-10-24 2007-06-28 Eastman Kodak Company Camera with light detection means
JP2007520040A (en) 2004-01-28 2007-07-19 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ lighting equipment
WO2005085766A2 (en) 2004-03-01 2005-09-15 University Of Washington Techtransfer Invention Licensing Polymer based distributive waveguide sensor for pressure and shear measurement
US7707001B2 (en) * 2004-03-09 2010-04-27 Nagoya Industrial Science Research Institute Control of object operating force, object gripping force and robot hands
US20080219521A1 (en) * 2004-04-16 2008-09-11 Validity Sensors, Inc. Method and Algorithm for Accurate Finger Motion Tracking
US20060008129A1 (en) * 2004-07-07 2006-01-12 Dong-Jae Lee Electronic device with camera and fingerprint security function
US20060119837A1 (en) * 2004-10-16 2006-06-08 Raguin Daniel H Diffractive imaging system and method for the reading and analysis of skin topology
WO2006098719A1 (en) 2005-03-09 2006-09-21 The Procter & Gamble Company Sensor responsive electric toothbrushes and methods of use
US20070288186A1 (en) 2006-02-10 2007-12-13 Saswati Datta Methods of use of substrate having properties of keratinous tissue
JP2008008746A (en) 2006-06-29 2008-01-17 Univ Of Tokyo Tactile sensor using reflected image
US20080284925A1 (en) * 2006-08-03 2008-11-20 Han Jefferson Y Multi-touch sensing through frustrated total internal reflection
US20100284565A1 (en) * 2006-09-11 2010-11-11 Validity Sensors, Inc. Method and apparatus for fingerprint motion tracking using an in-line array
US20080106258A1 (en) 2006-11-02 2008-05-08 Eduardo Torres-Jara Compliant tactile sensor

Non-Patent Citations (15)

* Cited by examiner, † Cited by third party
Title
Begej, "Planar and Finger-Shaped Optical Tactile Sensors for Robotic Applications" IEEE Journal of Robotics and Automation, vol. 4, No. 5, Oct. 1988, pp. 472-484.
English Translation of Japanese Notice of Rejection issued on Aug. 7, 2012 in connection with Japanese Patent Appln. No. 2011-514834, 2 pages.
Full et al., "Maximum single leg force production: Cockroaches Righting on Photoelastic Gelatin" The Journal of Experimental Biology 198, 1995, pp. 2441-2452.
Graz et al., "Flexible ferroelectret field-effect transistor for large-area sensor skins and microphones" Applied Physics Letters, 89, 2006, pp. 073501-1-073501-3.
Heo et al., "Tactile Sensor arrays using fiber Bragg grating sensors" Sensors and Actuators A 126, 2006 pp. 312-327.
Hritsu et al., "The performance of a deformable-membrane tactile sensor: basic results on geometrically-defined tasks" IEEE International Conference on Robotics and Automation 2000, pp. 508-513.
International Preliminary Report on Patentability issued on Jan. 6, 2011 in connection with International Application PCT/US2009/047930, 6 pages.
International Search Report and Written Opinion issued on Jan. 21, 2010 in connection with International Application PCT/US2009/047930, 11 pages.
Kajimoto et al., "Active Tactile Sensor using Deformable Sheet Reflector" Technical Digest of the 16th Sensor Symposium 1998, pp. 99-104.
Lee et al., "A Modular Expandable Tactile Sensor Using Flexible Polymer" 18th IEEE International Conference 2005, pp. 642-645.
Maekawa et al., "Development of a Finger-Shaped Tactile Sensor and its Evaluation by Active Touch" Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, May 1992, pp. 1327-1334.
Maheshwari et al., "High-Resolution Thin Film Device to Sense Texture by Touch" Science, vol. 312, Jun. 9, 2006, pp. 1501-1504.
Maheshwari et al., "Tactile Sensors to Sense Touch on a Par with a Human Finger" Angewandte Chem. Int. Ed. 2008, 47, pp. 7808-7826.
NODA, "300nm-Thick Cantilever in PDMS for Tactile Sensing" 2005 IEEE pp. 283-286.
O'Gorman et al., "Fingerprint Verification" Chapter 2, Biometrics:Personal Identification in Networked Society1999, 1-22.

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10837916B2 (en) 2017-03-09 2020-11-17 Spirit Aerosystems, Inc. Optical measurement device for inspection of discontinuities in aerostructures

Also Published As

Publication number Publication date
USRE45541E1 (en) 2015-06-02
EP4321975A3 (en) 2024-06-05
EP2294375A4 (en) 2017-05-03
EP2294375B1 (en) 2023-12-20
USRE45266E1 (en) 2014-12-02
JP5449336B2 (en) 2014-03-19
WO2009155501A3 (en) 2010-03-11
WO2009155501A2 (en) 2009-12-23
US8411140B2 (en) 2013-04-02
JP2011525284A (en) 2011-09-15
EP2294375B8 (en) 2024-02-14
USRE45176E1 (en) 2014-10-07
EP2294375A2 (en) 2011-03-16
USRE44856E1 (en) 2014-04-22
US20090315989A1 (en) 2009-12-24
EP4321975A2 (en) 2024-02-14

Similar Documents

Publication Publication Date Title
USRE45578E1 (en) Tactile sensor using elastomeric imaging
CN111448570B (en) Optically sensing a fingerprint or other pattern on or near a display screen with an optical detector integrated into the display screen
CN111950529B (en) Optical fingerprint sensor with non-contact imaging capability
CN111401243B (en) Optical sensor module and electronic device thereof
US9538056B2 (en) High-resolution surface measurement systems and methods
CN111316287B (en) Lens-pinhole array design in ultra-thin underscreen optical sensors for on-screen fingerprint sensing
CN109196524B (en) Electronic device for detecting fingerprint through optical sensing and operation method thereof
US10949643B2 (en) On-LCD screen optical fingerprint sensing based on optical imaging with lens-pinhole module and other optical designs
CA1286032C (en) Optical scanning and recording apparatus for fingerprints
CN1411553A (en) Sensor with holographic multiplexed imae display
CN109255323A (en) A kind of fingerprint recognition structure, display base plate and display device
CN113892019A (en) Tactile sensor
CN111727439B (en) Device for direct optical capture of skin prints and documents
CN102742258A (en) Image processing device
JP2011502249A (en) Optical device for observing millimeter or submillimeter structural details of objects with specular behavior
WO2022052663A1 (en) Biometric sensing device
US20040135739A1 (en) Three-dimensional image display method, device for the same, light direction detector, and light direction detecting method
TWM371274U (en) Multi-dimensional touch control module for generating multi-dimensional sensitizing areas
JP2980051B2 (en) Fingerprint detection method and device
CN112652060A (en) Multi-modal visual and tactile sensing system and method based on particle image velocimetry
CN113994393A (en) Multi-color illumination in optical fingerprint sensors for anti-counterfeiting
CN111788574A (en) OLED display screen with integrated photodetector TFT
CN112335049A (en) Imaging assembly, touch screen, camera shooting module, intelligent terminal, camera and distance measuring method
CN1920477A (en) Sensing method and device for deformation of contact surface
WO2023228527A1 (en) Optical tactile sensor and sensor system

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: NATIONAL SCIENCE FOUNDATION, VIRGINIA

Free format text: CONFIRMATORY LICENSE;ASSIGNOR:MASSACHUSETTS INSTITUTE OF TECHNOLOGY;REEL/FRAME:046310/0488

Effective date: 20180504

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 12