WO2019086885A1 - Apparatus and method for providing tactile stimulus - Google Patents
Apparatus and method for providing tactile stimulus Download PDFInfo
- Publication number
- WO2019086885A1 WO2019086885A1 PCT/GB2018/053174 GB2018053174W WO2019086885A1 WO 2019086885 A1 WO2019086885 A1 WO 2019086885A1 GB 2018053174 W GB2018053174 W GB 2018053174W WO 2019086885 A1 WO2019086885 A1 WO 2019086885A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- physical property
- haptic device
- target surface
- active surface
- tactile
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/014—Hand-worn input/output arrangements, e.g. data gloves
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
- A61F9/00—Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
- A61F9/08—Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/001—Teaching or communicating with blind persons
- G09B21/003—Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays
Definitions
- the present invention relates to haptic devices for providing tactile stimuli to users.
- Visually impaired persons may place a greater reliance on tactile stimuli to determine properties of objects into which they come into contact.
- many surface properties of objects may not readily provide an appropriate tactile stimulus. Examples may include optical properties such as colour, shade, brightness, pattern (including writing and images printed or displayed on the surface), reflectivity, absorptivity etc.
- Further examples may include surface texture which has sufficiently fine features that the normal sensitivity to touch of a human finger is not capable of resolving detail of the surface texture.
- An object of the invention is to provide a convenient, portable device suitable for providing a tactile stimulus to a user based on or derived from features of a target surface, to assist the user in determining physical properties of that target surface which might otherwise be difficult or impossible to assess using unaided touch.
- the present invention provides a haptic device comprising: an active surface comprising a tactile transducer configured to generate variable tactile modulation of the active surface according to a control signal;
- a physical property sensor configured to determine a physical property of a target surface adjacent to the haptic device and to provide a surface characteristic signal indicative of the physical property thereof
- a controller configured to receive the surface characteristic signal from the physical property sensor and to generate the control signal as a function of the physical property of the target surface.
- the active surface may have a surface area corresponding to the area of the distal phalanx of a user's finger or thumb.
- the active surface may be disposed directly over the physical property sensor such that, in use, the target surface lies immediately below a user's finger resting on the active surface.
- the physical property sensor may comprise one or more of an electromagnetic sensor, an optical sensor, a mechanical sensor, a stylus, and a surface contour probe.
- the physical property sensor may comprise an optical sensor.
- the haptic device may further include an optical character recognition engine configured to provide a character signal as the control signal.
- the controller may be configured to generate the variable tactile modulation of the active surface to simulate one or more Braille characters corresponding to the character signal.
- the physical property sensor may comprise an optical sensor and the controller may be configured to generate the variable tactile modulation of the active surface as a function of colour and / or optical patterns on the target surface.
- the physical property sensor may comprise an optical sensor and the controller may be configured to generate the variable tactile modulation of the active surface as a function of intensity of patterns on the target surface.
- the controller may be configured to generate the variable tactile modulation of the active surface as a function of texture of the target surface.
- the haptic device may be configured as a puck or a mouse for positioning between the target surface and a user's finger.
- the haptic device may be configured to roll or glide over the target surface by manipulation with a single hand or finger.
- the haptic device may be configured as a glove or thimble for attachment over a user's finger or hand.
- the haptic device may be configured to provide a plurality of surface characteristic signals indicative of a respective plurality of physical properties.
- the controller may be configured to generate control signals as a function of the plurality of physical properties so as to cause tactile modulation of the active surface representative of the plural physical properties contemporaneously or sequentially.
- the present invention provides a method of providing a tactile representation of physical properties of a target surface comprising:
- a physical property of a target surface adjacent to a haptic device sensing a physical property of a target surface adjacent to a haptic device and generating a surface characteristic signal indicative of the physical property thereof; receiving the surface characteristic signal from the physical property sensor and generating a control signal as a function of the physical property of the target surface; generating a variable tactile modulation of an active surface of the haptic device according to the control signal.
- Figure 1 shows a schematic diagram of components of a haptic device for providing a user with tactile stimuli corresponding to physical features of a target surface adjacent to the haptic device;
- Figure 2 shows a perspective view of an external arrangement of haptic device similar to that of figure 1.
- the descriptors relating to relative orientation and position such as “top”, “bottom”, “horizontal”, “vertical”, “left”, “right”, “up”, “down”, “front”, “back”, as well as any adjective and adverb derivatives thereof, are used in the sense of the orientation of device as presented in the drawings. However, such descriptors are not intended to be in any way limiting to an intended use of the described or claimed invention.
- a haptic device 20 comprises a housing 3 (shown in dashed outline) which may be in the approximate form of a cube or cuboid.
- the housing 3 is seen resting on a target surface 5.
- On an upper or top surface 21 of the housing 3 is an active surface 2.
- the active surface comprises a tactile transducer 4 which is configured to generate variable tactile modulation of the active surface 2 according to a suitable control signal, such that a user whose finger 1 is positioned over the active surface 2 will be provided with a suitable tactile stimulus.
- the tactile transducer 4 comprises an array of electrically actuatable pins 22 each of which can be driven in an axial direction, i.e. up and down as viewed in figure 1.
- the upper ends of the pins 22 define the active surface 2 which can be modulated over its surface area according to the relative positions of each of the array of pins 4.
- active surface 2 may be deployed.
- the pins 22 may lie beneath a flexible, displaceable membrane or other surface layer and operate to locally displace the membrane in the vicinity of the respective pin.
- the active surface may be implemented by way of a sheet having an array of compartments each of which can be independently inflated and deflated to provide tactile modulation of the active surface.
- the tactile transducer 4 may comprise any suitable mechanism capable of locally displacing regions of an active surface 2, e.g. in a direction orthogonal to a rest plane or reference plane of the active surface.
- a physical property sensor 7 which is configured to determine a physical property of the target surface 5 adjacent to the haptic device 20.
- the physical property sensor 7 is disposed on or associated with a lower or bottom surface 24 in a position that is preferably opposite to and in approximate alignment with the active surface 2.
- the active surface 2 may be positioned directly over the physical property sensor 7 such that a user's finger 1 resting on the active surface will lie directly over a sensed part of the target surface 5.
- the active surface 2 has a surface area which approximately corresponds to the area of the distal phalanx of a user's finger or thumb.
- the physical property sensor 7 comprises an optical imaging device such as a camera, a laser scanner, a light sensor, an infra-red sensor.
- Other sensors are possible, e.g. any electromagnetic sensing device such as an optical or thermal imaging sensor, an ultrasound sensor, or a mechanical sensor such as a stylus or a surface contour probe.
- the physical property sensor 7 may comprise any sensor, using any suitable modality, that is capable of resolving some physical property (e.g. physical attribute or physical feature) of the target surface 5.
- Such physical properties may include one or more of colour, shade, brightness, pattern (including writing, text, numerals, symbols, images), reflectivity, absorptivity, temperature, surface texture, engraving, etching etc.
- the haptic device 20 may further include an illumination source 10 for illuminating a field of view 23 of the camera 7.
- An illumination source 10 could be provided for sensors using other modalities, if required.
- the haptic device 20 further includes a controller 6 which is configured to receive an output signal from the physical property sensor 7.
- the output signal of the sensor 7 provides a surface characteristic signal which is indicative of a physical property of the target surface 5, e.g. within the field of view 23.
- the controller 6 may also include a communications module 11 for communication with other modules in the haptic device and / or for communication with other modules external to the haptic device.
- the haptic device 20 also includes a power source or power module 13 for providing electrical power to the component parts.
- the power source may be any suitable type such as a battery or other charge storage device or a fuel cell. Alternatively, the device could be powered from an external power supply.
- the haptic device 20 may also include a tracking sensor 12 which is configured to determine the position of the device on or relative to the target surface 5. This could be achieved, for example, when the target surface 5 is a display screen of a tablet computer 9, by coordinate-tracking circuitry. In another arrangement, the haptic device 20 may include a stylus 8 or pointer configured to engage the target surface 5.
- the computer may therefore sense the position of the haptic device and feed this back to the tracking sensor 12 via communications module 11 , e.g. by way of a suitable communication channel between the computer and the haptic device which may be a wired or wireless communication channel such as Bluetooth.
- the controller 6 is configured to receive the surface characteristic signal from the physical property sensor 7 and generate the control signal for driving the active surface 2.
- the control signal may be generated as a function of the sensed physical property of the target surface 2 in a manner to be described.
- Figure 2 shows an external perspective view illustrating a cuboid, 'puck'-style haptic device 20 in use. Exemplary methods of use are now described.
- a user's finger-tip 1 is placed on the active surface 2 incorporated into the device housing 3.
- the device 20 is placed on the target surface 5 to be sampled and may be moved around the target surface 5 freely using the finger or hand.
- the device 20 may be in the form of a puck which glides freely over the target surface 5 when dragged by the finger 1.
- the device 20 could be gripped on two sides between thumb and second finger to move it around the active surface while the first finger rests on the active surface 2.
- the tactile pattern of transducer elements such as pins 22 that is presented by the active surface 2 may be updated continuously by the controller 6 according to the physical properties of the target surface sensed by the physical property sensor.
- This may be further updated according to the position of the device 3 on the target surface 5 and a tactile pattern required for that position.
- the tactile stimulus provided to the user can be derived from the visible display (such as images or text) using an imaging device as the physical property sensor 7.
- the field of view 23 of the imaging device 7 may be adjustable according to the size and resolution of the active surface 2.
- the tactile sensation produced by the active surface can be a direct representation of the sensed physical properties of the target surface, or an artificially generated (e.g. synthesised) representation of the physical properties of the target surface.
- the texture felt when passing the device over a smooth surface could appear smooth, while over a rough surface could be rough or characteristic or indicative of the surface.
- changes or boundaries in physical properties of the surface e.g. colour changes in an image
- these can be represented by corresponding spatial changes in the tactile sensation.
- the representation of the sensed physical properties could be an amplified version of texture or relief of the target surface.
- artificially synthesised representations could comprise Braille conversions of text, for example.
- the controller 6 may be configured with an algorithm to perform optical character recognition on the imaged surface 5 and then produce a control signal to generate a Braille pattern of one or more Braille characters on the active surface 2. Colours of the target surface could be represented by different tactile patterns.
- the controller 6 may be configured to generate a variable tactile modulation of the active surface as a function of colour and / or optical patterns or intensity of patterns displayed on the target surface 5.
- the representation of the sensed physical properties by the active surface could include, for example, (i) vibration of the active surface at one or more different frequencies and / or in one or more different patterns to represent one sensed physical property of the target surface, such as temperature, together with (ii) tactile texture to represent another physical property such as colour, together with (iii) relief / embossing to represent an image.
- the various different types of modulation of the active surface exemplified above could thus be implemented contemporaneously or sequentially to represent different sensed physical properties of the target surface.
- multiple physical property sensors may be deployed in the device, each providing a surface characteristic signal indicative of the relevant physical property or properties.
- the expression “multiple physical property sensors” is intended to encompass a multimodality sensor, such as a camera capable of sensing colour and light intensity.
- the controller may be configured to receive multiple surface characteristic signals from the sensors simultaneously or in a time multiplexed manner, and thereby generate multiple control signals for tactile modulation of the active surface contemporaneously or sequentially.
- the control signal or signals are thereby generated as a function of the physical property or properties of the target surface, and the expression "as a function of” is intended to encompass any transform from particular surface characteristic to a representative tactile modulation applied to the active surface.
- the device can be used in an instinctive manner by running the device over a surface with a finger in much the same way as feeling a surface or traversing lines of text on a page. If the device is passing over images, the active surface 2 can be modulated according to the shapes and boundaries in the image overwhich the device passes. If the device is passing over a line of text, the active surface 2 can be modulated to provide a succession of braille characters.
- the device could incorporate a guidance mechanism configured to track movement of the device relative to some feature of the target surface as the device is dragged over the target surface.
- the device may track its position relative to a line of text as the device traverses the target surface. Deviation from the line of text or other feature of the target surface could be fed back to the user by some suitable feedback modality (e.g. sound, vibration etc) to enable the user to maintain a correct track across the target surface.
- the haptic device 20 is being used in conjunction with a touch-sensitive screen 9 (such as on a tablet, laptop or smartphone), the position of the device may be tracked using sensing of the position of the device by the touch screen, and the tactile pattern / modulation of the active surface 2 may be modified or adapted for that position and notified to the device 20 by the communication channel to the communications module 11.
- a touch-sensitive screen 9 such as on a tablet, laptop or smartphone
- the sensed physical property from the physical sensor 7, being represented on the active surface 2 could be adapted, modified or otherwise controlled according to the computer display characteristic known to be present at the position of the device.
- the computer may indicate whether the controller should be producing a control signal for the tactile transducer 4 that is indicative of characters and numerals (e.g. to be represented by Braille) or graphics (e.g. to be represented by texture or image boundary tactile sensations).
- the computer can effectively perform mode-switching of the haptic device and could mitigate some processing burden on the controller of the haptic device.
- the computer may sense the position of the haptic device on the screen 9 and communicate to the controller 6 the type of control signal to be produced according to the display at that position.
- the controller may include an OCR system configured to distinguish between graphics and text.
- the haptic device 20 described can make flat or untextured surfaces appear to have texture, emulating the sensation of touch, particularly as felt through a user's fingertip.
- the device can be configured as a compact and low cost means for providing a user with a tactile representation of a target surface independent of the actual tactile properties of the target surface, e.g. paper or the glass of an electronic tablet display screen, for example.
- the haptic device can be conveniently made self-contained and easily portable to fit in a pocket etc.
- the device 20 is suited to a wide range of target surface types - passive surfaces such as paper or active surfaces such as computer display screens.
- the texture sensation delivered by the active surface 2 to the user can be generated directly from the visual properties of the surface (lightness, darkness, contrast or colours). For instance, the letters on a page of text can be raised for the visually impaired, or a photograph converted to a virtual raised surface. Text may be converted to Braille or any suitable code for the visually impaired.
- the device can be packaged as a small, self-contained unit such as a puck to be moved around by a finger tip or fingers, as described above.
- the housing of the puck could be any suitable shape.
- the device could alternatively be configured to be attached to the fingertip (as a thimble) for direct movement with and on the finger, or it could be integrated into a larger device such as a mouse pointing device for manipulation with the whole hand.
- one or more of the haptic devices could be integrated into a glove, e.g. at each finger end. Multiple devices could be used to attach to multiple finger tips.
- a single device may have multiple active areas for two or more fingers simultaneously.
- the device may be configured with a smooth low friction surface on its lower face 24 (such as using Teflon pads) to easily glide over the target surface.
- a smooth low friction surface such as using Teflon pads
- one or more rollers or similar wheels could be used.
- the human fingertip has particularly high sensitivity to surface texture with high spatial resolution. People have a natural ability to build an effective internal representation of a surface from touch alone and do not need to feel the whole surface simultaneously. This makes the device very compact and instinctive to use.
- An exemplary device could have external width dimensions of 1 to 4 cm when in the "puck" format, e.g. suitable for an active surface sized for sensing by a user's fingertip area of approximately 1 cm x 1 cm. Other dimensions of device may be used for other contexts. Advantages of the haptic device include the following.
- Text can either be recognised by its actual shape or by using Optical Character Recognition (OCR) software to translate the character(s) into a pre-defined pattern, such as Braille.
- OCR Optical Character Recognition
- Visually impaired users can either scan ordinary printed books or computer screen surfaces to read the content by sliding the haptic device over the text. Alternatively, the device may be scanned over notices or photographs to produce a tactile version of any image.
- Textures related to items shown pictorially can be generated synthetically allowing the user to gain an impression of surface properties.
- Colours on the target surface can be presented as virtual texturing (stippling or hatching, for instance) for the benefit of colour blind users.
- One or more target surface sense modes may be used for the device to function, such as optical (with an optical sensor 7), mechanical (e.g. with a stylus) and a positioning sensing system such as tracking sensor 12 for determining relative position on the target surface. 5. Buttons or controls may be incorporated into the device to allow the user to change operating mode as required.
- the functionality may be useful not only to blind and visually impaired users, but also users who may require interaction with a target surface by touch while maintaining visual focus elsewhere.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Veterinary Medicine (AREA)
- Public Health (AREA)
- Animal Behavior & Ethology (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Business, Economics & Management (AREA)
- Life Sciences & Earth Sciences (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Vascular Medicine (AREA)
- Heart & Thoracic Surgery (AREA)
- Biomedical Technology (AREA)
- Ophthalmology & Optometry (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
A haptic device comprises an active surface comprising a tactile transducer configured to generate variable tactile modulation of the active surface according to a control signal. A physical property sensor is configured to determine a physical property of a target surface adjacent to the haptic device and to provide a surface characteristic signal indicative of the physical property thereof. A controller is configured to receive the surface characteristic signal from the physical property sensor and to generate the control signal as a function of the physical property of the target surface to thereby drive modulation of the active surface.
Description
APPARATUS AND METHOD FOR PROVIDING TACTILE STIMULUS
The present invention relates to haptic devices for providing tactile stimuli to users. Visually impaired persons may place a greater reliance on tactile stimuli to determine properties of objects into which they come into contact. However, many surface properties of objects may not readily provide an appropriate tactile stimulus. Examples may include optical properties such as colour, shade, brightness, pattern (including writing and images printed or displayed on the surface), reflectivity, absorptivity etc. Further examples may include surface texture which has sufficiently fine features that the normal sensitivity to touch of a human finger is not capable of resolving detail of the surface texture.
An object of the invention is to provide a convenient, portable device suitable for providing a tactile stimulus to a user based on or derived from features of a target surface, to assist the user in determining physical properties of that target surface which might otherwise be difficult or impossible to assess using unaided touch.
According to one aspect, the present invention provides a haptic device comprising: an active surface comprising a tactile transducer configured to generate variable tactile modulation of the active surface according to a control signal;
a physical property sensor configured to determine a physical property of a target surface adjacent to the haptic device and to provide a surface characteristic signal indicative of the physical property thereof;
a controller configured to receive the surface characteristic signal from the physical property sensor and to generate the control signal as a function of the physical property of the target surface.
The active surface may have a surface area corresponding to the area of the distal phalanx of a user's finger or thumb. The active surface may be disposed directly over the physical property sensor such that, in use, the target surface lies immediately below a user's finger resting on the active surface. The physical property sensor may comprise one or more of an electromagnetic sensor, an optical sensor, a mechanical sensor, a stylus, and a surface contour probe. The physical property sensor may comprise an optical sensor. The haptic device may further include an optical character recognition engine configured to provide a character signal as the control signal. The controller may
be configured to generate the variable tactile modulation of the active surface to simulate one or more Braille characters corresponding to the character signal. The physical property sensor may comprise an optical sensor and the controller may be configured to generate the variable tactile modulation of the active surface as a function of colour and / or optical patterns on the target surface. The physical property sensor may comprise an optical sensor and the controller may be configured to generate the variable tactile modulation of the active surface as a function of intensity of patterns on the target surface. The controller may be configured to generate the variable tactile modulation of the active surface as a function of texture of the target surface. The haptic device may be configured as a puck or a mouse for positioning between the target surface and a user's finger. The haptic device may be configured to roll or glide over the target surface by manipulation with a single hand or finger. The haptic device may be configured as a glove or thimble for attachment over a user's finger or hand. The haptic device may be configured to provide a plurality of surface characteristic signals indicative of a respective plurality of physical properties. The controller may be configured to generate control signals as a function of the plurality of physical properties so as to cause tactile modulation of the active surface representative of the plural physical properties contemporaneously or sequentially. According to another aspect, the present invention provides a method of providing a tactile representation of physical properties of a target surface comprising:
sensing a physical property of a target surface adjacent to a haptic device and generating a surface characteristic signal indicative of the physical property thereof; receiving the surface characteristic signal from the physical property sensor and generating a control signal as a function of the physical property of the target surface; generating a variable tactile modulation of an active surface of the haptic device according to the control signal.
Embodiments of the present invention will now be described by way of example and with reference to the accompanying drawings in which:
Figure 1 shows a schematic diagram of components of a haptic device for providing a user with tactile stimuli corresponding to physical features of a target surface adjacent to the haptic device;
Figure 2 shows a perspective view of an external arrangement of haptic device similar to that of figure 1.
Throughout the present specification, the descriptors relating to relative orientation and position, such as "top", "bottom", "horizontal", "vertical", "left", "right", "up", "down", "front", "back", as well as any adjective and adverb derivatives thereof, are used in the sense of the orientation of device as presented in the drawings. However, such descriptors are not intended to be in any way limiting to an intended use of the described or claimed invention.
With reference to figure 1 , a haptic device 20 comprises a housing 3 (shown in dashed outline) which may be in the approximate form of a cube or cuboid. The housing 3 is seen resting on a target surface 5. On an upper or top surface 21 of the housing 3 is an active surface 2. The active surface comprises a tactile transducer 4 which is configured to generate variable tactile modulation of the active surface 2 according to a suitable control signal, such that a user whose finger 1 is positioned over the active surface 2 will be provided with a suitable tactile stimulus.
In the illustrated arrangement of figure 1 , the tactile transducer 4 comprises an array of electrically actuatable pins 22 each of which can be driven in an axial direction, i.e. up and down as viewed in figure 1. In doing so, the upper ends of the pins 22 define the active surface 2 which can be modulated over its surface area according to the relative positions of each of the array of pins 4.
Other forms of active surface 2 may be deployed. For example, in other configurations, the pins 22 may lie beneath a flexible, displaceable membrane or other surface layer and operate to locally displace the membrane in the vicinity of the respective pin. Alternatively, the active surface may be implemented by way of a sheet having an array of compartments each of which can be independently inflated and deflated to provide tactile modulation of the active surface. In a general aspect, the tactile transducer 4 may comprise any suitable mechanism capable of locally displacing regions of an active surface 2, e.g. in a direction orthogonal to a rest plane or reference plane of the active surface.
Also provided within the housing 3 of the haptic device 20 is a physical property sensor 7 which is configured to determine a physical property of the target surface 5 adjacent to the haptic device 20. In the arrangement shown, the physical property sensor 7 is disposed on or associated with a lower or bottom surface 24 in a position that is preferably opposite to and in approximate alignment with the active surface 2. With this configuration, in use,
the active surface 2 may be positioned directly over the physical property sensor 7 such that a user's finger 1 resting on the active surface will lie directly over a sensed part of the target surface 5. In the preferred arrangement shown, the active surface 2 has a surface area which approximately corresponds to the area of the distal phalanx of a user's finger or thumb.
In the arrangement shown in figure 1 , the physical property sensor 7 comprises an optical imaging device such as a camera, a laser scanner, a light sensor, an infra-red sensor. Other sensors are possible, e.g. any electromagnetic sensing device such as an optical or thermal imaging sensor, an ultrasound sensor, or a mechanical sensor such as a stylus or a surface contour probe. In a general aspect, the physical property sensor 7 may comprise any sensor, using any suitable modality, that is capable of resolving some physical property (e.g. physical attribute or physical feature) of the target surface 5. Such physical properties may include one or more of colour, shade, brightness, pattern (including writing, text, numerals, symbols, images), reflectivity, absorptivity, temperature, surface texture, engraving, etching etc.
For a camera-type sensor 7 as shown in figure 1 , the haptic device 20 may further include an illumination source 10 for illuminating a field of view 23 of the camera 7. An illumination source 10 could be provided for sensors using other modalities, if required.
The haptic device 20 further includes a controller 6 which is configured to receive an output signal from the physical property sensor 7. The output signal of the sensor 7 provides a surface characteristic signal which is indicative of a physical property of the target surface 5, e.g. within the field of view 23.
The controller 6 may also include a communications module 11 for communication with other modules in the haptic device and / or for communication with other modules external to the haptic device.
The haptic device 20 also includes a power source or power module 13 for providing electrical power to the component parts. The power source may be any suitable type such as a battery or other charge storage device or a fuel cell. Alternatively, the device could be powered from an external power supply.
The haptic device 20 may also include a tracking sensor 12 which is configured to determine the position of the device on or relative to the target surface 5. This could be achieved, for example, when the target surface 5 is a display screen of a tablet computer 9, by coordinate-tracking circuitry. In another arrangement, the haptic device 20 may include a stylus 8 or pointer configured to engage the target surface 5. Where the target surface 5 is an active or touch sensitive screen of a computer display 9, the computer may therefore sense the position of the haptic device and feed this back to the tracking sensor 12 via communications module 11 , e.g. by way of a suitable communication channel between the computer and the haptic device which may be a wired or wireless communication channel such as Bluetooth.
The controller 6 is configured to receive the surface characteristic signal from the physical property sensor 7 and generate the control signal for driving the active surface 2. The control signal may be generated as a function of the sensed physical property of the target surface 2 in a manner to be described.
Figure 2 shows an external perspective view illustrating a cuboid, 'puck'-style haptic device 20 in use. Exemplary methods of use are now described.
A user's finger-tip 1 is placed on the active surface 2 incorporated into the device housing 3. The device 20 is placed on the target surface 5 to be sampled and may be moved around the target surface 5 freely using the finger or hand. For example, the device 20 may be in the form of a puck which glides freely over the target surface 5 when dragged by the finger 1. Alternatively, the device 20 could be gripped on two sides between thumb and second finger to move it around the active surface while the first finger rests on the active surface 2. The tactile pattern of transducer elements such as pins 22 that is presented by the active surface 2 may be updated continuously by the controller 6 according to the physical properties of the target surface sensed by the physical property sensor. This may be further updated according to the position of the device 3 on the target surface 5 and a tactile pattern required for that position.
In this way, the tactile stimulus provided to the user can be derived from the visible display (such as images or text) using an imaging device as the physical property sensor 7. The field of view 23 of the imaging device 7 may be adjustable according to the size and resolution of the active surface 2.
The tactile sensation produced by the active surface can be a direct representation of the sensed physical properties of the target surface, or an artificially generated (e.g. synthesised) representation of the physical properties of the target surface. For instance, the texture felt when passing the device over a smooth surface could appear smooth, while over a rough surface could be rough or characteristic or indicative of the surface. When passing over changes or boundaries in physical properties of the surface, e.g. colour changes in an image, these can be represented by corresponding spatial changes in the tactile sensation. The representation of the sensed physical properties could be an amplified version of texture or relief of the target surface.
Alternatively, artificially synthesised representations could comprise Braille conversions of text, for example. In such an arrangement, the controller 6 may be configured with an algorithm to perform optical character recognition on the imaged surface 5 and then produce a control signal to generate a Braille pattern of one or more Braille characters on the active surface 2. Colours of the target surface could be represented by different tactile patterns. In a general aspect, the controller 6 may be configured to generate a variable tactile modulation of the active surface as a function of colour and / or optical patterns or intensity of patterns displayed on the target surface 5.
In another example, the representation of the sensed physical properties by the active surface could include, for example, (i) vibration of the active surface at one or more different frequencies and / or in one or more different patterns to represent one sensed physical property of the target surface, such as temperature, together with (ii) tactile texture to represent another physical property such as colour, together with (iii) relief / embossing to represent an image. The various different types of modulation of the active surface exemplified above could thus be implemented contemporaneously or sequentially to represent different sensed physical properties of the target surface. In a general aspect, multiple physical property sensors may be deployed in the device, each providing a surface characteristic signal indicative of the relevant physical property or properties. (The
expression "multiple physical property sensors" is intended to encompass a multimodality sensor, such as a camera capable of sensing colour and light intensity.) The controller may be configured to receive multiple surface characteristic signals from the sensors simultaneously or in a time multiplexed manner, and thereby generate multiple control signals for tactile modulation of the active surface contemporaneously or sequentially. The control signal or signals are thereby generated as a function of the physical property or properties of the target surface, and the expression "as a function of" is intended to encompass any transform from particular surface characteristic to a representative tactile modulation applied to the active surface.
The device can be used in an instinctive manner by running the device over a surface with a finger in much the same way as feeling a surface or traversing lines of text on a page. If the device is passing over images, the active surface 2 can be modulated according to the shapes and boundaries in the image overwhich the device passes. If the device is passing over a line of text, the active surface 2 can be modulated to provide a succession of braille characters.
In another arrangement, the device could incorporate a guidance mechanism configured to track movement of the device relative to some feature of the target surface as the device is dragged over the target surface. For example, the device may track its position relative to a line of text as the device traverses the target surface. Deviation from the line of text or other feature of the target surface could be fed back to the user by some suitable feedback modality (e.g. sound, vibration etc) to enable the user to maintain a correct track across the target surface.
If the haptic device 20 is being used in conjunction with a touch-sensitive screen 9 (such as on a tablet, laptop or smartphone), the position of the device may be tracked using sensing of the position of the device by the touch screen, and the tactile pattern / modulation of the active surface 2 may be modified or adapted for that position and notified to the device 20 by the communication channel to the communications module 11.
In this way, the sensed physical property from the physical sensor 7, being represented on the active surface 2 could be adapted, modified or otherwise controlled according to the computer display characteristic known to be present at the position of the device. For example, the computer may indicate whether the controller should be producing a control
signal for the tactile transducer 4 that is indicative of characters and numerals (e.g. to be represented by Braille) or graphics (e.g. to be represented by texture or image boundary tactile sensations). In this way, the computer can effectively perform mode-switching of the haptic device and could mitigate some processing burden on the controller of the haptic device.
For example, for a mixed display of text and graphics, the computer may sense the position of the haptic device on the screen 9 and communicate to the controller 6 the type of control signal to be produced according to the display at that position. Alternatively, the controller may include an OCR system configured to distinguish between graphics and text.
The haptic device 20 described can make flat or untextured surfaces appear to have texture, emulating the sensation of touch, particularly as felt through a user's fingertip. The device can be configured as a compact and low cost means for providing a user with a tactile representation of a target surface independent of the actual tactile properties of the target surface, e.g. paper or the glass of an electronic tablet display screen, for example. The haptic device can be conveniently made self-contained and easily portable to fit in a pocket etc.
The device 20 is suited to a wide range of target surface types - passive surfaces such as paper or active surfaces such as computer display screens. The texture sensation delivered by the active surface 2 to the user can be generated directly from the visual properties of the surface (lightness, darkness, contrast or colours). For instance, the letters on a page of text can be raised for the visually impaired, or a photograph converted to a virtual raised surface. Text may be converted to Braille or any suitable code for the visually impaired.
The device can be packaged as a small, self-contained unit such as a puck to be moved around by a finger tip or fingers, as described above. The housing of the puck could be any suitable shape. The device could alternatively be configured to be attached to the fingertip (as a thimble) for direct movement with and on the finger, or it could be integrated into a larger device such as a mouse pointing device for manipulation with the whole hand. In another configuration, one or more of the haptic devices could be integrated into a glove, e.g. at each finger end. Multiple devices could be used to attach to multiple finger tips. A
single device may have multiple active areas for two or more fingers simultaneously. In some arrangements, such as a puck or mouse, the device may be configured with a smooth low friction surface on its lower face 24 (such as using Teflon pads) to easily glide over the target surface. Alternatively or in addition, one or more rollers or similar wheels could be used.
The human fingertip has particularly high sensitivity to surface texture with high spatial resolution. People have a natural ability to build an effective internal representation of a surface from touch alone and do not need to feel the whole surface simultaneously. This makes the device very compact and instinctive to use. An exemplary device could have external width dimensions of 1 to 4 cm when in the "puck" format, e.g. suitable for an active surface sized for sensing by a user's fingertip area of approximately 1 cm x 1 cm. Other dimensions of device may be used for other contexts. Advantages of the haptic device include the following.
1. Text can either be recognised by its actual shape or by using Optical Character Recognition (OCR) software to translate the character(s) into a pre-defined pattern, such as Braille. Visually impaired users can either scan ordinary printed books or computer screen surfaces to read the content by sliding the haptic device over the text. Alternatively, the device may be scanned over notices or photographs to produce a tactile version of any image.
2. Textures related to items shown pictorially can be generated synthetically allowing the user to gain an impression of surface properties.
3. Colours on the target surface can be presented as virtual texturing (stippling or hatching, for instance) for the benefit of colour blind users. 4. One or more target surface sense modes may be used for the device to function, such as optical (with an optical sensor 7), mechanical (e.g. with a stylus) and a positioning sensing system such as tracking sensor 12 for determining relative position on the target surface.
5. Buttons or controls may be incorporated into the device to allow the user to change operating mode as required.
6. Multiple active surfaces can be arranged under several fingertips to extend the resolution and reach of the device
7. The functionality may be useful not only to blind and visually impaired users, but also users who may require interaction with a target surface by touch while maintaining visual focus elsewhere.
It would also be possible to provide a direct or indirect viewing "window" for the device such that the target surface being sensed by the physical property sensor is also visible or displayed on the device or elsewhere to provide direct visual correlation with the sensed physical property.
Other embodiments are intentionally within the scope of the accompanying claims.
Claims
1. A haptic device comprising:
an active surface comprising a tactile transducer configured to generate variable tactile modulation of the active surface according to a control signal;
a physical property sensor configured to determine a physical property of a target surface adjacent to the haptic device and to provide a surface characteristic signal indicative of the physical property thereof;
a controller configured to receive the surface characteristic signal from the physical property sensor and to generate the control signal as a function of the physical property of the target surface.
2. The haptic device of claim 1 in which the active surface has a surface area corresponding to the area of the distal phalanx of a user's finger or thumb.
3. The haptic device of claim 1 in which the active surface is disposed directly over the physical property sensor such that, in use, the target surface lies immediately below a user's finger resting on the active surface.
4. The haptic device of claim 1 in which the physical property sensor comprises one or more of an electromagnetic sensor, an optical sensor, a mechanical sensor, a stylus, and a surface contour probe.
5. The haptic device of claim 4 in which the physical property sensor comprises an optical sensor and the haptic device further includes an optical character recognition engine configured to provide a character signal as the control signal.
6. The haptic device of claim 5 in which the controller is configured to generate the variable tactile modulation of the active surface to simulate one or more Braille characters corresponding to the character signal.
7. The haptic device of claim 4 in which the physical property sensor comprises an optical sensor and the controller is configured to generate the variable tactile modulation of the active surface as a function of colour and / or optical patterns on the target surface.
8. The haptic device of claim 4 in which the physical property sensor comprises an optical sensor and the controller is configured to generate the variable tactile modulation of the active surface as a function of intensity of patterns on the target surface.
9. The haptic device of claim 4 in which the controller is configured to generate the variable tactile modulation of the active surface as a function of texture of the target surface.
10. The haptic device of any preceding claim configured as a puck or a mouse for positioning between the target surface and a user's finger.
1 1. The haptic device of claim 10 further configured to roll or glide over the target surface by manipulation with a single hand or finger.
12. The haptic device of claim 1 configured as a glove or thimble for attachment over a user's finger or hand.
13. The haptic device of claim 1 configured to provide a plurality of surface characteristic signals indicative of a respective plurality of physical properties, and wherein the controller is configured to generate control signals as a function of the plurality of physical properties so as to cause tactile modulation of the active surface representative of the plural physical properties contemporaneously or sequentially.
14. A method of providing a tactile representation of physical properties of a target surface comprising:
sensing a physical property of a target surface adjacent to a haptic device and generating a surface characteristic signal indicative of the physical property thereof; receiving the surface characteristic signal from the physical property sensor and generating a control signal as a function of the physical property of the target surface; generating a variable tactile modulation of an active surface of the haptic device according to the control signal.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1718051.4 | 2017-11-01 | ||
GBGB1718051.4A GB201718051D0 (en) | 2017-11-01 | 2017-11-01 | apparatus and method for providing tactile stimulus |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2019086885A1 true WO2019086885A1 (en) | 2019-05-09 |
Family
ID=60580180
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/GB2018/053174 WO2019086885A1 (en) | 2017-11-01 | 2018-11-01 | Apparatus and method for providing tactile stimulus |
Country Status (2)
Country | Link |
---|---|
GB (1) | GB201718051D0 (en) |
WO (1) | WO2019086885A1 (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1997030415A1 (en) * | 1996-02-13 | 1997-08-21 | Sears James T | Tactiley-guided, voice-output reading apparatus |
US6159013A (en) * | 1996-01-19 | 2000-12-12 | Parienti; Raoul | Portable reading device for the blind |
WO2002006916A2 (en) * | 2000-07-18 | 2002-01-24 | Yishay Langenthal | Reading aid for the blind |
DE102005001064A1 (en) * | 2005-01-07 | 2006-07-20 | Jonas Baumann | Blind person's reading device directly for converting printed text into Braille by use of a finger-mounted unit with a programmable electronic circuit that reads in the text and converts it into Braille via a tactile output |
US20110155044A1 (en) * | 2007-12-21 | 2011-06-30 | David Burch | Kinesthetically concordant optical, haptic image sensing device |
US20160224116A1 (en) * | 2013-10-18 | 2016-08-04 | Douglas Hagedorn | Systems and Methods for Non-Visual Spatial Interfacing with a Computer |
-
2017
- 2017-11-01 GB GBGB1718051.4A patent/GB201718051D0/en not_active Ceased
-
2018
- 2018-11-01 WO PCT/GB2018/053174 patent/WO2019086885A1/en active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6159013A (en) * | 1996-01-19 | 2000-12-12 | Parienti; Raoul | Portable reading device for the blind |
WO1997030415A1 (en) * | 1996-02-13 | 1997-08-21 | Sears James T | Tactiley-guided, voice-output reading apparatus |
WO2002006916A2 (en) * | 2000-07-18 | 2002-01-24 | Yishay Langenthal | Reading aid for the blind |
DE102005001064A1 (en) * | 2005-01-07 | 2006-07-20 | Jonas Baumann | Blind person's reading device directly for converting printed text into Braille by use of a finger-mounted unit with a programmable electronic circuit that reads in the text and converts it into Braille via a tactile output |
US20110155044A1 (en) * | 2007-12-21 | 2011-06-30 | David Burch | Kinesthetically concordant optical, haptic image sensing device |
US20160224116A1 (en) * | 2013-10-18 | 2016-08-04 | Douglas Hagedorn | Systems and Methods for Non-Visual Spatial Interfacing with a Computer |
Also Published As
Publication number | Publication date |
---|---|
GB201718051D0 (en) | 2017-12-13 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN208013965U (en) | Optical fingerprint sensor arrangement under display for mitigating Moire effect | |
AU722853B2 (en) | Mouse-like input/output device with display screen and method for its use | |
Wall et al. | Sensory substitution using tactile pin arrays: Human factors, technology and applications | |
US6278441B1 (en) | Tactile interface system for electronic data display system | |
Heller et al. | Psychology of touch and blindness | |
Loomis et al. | Tactual perception | |
US6037882A (en) | Method and apparatus for inputting data to an electronic system | |
US7791597B2 (en) | Uniquely identifiable inking instruments | |
US9760241B1 (en) | Tactile interaction with content | |
JP3543695B2 (en) | Driving force generator | |
KR20180112847A (en) | A tactile information conversion device, a tactile information conversion method, and a tactile information conversion program | |
van Erp | Tactile navigation display | |
KR20180123525A (en) | A tactile information conversion apparatus, a tactile information conversion method, and a tactile information conversion program, and a device arrangement structure | |
US20110155044A1 (en) | Kinesthetically concordant optical, haptic image sensing device | |
WO2009073262A1 (en) | User input using proximity sensing | |
Bornschein et al. | Comparing computer-based drawing methods for blind people with real-time tactile feedback | |
Kato et al. | Double-sided printed tactile display with electro stimuli and electrostatic forces and its assessment | |
Mattioni et al. | The effects of verbal cueing on implicit hand maps | |
US8610965B2 (en) | Reproduction device, assembly of a reproductive device and an indication body, and a method for reproducing an image portion | |
KR101360980B1 (en) | Writing utensil-type electronic input device | |
Wang et al. | A haptic memory game using the STReSS2 tactile display | |
Postma et al. | Keep an eye on your hands: on the role of visual mechanisms in processing of haptic space | |
CN102822781A (en) | Coordinate input device, and program | |
WO2019086885A1 (en) | Apparatus and method for providing tactile stimulus | |
Heller | Influence of visual guidance on braille recognition: Low lighting also helps touch |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18800292 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 18800292 Country of ref document: EP Kind code of ref document: A1 |