WO2011117794A1 - Methods and devices for tactilely imparting information - Google Patents

Methods and devices for tactilely imparting information Download PDF

Info

Publication number
WO2011117794A1
WO2011117794A1 PCT/IB2011/051157 IB2011051157W WO2011117794A1 WO 2011117794 A1 WO2011117794 A1 WO 2011117794A1 IB 2011051157 W IB2011051157 W IB 2011051157W WO 2011117794 A1 WO2011117794 A1 WO 2011117794A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
gas
flow
nozzle
image
Prior art date
Application number
PCT/IB2011/051157
Other languages
French (fr)
Inventor
Boaz Ben-Moshe
Nir Shvalb
Original Assignee
Ariel - University Research And Development Company, Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ariel - University Research And Development Company, Ltd. filed Critical Ariel - University Research And Development Company, Ltd.
Publication of WO2011117794A1 publication Critical patent/WO2011117794A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/016Input arrangements with force or tactile feedback as computer generated output to the user
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001Teaching or communicating with blind persons
    • G09B21/003Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays

Definitions

  • the invention in some embodiments, relates to the field of imparting information, for example image information, for example to the visually- impaired. More particularly, in some embodiments, the invention relates to methods and devices for imparting information, such as image information, as a modulated flow of gas to a skin surface of a subject.
  • Visual perception is the ability to interpret information from visible light reaching the eye, achieved by the visual system.
  • the lens of the eye focuses an image of the surroundings onto the photoreceptive cells of the retina, which detect the photons of light in the image and respond by producing neural impulses that represent the image.
  • the neural impulses are sent as image information to the brain.
  • Blindness is the condition of lacking visual perception due to physiological or neurological factors. The extent of vision loss may vary. Total blindness involves complete loss of perception of form and visual light. Legal blindness is defined in North America and most of Europe as visual acuity of 20/200 (6/6) or less in the better eye with best correction possible. Approximately ten percent of those deemed legally blind have no vision; the rest have some vision, from light perception alone to relatively good acuity.
  • Visual perception enables humans to navigate or interact with the world, act fully independently, and be aware of events surrounding them.
  • Visually- imp aired and blind people require techniques and tools that allow them to complete daily activities using their remaining senses.
  • Such tools include a white cane, which is used to extend the user's range of touch sensation.
  • a white cane is usually swung in a low sweeping motion across the intended path of travel, to detect obstacles.
  • a white cane is only useful in detecting the presence of an object within the range of the cane.
  • a white cane does not provide any information regarding the appearance (size, shape, etc.) of an object and does not provide a complete image of what is before the user.
  • the invention in some embodiments, relates to methods and devices for imparting information to a human subject by directing a modulated flow of gas to a skin surface of the subject.
  • a method for tactilely imparting information to a subject comprising:
  • modulating a flow of gas in accordance with information to be imparted ; and directing the modulated flow of gas towards a skin surface of the subject thereby allowing the subject to tactilely sense the modulation of the flow of gas and thus to perceive the imparted information.
  • a device for tactilely imparting information to a subject comprising:
  • At least one nozzle configured to direct a flow of gas towards a skin surface of a subject
  • a controller for controlling the actuator to modulate the flow of gas in accordance with information to be imparted such that the modulated flow of gas constitutes a tactilely- sensible representation of information to be imparted.
  • the information is sensory information. In some embodiments, the information is visual information. In some embodiments, the information is image information, and the modulating of the flow of gas is such that the flow of gas constitutes a tactilely- sensible representation of the image.
  • Embodiments of methods and/or devices of the invention may involve performing or completing selected tasks manually, automatically, or a combination thereof.
  • Some embodiments of the invention are implemented with the use of components that comprise hardware, software, firmware or combinations thereof.
  • some components are general-purpose components such as general-purpose computers or processors.
  • some components are dedicated or custom components such as circuits, integrated circuits or software.
  • some of an embodiment is implemented as a plurality of software instructions executed by a data processor, for example which is part of a general-purpose or custom computer.
  • the data processor or computer comprises volatile memory for storing instructions and/or data and/or a non-volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data.
  • implementation includes a network connection.
  • implementation includes a user interface, generally comprising one or more of input devices (e.g., allowing input of commands and/or parameters) and output devices (e.g., allowing reporting parameters of operation and results.
  • FIGS. 1A and IB schematically depict an embodiment of a device as described herein worn by a user
  • FIG. 2 is a flow chart describing an embodiment of a method as described herein;
  • FIGS. 3A-3D schematically depict distance images (3A and 3C) and corresponding modulated pixelated gas-flow representing the distance images (3B and 3D, respectively);
  • FIGS. 4A-4C schematically depict an embodiment of a device as described herein; and
  • FIGS. 5 A and 5B schematically depict vector image-representations (5 A and 5D) of distance images (3A and 3C, respectively).
  • the invention in at least some embodiments, relates to methods and devices for imparting information tactilely, as a modulated flow of gas to a skin surface of a subject.
  • a method for tactilely imparting information to a subject comprising:
  • the subject is a human subject, in some embodiments a visually- impaired human subject.
  • the modulating of the flow of gas is such that the modulated flow of gas constitutes a representation of the information.
  • any suitable information is imparted using the method described herein.
  • any suitable information is imparted using the method described herein, for example sensory information, such as visual information, i.e., information usually perceived by a human using vision.
  • the visual information is image information and the modulating of the flow of gas is such that the flow of gas constitutes a tactile ly- sensible representation of the image.
  • the flow of gas is modulated so as to constitute a representation of the information to be imparted that is tactilely perceivable by the subject, allowing the subject to "feel" the information on the skin, for example, when the information is image information, the subject 'feels' the image on the skin.
  • the gas flow is modulated in order to provide information regarding the size and shape of an object, and the direction and distance of the object from the subject, as in an image.
  • the information is image information of a monochrome image In some embodiments, the information is image information of a color image. In some embodiments, the information is image information of a distance image.
  • information from any suitable source is imparted using the method described herein.
  • the information is real information, e.g., real acquired images of the real world.
  • the information is virtual information, e.g., generated images of a virtual world or reality.
  • the information is imparted in real-time, that is to say, imparted substantially immediately with acquisition (real information) or generation (virtual information).
  • the information is stored in a storage medium and is recovered from the storage medium and imparted when desired, at a time substantially different from when acquired (real information) or generated (virtual information).
  • the information is real information and the method further comprises acquiring the real information, and (substantially immediately) modulating the flow of gas to impart the real information in real-time.
  • the real information is acquired and imparted in real-time, that is to say, the real information is acquired and substantially immediately imparted to a user in accordance with the teachings herein.
  • the information is real image information of an image that is acquired and the information is imparted to the subject substantially immediately by modulating the flow of gas in accordance with the acquired image such that the flow of gas constitutes a tactile ly- sensible representation of the acquired image.
  • Such embodiments are useful, for example in allowing a visually-impaired person to "see" the real world and to navigate therein.
  • an image is a distance image acquired using a distance camera (also known as a range camera) using any suitable technology, e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect, Microsoft Corporation, Redmond, Washington, USA), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture techniques.
  • the image is a monochrome image or a color image acquired using a digital camera or image sensor as known in the art of digital photography and cellular telephony. Real-time imparting of virtual information
  • the information is virtual information (e.g., image information of a computer-generated image in a virtual world or reality) and the method further comprises generating the information, and (substantially immediately) modulating the flow of gas to impart the virtual information in real-time.
  • the information is generated and imparted in real-time, that is to say, the information is generated and substantially immediately imparted to a user in accordance with the teachings herein.
  • the information is image information of an image that is generated (in the usual way, for example as known in the field of animation or in the field of video gaming) and the information is imparted to the subject substantially immediately by modulating the flow of gas in accordance with the generated image such that the flow of gas constitutes a tactile ly- sensible representation of the generated image.
  • Such embodiments are useful, for example in allowing a visually-impaired person to "see” a virtual world, for example, allowing playing of an interactive video game or to consume animated video entertainment such as cartoons, stop-action films (e.g., Wallace and Gromit, Aardman Animations Ltd., Bristol, UK) or computer-generated films.
  • the information is stored, and the method further comprises retrieving the stored information and then modulating the flow of gas to impart the retrieved information, e.g., pre-generated virtual information such as image information of an animated movie generated and stored on electronic storage media or recorded real information such as real image information acquired using a camera as described above, recorded and stored on electronic storage media.
  • the stored information is retrieved and subsequently imparted to a user in accordance with the teachings herein.
  • Such embodiments are useful, for example, in allowing a visually- impaired person to view a movie, an animated movie or to train in using the method. Modulation
  • the modulation of the flow of gas may be any suitable modulation, that is to say, characteristics of the gas flow change as a function of time in a sensible way to impart information to the subject. It is important to note that it is known that a skin surface, especially root hair plexuses becomes desensitized to a constant stimulation such as a constant flow of air relatively relatively quickly but is very sensitive to a changing stimulation such as a fly or ant walking along the skin surface. Accordingly, modulation of a flow of gas for implementing the teachings herein changes to render the flow of gas tactilely sensible.
  • modulation comprises changing an intensity (mass of gas per unit area per unit time) of at least a portion of the gas-flow.
  • Typical usable intensities are similar to intensities of air exhaled by a person through the mouth.
  • a nearby object is represented by a more intense (higher rate of flow) gas-flow than a farther object to impart image information related to a distance image; or a darker- colored object is represented by a more intense gas-flow than a lighter- colored object to impart image information related to a monochrome or color image; or higher wavelength- colored (redder) object is represented by a more intense gas-flow than a lower-wavelength colored (e.g., more blue/indigo/violet) object to impart image information related to a color image.
  • modulation comprises changing a frequency of variation of intensity of at least a portion of the gas-flow.
  • Typical usable frequencies are between about 1 Hz and about 60 Hz.
  • a nearby object is represented by a more rapidly changing (higher frequency pulsed) gas-flow than a farther object to impart image information related to a distance image; or a darker-colored object is represented by a more rapidly changing gas-flow than a lighter- colored object to impart image information related to a monochrome image; or higher wavelength-colored (redder) object is represented by a more rapidly changing gas- flow than a lower- wavelength colored (more blue/indigo/violet) object to impart image information related to a color image.
  • modulation comprises changing a direction at which at least a portion of the gas flow reaches the skin surface.
  • changing a direction comprises changing an angle of incidence of the gas flow to the skin surface, typically between perpendicular (0°) to the skin surface and about 60° to the skin surface.
  • changing a direction comprises changing an orientation, in analogy to compass directions.
  • such gas flow is "pulsed", that is to say, there is a change of intensity as a function of time.
  • the intensity of such gas flow is substantially constant during the change of direction. For example, information about an object in an image (distance, monochrome or color) is "traced" on the skin surface of the subject by changing the direction of at least a portion of the gas flow.
  • modulation comprises changing a location of the skin surface at which at least a portion of the gas flow reaches the skin surface.
  • such gas flow is "pulsed", that is to say, there is a change of intensity as a function of time.
  • the intensity of such gas flow is substantially constant during the change of location. For example, an outline of an object in an image (distance, monochrome or color) is "traced" on the skin surface of the subject with at least a portion of the gas flow, optionally with a rate of change of intensity imparting distance information (e.g., from a distance image), intensity information (e.g., from a monochrome image) or color information (e.g., from a color image).
  • distance information e.g., from a distance image
  • intensity information e.g., from a monochrome image
  • color information e.g., from a color image
  • the flow of gas is pixelated, for example an object in an image (a distance, monochrome or color image) is "displayed" on the skin surface of the subject with a pixelated gas flow, optionally with a rate of change of intensity of each pixel imparting distance information (e.g., from a distance image), intensity information (e.g., from a monochrome image) or color information (e.g., from a color image).
  • distance information e.g., from a distance image
  • intensity information e.g., from a monochrome image
  • color information e.g., from a color image
  • the modulated flow of gas is directed to any suitable skin surface of a subject, preferably based on considerations of sensitivity to sensing modulation of a modulated air flow, interference with other tasks, aesthetic considerations and personal preference.
  • a preferred suitable skin surface is a skin surface with higher such sensitivity due to, for example, greater concentrations of touch-sensitive nerve-endings or root hair plexuses (light-touch mechanoreceptors that detect bending of hairs, enabling movement of the hairs to be detected even if the skin is not touched directly). That said, it is also preferred that the suitable skin surface be selected so that implementation of the teachings herein does not substantially interfere with performing other tasks.
  • a suitable skin surface is selected from the group consisting of a surface of an inner wrist, an arm, a nape of a neck, an earlobe, an ear canal, behind an ear, and especially a forearm.
  • other suitable skin surfaces with high sensitivity include skin surfaces selected from the group consisting of a surface of a hand, a lip, a tongue, a face and inside of a nostril.
  • skin surfaces selected from the group consisting of surfaces of a leg and of a torso.
  • Embodiments of the methods described herein are implemented using any suitable device. In some embodiments, it is preferred to use a device as described herein for implementing such methods.
  • a device for tactilely imparting information to a subject comprising:
  • At least one nozzle configured to direct a flow of gas towards a skin surface of a subject
  • a controller for controlling the actuator to modulate the flow of gas in accordance with information to be imparted such that a modulated flow of gas constitutes a tactilely- sensible representation of the information to be imparted.
  • a device as disclosed herein comprises at least one nozzle configured to direct a flow of gas towards a skin surface of a subject.
  • the at least one nozzle is one nozzle. In some embodiments, the at least one nozzle is at least two nozzles.
  • the at least one nozzle comprises an array of nozzles, e.g., an x by y array of nozzles, where x and y are independently integers of at least 4.
  • a nozzle array is useful in imparting image information as a pixelated flow of gas.
  • the controller activates an actuator so that each nozzle of the nozzle array corresponds to a pixel of an imparted image, and each pixel is "displayed” by directing a flow of gas at a skin surface from the corresponding nozzle.
  • an x by y array of pixels is "displayed" by an x by y array of nozzles. Conversion of image information to the pixel resolution of a nozzle array is easily performed using any suitable pixelization method or algorithm.
  • a device as disclosed herein comprises an actuator functionally associated with the at least one nozzle, the actuator for modulating the flow of gas from the at least one nozzles.
  • a typical actuator comprises one or more components for directing and changing an air flow from or through a nozzle such as valves, motors, step motors, bearings, moveable nozzle mounts, vanes, adjustable apertures, flaps and the like, for example, as known in the art of radio-controlled model aircraft.
  • the at least one nozzle is at least two nozzles
  • the actuator is configured to modulate a flow of gas from at least two of the nozzles independently, e.g., the at least one nozzle is a nozzle array and the actuator is configured to modulate the flow of gas from each of the nozzles of the nozzle array independently.
  • the actuator is configured to change an intensity of the gas- flow from at least one nozzle.
  • the actuator is configured to change a frequency of variation of intensity of the gas-flow from at least one nozzle.
  • the actuator is configured to change a direction of the gas-flow from at least one nozzle (in some embodiments the angle and/or in some embodiments the orientation in analogy to a compass), thereby changing a direction at which at least a portion of the gas-flow (the portion from that nozzle) is directed at a skin surface.
  • the actuator is configured to change a location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed. In some embodiments, the actuator is configured to translate at least one nozzle, thereby changing the location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed. In some embodiments, the actuator is configured to stop gas flow from at least one nozzle and to allow gas flow from at least one other nozzle, thereby changing the location of a skin surface at which at least a portion of the gas-flow is directed.
  • the actuator is configured to change a direction of the gas-flow from a nozzle (in some embodiments the angle and/or in some embodiments the orientation in analogy to a compass), thereby changing a location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed.
  • the actuator is configured to generate a pixelated flow of gas.
  • the at least one nozzle comprises a matrix of nozzles, each nozzle corresponding to a pixel and the actuator configured to control the flow of gas from each nozzle independently.
  • the at least one nozzle comprises a single nozzle and the actuator is configured to serially aim the nozzle (e.g., by rotating and/or translating the nozzle) to direct a flow of gas at different portions of a skin surface, each such portion constituting a pixel.
  • a device as disclosed herein comprises a controller for controlling an actuator to modulate the flow of gas in accordance with information to be imparted.
  • a controller is a component that, based on the information to be imparted, generates commands to the actuator that yield the appropriate modulation of the flow of gas such that the flow of gas constitutes a tactilely- sensible representation of the information.
  • a controller typically comprises a suitable general-purpose or custom-made device, for example an appropriately- configured general purpose digital computer (such as known in the field of "smartphones", e.g., Iphone 4 by Apple Incorporated, Cupertino, California, USA) with a user-input channel, an actuator-control output, a memory and a processor configured to convert information to be imparted to actuator-control output.
  • a suitable general-purpose or custom-made device for example an appropriately- configured general purpose digital computer (such as known in the field of "smartphones", e.g., Iphone 4 by Apple Incorporated, Cupertino, California, USA) with a user-input channel, an actuator-control output, a memory and a processor configured to convert information to be imparted to actuator-control output.
  • a user-input channel for example implemented as a touch screen with a graphic-user interface (such as well-known in the field of "smartphones") is used to control various parameters and user preferences for operating the device.
  • a graphic-user interface such as well-known in the field of "smartphones”
  • a controller is also configured to accept the information to be imparted.
  • a controller of a device further includes an information- input channel to accept information to be imparted to a user.
  • Such an information input channel is typically implemented as an electronic input (wired or wireless) to accept information to be imparted to a user.
  • the information is any information to be imparted as described hereinabove, including sensory information, visual information, and image information, in any suitable format, typical digital format.
  • the information is image information (e.g., of a distance image, of a monochrome image, of a color image) in the form of a digital image data file as known in the art.
  • a controller generally includes a processor configured to convert information to be imparted to actuator-control output, that is to say, commands for the actuator to modulate the flow of gas to represent the information to be imparted.
  • the processor functions as an image-representation generator, to convert image information to actuator-control output for the actuator to modulate the flow of gas to represent the image information.
  • the controller is configured to convert image information to actuator-control output for the actuator to modulate a flow of gas to constitute a tactilely- sensible pixelated representation of image information.
  • the processor functions as an image-representation generator to convert image information to actuator-control output for the actuator to produce a pixelated flow of gas to impart image information, including by directing a flow of gas (with optional changes in intensity and/or direction) at a plurality of distinct locations on the skin surface to "print" a pixelated representation of the image information to be imparted. Any suitable known method of image pixelation may be used in implementing such embodiments.
  • a plurality of pixels are "displayed" by a single nozzle, for example a single nozzle translatable in an x and y direction, or a linear array of nozzles moveable together in a single direction, or a rectangular array of fixed nozzles.
  • the size, shape and distance of an object in an image is represented by modulation of a pixelated gas-flow.
  • the image is represented such that portions (one or more pixels) of the gas flow representing closer objects change at higher frequency than portions (one or more pixels) of the gas flow representing farther objects.
  • an image is represented by a gas-flow made up of a 20 x 20 pixel array in a 5 cm x 5 cm array.
  • the controller is configured to convert image information to actuator-control output for the actuator to modulate a flow of gas to constitute a tactilely- sensible vector representation of image information.
  • the processor functions as an image-representation generator to convert image information to actuator-control output for the actuator to produce a vector image representation to impart image information, including by translating a nozzle with a continuous flow of gas (with optional changes in intensity and/or direction) to "draw" a vector representation of the image on the skin surface. Any suitable known method of vector image representation may be used in implementing such embodiments.
  • a device as disclosed herein further comprises a supporting component (in some embodiments also called a supporting backing component) to which the at least one nozzle is attached, the supporting component configured to be worn on a portion of a body of a subject so that the at least one nozzle is positioned to direct the flow of gas towards a skin surface of the subject.
  • a supporting component in some embodiments also called a supporting backing component
  • components in addition to the at least one nozzle such as the actuator and the controller are also attached to the supporting component.
  • the supporting component comprises a frame or container, e.g., of aluminum or plastic.
  • the supporting component includes a cover or shield to isolate a skin surface from external stimulus.
  • the supporting component is configured to be worn appropriately so that the at least one nozzle is positioned to direct the flow of gas towards a selected skin surface, as discussed above.
  • the skin surface is selected from the group consisting of a surface of an inner wrist, an arm, a nape of a neck, an earlobe, an ear canal, behind an ear, and especially of a forearm, of a hands, a tongue, a face and the inside of a nostril and in some embodiments, of a leg and of a torso.
  • the supporting component is configured to be worn on any suitable portion of the body of a subject as described above.
  • the supporting component typically includes (reversible) securing components to secure the supporting component in appropriate proximity and orientation to the selected skin surface, such as one or more of straps, snaps, ties, clips, hook-and-loop fasteners, and the like.
  • At least one nozzle attached to the supporting component faces inwards, so when the supporting component is properly worn by a subject, the at least one nozzle is directed at a skin surface of the subject.
  • At least one nozzle is immovably attached to the supporting component, so that flow of gas from the nozzle is directed to a specific location on a skin surface when the supporting component is properly worn.
  • At least one nozzle is rotatably attached to the supporting component, so that the direction of flow of gas from that nozzle can be changed by rotation of the nozzle by the actuator. In some embodiments, at least one nozzle is translatably attached to the supporting component so that the location on the skin surface at which a flow of gas from that nozzle is directed is changeable by translation of the nozzle by the actuator.
  • a device further comprises a power source (typically a rechargeable battery or similar) for supplying the power required for operation of components of the device such as the controller and actuator.
  • a power source typically a rechargeable battery or similar
  • such a power source is secured to the supporting component.
  • the flow of gas through the at least one nozzle is of any suitable gas.
  • a gas is inert and/or harmless and/or non-flammable, for example, air, CO2, helium, neon, or N 2 .
  • the flow of gas through the at least one nozzle is provided by any suitable component or device.
  • the device further comprises a gas-flow generating component to provide a flow of gas through the at least one nozzle.
  • the gas-flow generating component comprises a compressed gas container, e.g., of CO2, air, helium, neon or N2.
  • the gas-flow generating component comprises a gas-flow generator, e.g., a portable air pump, that generates a gas flow by drawing in and compressing ambient air.
  • the information to be imparted is provided to the controller in any suitable fashion.
  • information is stored on the controller.
  • the controller comprises an information input channel and information to be imparted is provided from any suitable source, e.g., a computer, a gaming console, a mobile telephone or an image storage device.
  • the device is configured to impart acquired information in real- time.
  • an information acquirer is functionally associated with the controller through an information input channel.
  • a device further comprises, in communication with the controller, an information acquirer to acquire information to be imparted and to provide the acquired information to the controller, and the controller is configured to control the actuator to modulate the flow of gas in accordance with the acquired information in real time.
  • the information acquirer is in wired communication with the controller.
  • the information acquirer is in wireless communication with the controller, (e.g., using Bluetooth® technology).
  • an information acquirer is a component of the device.
  • an information acquirer is an image acquirer, e.g., a camera.
  • an image acquirer is a distance camera (also known as a range camera) that generates an image of distance as a function of x and y, using any suitable technology, e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect from Microsoft, Redmond, Washington), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture.
  • a distance camera also known as a range camera
  • any suitable technology e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect from Microsoft, Redmond, Washington), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture.
  • an image acquirer is a monochrome or color camera, e.g., a visible light camera as known in the art of cellular telephony. In some embodiments, an image acquirer is an infrared camera or a thermal imaging camera.
  • the information acquirer is configured to be (reversibly) worn by the subject. In some embodiments, the information acquirer is configured to be worn on the head of the subject, e.g., attached to a head band, a cap or the like.
  • some device components are in wired communication with other device components.
  • some device components are in wireless communication with other device components, for example using Bluetooth® technology.
  • an image acquirer and a controller make up a single physical unit configured to be worn on the head of a subject while the actuator, at least one nozzle and gas-flow supply make up a single physical unit configured to be worn on the arm of a subject, where the controller and the actuator are in wireless communication.
  • the controller and the actuator are in wireless communication.
  • an image acquirer makes up a single physical unit configured to be worn on the head of a subject while the controller, the actuator, at least one nozzle and gas-flow supply make up a single physical unit configured to be worn on the arm of a subject, where the image acquirer and the controller are in wireless communication.
  • Such an embodiment allows a reduced size of components that are (uncomfortably) worn on the head.
  • Figures 1A and IB an embodiment of a device in accordance with the teachings herein including two separate physical units, an information imparting (actuator) unit 10 configured to be worn on the forearm of a human subject and an image-data acquiring (generation) unit 30 configured to be worn on the head of a human subject, is schematically depicted.
  • Information-imparting unit 10 comprises a plurality of nozzles 12, attached to a flexible supporting (backing surface) component 14 of silicone rubber, within a fixed-area rigid grid of polycarbonate to constitute a 5 by 5 nozzle array having a total of 25 nozzles.
  • Supporting component 14 comprises a reversible attachment component 24 (straps with hook-and loop fasteners such as Velcro®) for securing information-imparting unit 10 over a skin surface of a subject, a forearm 16, such that nozzles 12 face the skin surface.
  • Information-imparting unit 10 further comprises a reservoir 18 of compressed N2 gas in fluid communication with nozzles 12 through a manifold pipe (not apparent in the Figures).
  • An actuator 20 of the device comprises twenty- five individually- operable adjustable- aperture valves, each valve associated with a branch of the manifold pipe that directs nitrogen gas from reservoir 18 to a different nozzle 12.
  • Each valve of actuator 20 is configured to independently prevent gas flow from an associated nozzle 12, or to allow a certain gas flow to exit from an associated nozzle 12.
  • actuator 20 is configured to independently change an intensity of a gas-flow from each of nozzles 12 (by opening or closing an associated valve), to independently change frequency of variation of intensity of a gas-flow from each of nozzles 12 (by changing the rate of opening/closing of an associated valve) and is configured to change a location of a skin surface at which at least a portion of the gas-flow is directed (by opening a valve associated with a given nozzle 12 to allow gas flow from that nozzle while closing a valve associated with a different nozzle 12 to stop gas flow from that nozzle).
  • Image-data acquiring component 30 comprises a distance-image acquirer (image- acquisition component) time-of-flight camera 32, attached to a removable headband 34, for positioning on the head of the user, and a controller 36 (a digital computational device similar to a smartphone) configured to function as an image-representation generator, for accepting image information (data) acquired by camera 32 as input and for controlling release of gas from reservoir 18 as a modulated gas-flow through nozzles 12, by controlling actuator 20 to modulate the flow of gas from nozzles 12 in accordance with information to be imparted.
  • a controller 36 a digital computational device similar to a smartphone
  • controller 36 received an image from camera 32, pixelates the image to the resolution of the nozzle array (5 by 5) where the intensity of gas flow from a pixel is related to a distance to an object in the image (closer is more intense gas flow) and sends a series of commands in real time to control actuator 20 to modulate a gas-flow to constitute a representation of an image represented by the image data.
  • the commands are wirelessly transmitted by controller 36 using Bluetooth® transceiver 38 to Bluetooth transceiver 22 located on actuator component 10, in communication with actuator 20.
  • actuator 20 controls the flow of gas from each nozzle 12 (e.g., by opening and closing valves) in such a way that the image acquired by camera 32 is represented by a modulated pixelated gas-flow from the array of nozzles 12 so that the pixelated gas-flow constitutes a tactile ly- sensible representation of the image acquired by camera 32.
  • FIG. 2 presents an overview of an embodiment of a method of imparting information as described herein as a flow chart.
  • Camera 32 acquires an image and generates image data representing the image.
  • a controller 36 functioning as an image-representation generator accepts the image data as input and generates actuator commands that are sent to an actuator 20 through an actuator-control output.
  • An actuator 20 follows the actuator commands, modulating the flow of gas through the array of nozzles 12 to form a modulated pixelated gas-flow directed towards a skin surface so that the pixelated gas-flow constitutes a tactile ly- sensible representation of the image acquired by camera 32.
  • information such as image data is provided to a controller 36 by some component other than an image-acquisition component, e.g., a component that provides stored or computer-generated image data.
  • Figure 3A is a schematic depiction of a distance image acquired by a distance camera such as 32.
  • Figure 3B is a schematic depiction of a corresponding modulated pixelated gas flow generated by a nozzle array such as depicted in Figure IB.
  • Figure 3A are modulated by pulsing (changing the intensity of flow) at an 80 Hz frequency
  • air flowing in pixels corresponding to a further object (pole 44 at 4 meter) in the distance image is pulsed at 20 Hz frequency
  • air flowing in pixels corresponding to objects at infinite (greater than 8 meter) in the distance image is stopped completely.
  • Figure 3C is a schematic depiction of a distance image similar to that of Figure 3 A, where the camera is further away from the objects in the image.
  • Figure 3D is a schematic depiction of a corresponding modulated pixelated gas flow generated by a nozzle array such as depicted in Figure IB.
  • air flowing in pixels corresponding to a close object (person 42 at 4 meter) in the distance image are modulated by pulsing at an 20 Hz frequency
  • air flowing in pixels corresponding to a further object (pole 44 at 7 meters) in the distance image is pulsed at 10 Hz frequency
  • air flowing in pixels corresponding to objects at infinite (greater than 8 meter) in the distance image is stopped completely.
  • Device 50 is configured to acquire distance images and impart information from the acquired images to a subject using device 50 by modulating a flow of gas in accordance with the information to be imparted such that the modulated flow of gas constitutes a tactilely- sensible representation of information to be imparted.
  • Device 50 comprises two physically separate units: information- imparting unit 52 depicted in detail in Figures 4A and 4B configured to be worn on the forearm of a human subject and an image-data acquiring unit 54 depicted in detail in Figure 4C, configured to be worn on the head of a human subject.
  • Information-imparting unit 52 comprises a rigid supporting component 56, substantially a closed rigid box of polyethylene that acts to shield a selected skin surface from ambient stimulus and contains or supports other device components and includes straps 24 to allow information-imparting unit 52 to be reversibly worn on the forearm of a human subject.
  • a single nozzle 12 (hidden from view) configured to direct a flow of gas towards a skin surface of a subject properly wearing information-imparting unit 52;
  • controller 36 for controlling actuator 20 to modulate the flow of gas in accordance with image information to be imparted to a subject such that the modulated flow of gas constitutes a tactilely- sensible representation of the information
  • a portable air pump 58 as a gas-flow generator
  • a Li-ion battery 60 as a power source for the other components.
  • Actuator 20 comprises a rigid square 4cm by 4 cm frame 62 immovably- contained inside supporting component 56.
  • Two electrical step motors 64a and 64b (as known in the art of RC model airplanes) are immovably secured to two adjacent edges 62a and 62b of frame 62.
  • To a rotor 66a and 66b of each respective motor 64 is attached a proximal end of a respective slotted bar 68a and 68b. Closer to the distal ends of the bars, slotted bars 68a and 68b overlap.
  • a support pin 70 In the hole defined by the overlapping slots of slotted bars 68a and 68b is slidingly secured.
  • Nozzle 12 is a standard gas nozzle secured to support pin 70 so as to be directed perpendicularly to a skin surface when information-imparting unit 52 is properly worn by a human subject in fluid communication with portable air pump 58 through pipe 72.
  • Valve 74 comprises two controllable modules, needle valve 76 with an associated electrical motor that allows precise control of the rate of gas flow from air pump 58 to nozzle 12 and rotatable perforated disk 78 with an associated electrical motor that allows control and variation of the frequency of the change of air flow intensity from nozzle 12. Specifically, when a perforation of disk 78 is located across the lumen of pipe 72, air flows to nozzle 12 but when a solid portion of disk 78 is located across the lumen of pipe 72, air is blocked from passing to nozzle 12.
  • actuator 20 is configured to change an intensity of a gas-flow from nozzle 12 (using valve 74) to change a frequency of variation of intensity of a gas-flow from nozzle (by changing a rate of rotation of disk 78 of valve 74) and to change a location of a skin surface at which gas-flow from nozzle 12 is directed (using motors 64a and 64b to translate nozzle 12 inside frame 62).
  • Controller 36 (a portable digital computer, similar to that used in an Iphone 4 by Apple Incorporated, Cupertino, California, USA) having a touch-screen (not depicted) and associated hardware and software components constituting a user-input channel for accepting user commands, an information-input channel 80 in wired communication with a Bluetooth® transceiver 38 for accepting image information as an image in a standard format (e.g., any format produced by an image acquirer, see below) and a processor 82 for accepting image information from information-input channel 80, processor 82 configured to convert the accepted image information to commands to actuator 20 to modulate a flow of gas to represent the received image.
  • a standard format e.g., any format produced by an image acquirer, see below
  • Image-data acquiring unit 54 includes a distance camera 84 (similar to a Kinect by Microsoft Corporation, Redmond, Washington, USA) as a distance-image acquirer attached to a head band 34, a Bluetooth® transceiver 38 to provide wireless communication between distance camera 84 and controller 36, specifically, to information input channel 80 of controller 36. Not depicted is a battery supplying power for the components of image-data acquiring unit 54.
  • image acquiring unit 54 is worn on the head of a subject with the help of headband 34 and information-imparting unit 52 is worn on the forearm of a subject with the help of straps 24.
  • the various components of device 50 are activated.
  • Distance camera 84 acquires a distance image in the usual way and transmits the acquired image using transceiver 38 to controller 36 through information-input channel 80 via transceiver 22.
  • processor 82 of controller 36 applies a vector image representation algorithm to convert the acquired image received from distance camera 84 to a vector image, where objects in the image are represented by a set of vectors in a plane, each such vector having an intensity value related to the distance to the represented object.
  • controller 36 then controls actuator 20 to modulate the flow of gas directed by nozzle 12 towards the forearm skin surface of the subject to constitute a tactilely- sensible representation of information to be imparted.
  • controller 36 translates each vector in the vector image to a linear motion of nozzle 12 having a direction and length matching the vector, and an intensity of air flow related to the distance to an object represented by the vector, where greater intensity corresponds to a closer object.
  • controller 36 uses actuator 20 and nozzle 12 to "draw" a vector representation of the acquired image on the skin surface.
  • Figure 5A is a schematic depiction of a modulated gas flow directed at a skin surface by device 50 that is substantially a vector image representation of the distance image depicted in Figure 3 A.
  • the vertical line 44 at the left having a continuous low intensity air flow of 20 (arbitrary units) corresponds to a far object (pole 44 at 4 meter) while the three lines 42a, 42b and 42c having a continuous high intensity air flow of 80 (arbitrary units) correspond to a near object (person 42 at 1 meter).
  • Figure 5B is a schematic depiction of a modulated gas flow directed at a skin surface by device 50 that is substantially a vector image representation of the distance image depicted in Figure 3C.
  • the vertical arrow 44 at the left having a continuous low intensity air flow of 10 (arbitrary units) corresponds to a far object (pole 44 at 7 meter) while line 42 having a continuous high intensity air flow of 20 (arbitrary units) corresponds to a near object (person 42 at 4 meter).
  • actuator 20 of device 50 is also configured to generate a pixelated flow of gas, requiring only appropriate configuration of controller 36 to control actuator 20 to modulate the flow of gas through nozzle 12 to constitute a tactile ly- sensible pixelated representation of image information.
  • processor 82 of controller 36 is configured to produce a pixelated representation of an image to be imparted to a subject and subsequently processor 82 controls actuator 20 to serially translate nozzle 12 to the locations inside frame 62 that correspond to pixels, and at each pixel direct a flow of gas from nozzle 12, the flow of gas having the appropriate intensity or frequency.
  • imparting of image information is discussed in detail. As noted above, in some embodiments, other types of information are imparted to a user.
  • generally useful information is imparted to a user.
  • text information is imparted to the user, for example as a series of symbols, letters or numerals "drawn” or "printed” on a skin surface with a modulated flow of gas as described herein.
  • the approach of a time limit e.g., during playing a game, or when a specific action needs to be taken
  • a flow of gas having a periodic intensity, which intensity and/or frequency increase with the approach of the time limit.
  • sensory information e.g., information usually imparted by one of the five senses
  • visual information i.e., information usually perceived by a human using vision
  • a modulated flow of gas is used to impart information about the presence of an obstacle in front of a visually- imp aired person, not as an tactilely- sensible "image” of modulated gas, but as a symbolic "warning", for example a specifically modulated gas flow symbol.
  • Such information can be, for example, of a specific color (a red, amber or green traffic light), of a specific shape (e.g., a stop sign) or movement (e.g., a vehicle approaching from one of the sides of the subject).
  • imparting of information is imparted at one localized skin surface, specifically an area of skin on one forearm.
  • information is imparted concurrently (in some cases, simultaneously) at two different skin surfaces, e.g., two forearms, a forearm and the nape of the neck.
  • the information imparted at each of the two different surfaces is different, for example, one arm for image information and the other arm (or nape of the neck) for symbolic or complementary information.
  • the information imparted at the two different surfaces, taken together, represents, and is interpreted by the user as, a stereo image representation.
  • the teachings herein impart information about visual stimuli in the surroundings to a human subject, in a form which can be understood by such a subject.
  • the subject is provided with information regarding the size, shape and distance of an object within a detection range, without requiring actual contact with object.
  • the hands of the subject are left free, such that, for example, a visually- impaired subject may perform manual tasks while implementing the teachings herein, or optionally, use a white cane in addition to the teachings herein.
  • the teachings herein do not rely on audio signals, such that the hearing of the subject is still available to obtain additional information about the surroundings, increasing the total amount of information potentially imparted and received by the subject.
  • Embodiments of the teachings herein allow the visually- imp aired to participate in heretofore unavailable entertainment such as video gaming, watching movies and the like.
  • Embodiments of the teachings herein may also be used, for example, by normally- sighted rescue workers in smoke-filled buildings, in order to help locate trapped individuals.
  • imparting visual information using prior art methods may be impossible due to difficulties in viewing a visual display of any kind in the presence of dense smoke, particularly where the face of the rescuer may be at least partially covered with a gas mask or other breathing device.
  • Embodiments of the teachings herein may be used for imparting information in an additional, yet-unused, mode.
  • the teachings herein are implemented as an additional component of a game, for example a multimedia game.
  • information is imparted in other fields, for example, in controlling a vehicle (e.g., driving) or other fields where it is desired to impart information.

Abstract

Disclosed are methods and devices for imparting information to a human subject by directing a modulated flow of gas to a skin surface of the subject. Some embodiments are useful in imparting image information to a subject, for example a visually- imp aired subject.

Description

METHODS AND DEVICES FOR TACTILELY IMPARTING INFORMATION
RELATED APPLICATION
The present application gains priority from U.S. Provisional Patent Application No. 61/315,955 filed 21 March 2010, which is included by reference as if fully set forth herein.
FIELD AND BACKGROUND OF THE INVENTION
The invention, in some embodiments, relates to the field of imparting information, for example image information, for example to the visually- impaired. More particularly, in some embodiments, the invention relates to methods and devices for imparting information, such as image information, as a modulated flow of gas to a skin surface of a subject.
Visual perception is the ability to interpret information from visible light reaching the eye, achieved by the visual system. In humans, the lens of the eye focuses an image of the surroundings onto the photoreceptive cells of the retina, which detect the photons of light in the image and respond by producing neural impulses that represent the image. The neural impulses are sent as image information to the brain.
Blindness is the condition of lacking visual perception due to physiological or neurological factors. The extent of vision loss may vary. Total blindness involves complete loss of perception of form and visual light. Legal blindness is defined in North America and most of Europe as visual acuity of 20/200 (6/6) or less in the better eye with best correction possible. Approximately ten percent of those deemed legally blind have no vision; the rest have some vision, from light perception alone to relatively good acuity.
Visual perception enables humans to navigate or interact with the world, act fully independently, and be aware of events surrounding them. Visually- imp aired and blind people require techniques and tools that allow them to complete daily activities using their remaining senses. Such tools include a white cane, which is used to extend the user's range of touch sensation. A white cane is usually swung in a low sweeping motion across the intended path of travel, to detect obstacles.
A white cane is only useful in detecting the presence of an object within the range of the cane. A white cane does not provide any information regarding the appearance (size, shape, etc.) of an object and does not provide a complete image of what is before the user. SUMMARY OF THE INVENTION
The invention, in some embodiments, relates to methods and devices for imparting information to a human subject by directing a modulated flow of gas to a skin surface of the subject.
According to an aspect of some embodiments of the invention, there is provided a method for tactilely imparting information to a subject, comprising:
modulating a flow of gas in accordance with information to be imparted; and directing the modulated flow of gas towards a skin surface of the subject thereby allowing the subject to tactilely sense the modulation of the flow of gas and thus to perceive the imparted information.
According to an aspect of some embodiments of the invention there is provided a device for tactilely imparting information to a subject, comprising:
a) at least one nozzle configured to direct a flow of gas towards a skin surface of a subject;
b) functionally associated with the at least one nozzle, an actuator for modulating the flow of gas; and
c) a controller for controlling the actuator to modulate the flow of gas in accordance with information to be imparted such that the modulated flow of gas constitutes a tactilely- sensible representation of information to be imparted.
In some embodiments, the information is sensory information. In some embodiments, the information is visual information. In some embodiments, the information is image information, and the modulating of the flow of gas is such that the flow of gas constitutes a tactilely- sensible representation of the image.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. In case of conflict, the specification, including definitions, will control.
As used herein, the terms "comprising", "including", "having" and grammatical variants thereof are to be taken as specifying the stated features, integers, steps or components but do not preclude the addition of one or more additional features, integers, steps, components or groups thereof. These terms encompass the terms "consisting of and "consisting essentially of. As used herein, the indefinite articles "a" and "an" mean "at least one" or "one or more" unless the context clearly dictates otherwise.
Embodiments of methods and/or devices of the invention may involve performing or completing selected tasks manually, automatically, or a combination thereof. Some embodiments of the invention are implemented with the use of components that comprise hardware, software, firmware or combinations thereof. In some embodiments, some components are general-purpose components such as general-purpose computers or processors. In some embodiments, some components are dedicated or custom components such as circuits, integrated circuits or software.
For example, in some embodiments, some of an embodiment is implemented as a plurality of software instructions executed by a data processor, for example which is part of a general-purpose or custom computer. In some embodiments, the data processor or computer comprises volatile memory for storing instructions and/or data and/or a non-volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data. In some embodiments, implementation includes a network connection. In some embodiments, implementation includes a user interface, generally comprising one or more of input devices (e.g., allowing input of commands and/or parameters) and output devices (e.g., allowing reporting parameters of operation and results. BRIEF DESCRIPTION OF THE FIGURES
Some embodiments of the invention are described herein with reference to the accompanying figures. The description, together with the figures, makes apparent to a person having ordinary skill in the art how some embodiments of the invention may be practiced. The figures are for the purpose of illustrative discussion and no attempt is made to show structural details of an embodiment in more detail than is necessary for a fundamental understanding of the invention. For the sake of clarity, some objects depicted in the figures are not to scale.
In the Figures:
FIGS. 1A and IB schematically depict an embodiment of a device as described herein worn by a user;
FIG. 2 is a flow chart describing an embodiment of a method as described herein; FIGS. 3A-3D schematically depict distance images (3A and 3C) and corresponding modulated pixelated gas-flow representing the distance images (3B and 3D, respectively); FIGS. 4A-4C schematically depict an embodiment of a device as described herein; and
FIGS. 5 A and 5B schematically depict vector image-representations (5 A and 5D) of distance images (3A and 3C, respectively).
DESCRIPTION OF SOME EMBODIMENTS OF THE INVENTION
The invention, in at least some embodiments, relates to methods and devices for imparting information tactilely, as a modulated flow of gas to a skin surface of a subject.
The principles, uses and implementations of the teachings of the invention may be better understood with reference to the accompanying description and figures. Upon perusal of the description and figures present herein, one skilled in the art is able to implement the teachings of the invention without undue effort or experimentation.
Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and/or methods set forth herein. The invention is capable of other embodiments or of being practiced or carried out in various ways. The phraseology and terminology employed herein are for descriptive purpose and should not be regarded as limiting. Methods for imparting information to a subject
According to an aspect of some embodiments of the invention, there is provided a method for tactilely imparting information to a subject, comprising:
a) modulating a flow of gas in accordance with information to be imparted; and b) directing the modulated flow of gas towards a skin surface of the subject, thereby allowing the subject to tactilely sense the modulation of the flow of gas and thus to perceive the imparted information. Generally, the subject is a human subject, in some embodiments a visually- impaired human subject.
In some embodiments, the modulating of the flow of gas is such that the modulated flow of gas constitutes a representation of the information.
Types of Information imparted
Depending on the embodiment, any suitable information is imparted using the method described herein. Depending on the embodiment, any suitable information is imparted using the method described herein, for example sensory information, such as visual information, i.e., information usually perceived by a human using vision. In some embodiments, the visual information is image information and the modulating of the flow of gas is such that the flow of gas constitutes a tactile ly- sensible representation of the image.
Specifically, in some embodiments the flow of gas is modulated so as to constitute a representation of the information to be imparted that is tactilely perceivable by the subject, allowing the subject to "feel" the information on the skin, for example, when the information is image information, the subject 'feels' the image on the skin. For example, in some such embodiments, the gas flow is modulated in order to provide information regarding the size and shape of an object, and the direction and distance of the object from the subject, as in an image.
In some embodiments, the information is image information of a monochrome image In some embodiments, the information is image information of a color image. In some embodiments, the information is image information of a distance image.
Source of Information imparted
Depending on the embodiment, information from any suitable source is imparted using the method described herein. In some embodiments, the information is real information, e.g., real acquired images of the real world. In some embodiments, the information is virtual information, e.g., generated images of a virtual world or reality. In some embodiments, the information is imparted in real-time, that is to say, imparted substantially immediately with acquisition (real information) or generation (virtual information). In some embodiments, the information is stored in a storage medium and is recovered from the storage medium and imparted when desired, at a time substantially different from when acquired (real information) or generated (virtual information).
Real-time imparting of real information
In some embodiments, the information is real information and the method further comprises acquiring the real information, and (substantially immediately) modulating the flow of gas to impart the real information in real-time. In some such embodiments, the real information is acquired and imparted in real-time, that is to say, the real information is acquired and substantially immediately imparted to a user in accordance with the teachings herein. For example, in some such embodiments the information is real image information of an image that is acquired and the information is imparted to the subject substantially immediately by modulating the flow of gas in accordance with the acquired image such that the flow of gas constitutes a tactile ly- sensible representation of the acquired image. Such embodiments are useful, for example in allowing a visually-impaired person to "see" the real world and to navigate therein.
For example, in some embodiments an image is a distance image acquired using a distance camera (also known as a range camera) using any suitable technology, e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect, Microsoft Corporation, Redmond, Washington, USA), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture techniques. For example, in some embodiments, the image is a monochrome image or a color image acquired using a digital camera or image sensor as known in the art of digital photography and cellular telephony. Real-time imparting of virtual information
In some embodiments, the information is virtual information (e.g., image information of a computer-generated image in a virtual world or reality) and the method further comprises generating the information, and (substantially immediately) modulating the flow of gas to impart the virtual information in real-time. In some such embodiments, the information is generated and imparted in real-time, that is to say, the information is generated and substantially immediately imparted to a user in accordance with the teachings herein.
For example, in some such embodiments the information is image information of an image that is generated (in the usual way, for example as known in the field of animation or in the field of video gaming) and the information is imparted to the subject substantially immediately by modulating the flow of gas in accordance with the generated image such that the flow of gas constitutes a tactile ly- sensible representation of the generated image. Such embodiments are useful, for example in allowing a visually-impaired person to "see" a virtual world, for example, allowing playing of an interactive video game or to consume animated video entertainment such as cartoons, stop-action films (e.g., Wallace and Gromit, Aardman Animations Ltd., Bristol, UK) or computer-generated films.
Imparting of stored information
In some embodiments, the information is stored, and the method further comprises retrieving the stored information and then modulating the flow of gas to impart the retrieved information, e.g., pre-generated virtual information such as image information of an animated movie generated and stored on electronic storage media or recorded real information such as real image information acquired using a camera as described above, recorded and stored on electronic storage media. In some such embodiments, when the subject desires to "view" the information, the stored information is retrieved and subsequently imparted to a user in accordance with the teachings herein. Such embodiments are useful, for example, in allowing a visually- impaired person to view a movie, an animated movie or to train in using the method. Modulation
Depending on the embodiment, the modulation of the flow of gas may be any suitable modulation, that is to say, characteristics of the gas flow change as a function of time in a sensible way to impart information to the subject. It is important to note that it is known that a skin surface, especially root hair plexuses becomes desensitized to a constant stimulation such as a constant flow of air relatively relatively quickly but is very sensitive to a changing stimulation such as a fly or ant walking along the skin surface. Accordingly, modulation of a flow of gas for implementing the teachings herein changes to render the flow of gas tactilely sensible.
In some embodiments, modulation comprises changing an intensity (mass of gas per unit area per unit time) of at least a portion of the gas-flow. Typical usable intensities are similar to intensities of air exhaled by a person through the mouth. For example, in some embodiments a nearby object is represented by a more intense (higher rate of flow) gas-flow than a farther object to impart image information related to a distance image; or a darker- colored object is represented by a more intense gas-flow than a lighter- colored object to impart image information related to a monochrome or color image; or higher wavelength- colored (redder) object is represented by a more intense gas-flow than a lower-wavelength colored (e.g., more blue/indigo/violet) object to impart image information related to a color image.
In some embodiments, modulation comprises changing a frequency of variation of intensity of at least a portion of the gas-flow. Typical usable frequencies are between about 1 Hz and about 60 Hz. For example, in some embodiments a nearby object is represented by a more rapidly changing (higher frequency pulsed) gas-flow than a farther object to impart image information related to a distance image; or a darker-colored object is represented by a more rapidly changing gas-flow than a lighter- colored object to impart image information related to a monochrome image; or higher wavelength-colored (redder) object is represented by a more rapidly changing gas- flow than a lower- wavelength colored (more blue/indigo/violet) object to impart image information related to a color image.
In some embodiments, modulation comprises changing a direction at which at least a portion of the gas flow reaches the skin surface. In some embodiments, changing a direction comprises changing an angle of incidence of the gas flow to the skin surface, typically between perpendicular (0°) to the skin surface and about 60° to the skin surface. In some embodiments, changing a direction comprises changing an orientation, in analogy to compass directions. In some such embodiments, such gas flow is "pulsed", that is to say, there is a change of intensity as a function of time. In some such embodiments, the intensity of such gas flow is substantially constant during the change of direction. For example, information about an object in an image (distance, monochrome or color) is "traced" on the skin surface of the subject by changing the direction of at least a portion of the gas flow.
In some embodiments, modulation comprises changing a location of the skin surface at which at least a portion of the gas flow reaches the skin surface. In some such embodiments, such gas flow is "pulsed", that is to say, there is a change of intensity as a function of time. In some such embodiments, the intensity of such gas flow is substantially constant during the change of location. For example, an outline of an object in an image (distance, monochrome or color) is "traced" on the skin surface of the subject with at least a portion of the gas flow, optionally with a rate of change of intensity imparting distance information (e.g., from a distance image), intensity information (e.g., from a monochrome image) or color information (e.g., from a color image).
In some embodiments, the flow of gas is pixelated, for example an object in an image (a distance, monochrome or color image) is "displayed" on the skin surface of the subject with a pixelated gas flow, optionally with a rate of change of intensity of each pixel imparting distance information (e.g., from a distance image), intensity information (e.g., from a monochrome image) or color information (e.g., from a color image).
Suitable skin surface
Depending on the embodiment, the modulated flow of gas is directed to any suitable skin surface of a subject, preferably based on considerations of sensitivity to sensing modulation of a modulated air flow, interference with other tasks, aesthetic considerations and personal preference. In some embodiments, a preferred suitable skin surface is a skin surface with higher such sensitivity due to, for example, greater concentrations of touch-sensitive nerve-endings or root hair plexuses (light-touch mechanoreceptors that detect bending of hairs, enabling movement of the hairs to be detected even if the skin is not touched directly). That said, it is also preferred that the suitable skin surface be selected so that implementation of the teachings herein does not substantially interfere with performing other tasks. Accordingly, in some embodiments, a suitable skin surface is selected from the group consisting of a surface of an inner wrist, an arm, a nape of a neck, an earlobe, an ear canal, behind an ear, and especially a forearm.
In some embodiments, other suitable skin surfaces with high sensitivity include skin surfaces selected from the group consisting of a surface of a hand, a lip, a tongue, a face and inside of a nostril.
In some embodiments, also suitable although having lesser sensitivity include skin surfaces selected from the group consisting of surfaces of a leg and of a torso.
Devices for imparting information to a subject
Embodiments of the methods described herein are implemented using any suitable device. In some embodiments, it is preferred to use a device as described herein for implementing such methods.
According to an aspect of some embodiments of the invention, there is provided a device for tactilely imparting information to a subject (especially a human subject, especially a visually- imp aired human subject) comprising:
a) at least one nozzle configured to direct a flow of gas towards a skin surface of a subject;
b) functionally associated with the at least one nozzle, an actuator for modulating the flow of gas; and
c) a controller for controlling the actuator to modulate the flow of gas in accordance with information to be imparted such that a modulated flow of gas constitutes a tactilely- sensible representation of the information to be imparted.
Nozzles
In some embodiments, a device as disclosed herein comprises at least one nozzle configured to direct a flow of gas towards a skin surface of a subject.
In some embodiments, the at least one nozzle is one nozzle. In some embodiments, the at least one nozzle is at least two nozzles.
In some embodiments, the at least one nozzle comprises an array of nozzles, e.g., an x by y array of nozzles, where x and y are independently integers of at least 4. In some such embodiments, a nozzle array is useful in imparting image information as a pixelated flow of gas. Specifically, the controller activates an actuator so that each nozzle of the nozzle array corresponds to a pixel of an imparted image, and each pixel is "displayed" by directing a flow of gas at a skin surface from the corresponding nozzle. In such a way, an x by y array of pixels is "displayed" by an x by y array of nozzles. Conversion of image information to the pixel resolution of a nozzle array is easily performed using any suitable pixelization method or algorithm.
Actuator
In some embodiments, a device as disclosed herein comprises an actuator functionally associated with the at least one nozzle, the actuator for modulating the flow of gas from the at least one nozzles. A typical actuator comprises one or more components for directing and changing an air flow from or through a nozzle such as valves, motors, step motors, bearings, moveable nozzle mounts, vanes, adjustable apertures, flaps and the like, for example, as known in the art of radio-controlled model aircraft.
In some embodiments, the at least one nozzle is at least two nozzles, and the actuator is configured to modulate a flow of gas from at least two of the nozzles independently, e.g., the at least one nozzle is a nozzle array and the actuator is configured to modulate the flow of gas from each of the nozzles of the nozzle array independently.
In some embodiments, the actuator is configured to change an intensity of the gas- flow from at least one nozzle.
In some embodiments, the actuator is configured to change a frequency of variation of intensity of the gas-flow from at least one nozzle.
In some embodiments, the actuator is configured to change a direction of the gas-flow from at least one nozzle (in some embodiments the angle and/or in some embodiments the orientation in analogy to a compass), thereby changing a direction at which at least a portion of the gas-flow (the portion from that nozzle) is directed at a skin surface.
In some embodiments, the actuator is configured to change a location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed. In some embodiments, the actuator is configured to translate at least one nozzle, thereby changing the location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed. In some embodiments, the actuator is configured to stop gas flow from at least one nozzle and to allow gas flow from at least one other nozzle, thereby changing the location of a skin surface at which at least a portion of the gas-flow is directed. In some embodiments, the actuator is configured to change a direction of the gas-flow from a nozzle (in some embodiments the angle and/or in some embodiments the orientation in analogy to a compass), thereby changing a location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed.
In some embodiments, the actuator is configured to generate a pixelated flow of gas. In some such embodiments, as described above the at least one nozzle comprises a matrix of nozzles, each nozzle corresponding to a pixel and the actuator configured to control the flow of gas from each nozzle independently. In some such embodiments, the at least one nozzle comprises a single nozzle and the actuator is configured to serially aim the nozzle (e.g., by rotating and/or translating the nozzle) to direct a flow of gas at different portions of a skin surface, each such portion constituting a pixel.
Controller
In some embodiments, a device as disclosed herein comprises a controller for controlling an actuator to modulate the flow of gas in accordance with information to be imparted. Generally, a controller is a component that, based on the information to be imparted, generates commands to the actuator that yield the appropriate modulation of the flow of gas such that the flow of gas constitutes a tactilely- sensible representation of the information. A controller typically comprises a suitable general-purpose or custom-made device, for example an appropriately- configured general purpose digital computer (such as known in the field of "smartphones", e.g., Iphone 4 by Apple Incorporated, Cupertino, California, USA) with a user-input channel, an actuator-control output, a memory and a processor configured to convert information to be imparted to actuator-control output.
Typically, a user-input channel, for example implemented as a touch screen with a graphic-user interface (such as well-known in the field of "smartphones") is used to control various parameters and user preferences for operating the device.
In some embodiments, a controller is also configured to accept the information to be imparted. Thus, in some embodiments a controller of a device further includes an information- input channel to accept information to be imparted to a user. Such an information input channel is typically implemented as an electronic input (wired or wireless) to accept information to be imparted to a user. The information is any information to be imparted as described hereinabove, including sensory information, visual information, and image information, in any suitable format, typical digital format. In some embodiments, the information is image information (e.g., of a distance image, of a monochrome image, of a color image) in the form of a digital image data file as known in the art.
As noted above, a controller generally includes a processor configured to convert information to be imparted to actuator-control output, that is to say, commands for the actuator to modulate the flow of gas to represent the information to be imparted. In some embodiments, the processor functions as an image-representation generator, to convert image information to actuator-control output for the actuator to modulate the flow of gas to represent the image information.
In some embodiments, the controller is configured to convert image information to actuator-control output for the actuator to modulate a flow of gas to constitute a tactilely- sensible pixelated representation of image information. In some such embodiments, the processor functions as an image-representation generator to convert image information to actuator-control output for the actuator to produce a pixelated flow of gas to impart image information, including by directing a flow of gas (with optional changes in intensity and/or direction) at a plurality of distinct locations on the skin surface to "print" a pixelated representation of the image information to be imparted. Any suitable known method of image pixelation may be used in implementing such embodiments. In some such embodiments, a plurality of pixels are "displayed" by a single nozzle, for example a single nozzle translatable in an x and y direction, or a linear array of nozzles moveable together in a single direction, or a rectangular array of fixed nozzles. In some such embodiments, the size, shape and distance of an object in an image is represented by modulation of a pixelated gas-flow. For example, the image is represented such that portions (one or more pixels) of the gas flow representing closer objects change at higher frequency than portions (one or more pixels) of the gas flow representing farther objects. In some such typical embodiments, an image is represented by a gas-flow made up of a 20 x 20 pixel array in a 5 cm x 5 cm array.
In some embodiments, the controller is configured to convert image information to actuator-control output for the actuator to modulate a flow of gas to constitute a tactilely- sensible vector representation of image information. In some such embodiments, the processor functions as an image-representation generator to convert image information to actuator-control output for the actuator to produce a vector image representation to impart image information, including by translating a nozzle with a continuous flow of gas (with optional changes in intensity and/or direction) to "draw" a vector representation of the image on the skin surface. Any suitable known method of vector image representation may be used in implementing such embodiments.
Supporting Component
In some embodiments, a device as disclosed herein further comprises a supporting component (in some embodiments also called a supporting backing component) to which the at least one nozzle is attached, the supporting component configured to be worn on a portion of a body of a subject so that the at least one nozzle is positioned to direct the flow of gas towards a skin surface of the subject. In some embodiments, components in addition to the at least one nozzle such as the actuator and the controller are also attached to the supporting component. In some embodiments, the supporting component comprises a frame or container, e.g., of aluminum or plastic. In some embodiments, the supporting component includes a cover or shield to isolate a skin surface from external stimulus.
Generally, the supporting component is configured to be worn appropriately so that the at least one nozzle is positioned to direct the flow of gas towards a selected skin surface, as discussed above. In some embodiments the skin surface is selected from the group consisting of a surface of an inner wrist, an arm, a nape of a neck, an earlobe, an ear canal, behind an ear, and especially of a forearm, of a hands, a tongue, a face and the inside of a nostril and in some embodiments, of a leg and of a torso.
Depending on the embodiment, the supporting component is configured to be worn on any suitable portion of the body of a subject as described above. To this end, the supporting component typically includes (reversible) securing components to secure the supporting component in appropriate proximity and orientation to the selected skin surface, such as one or more of straps, snaps, ties, clips, hook-and-loop fasteners, and the like.
In some embodiments, at least one nozzle attached to the supporting component faces inwards, so when the supporting component is properly worn by a subject, the at least one nozzle is directed at a skin surface of the subject.
In some embodiments, at least one nozzle is immovably attached to the supporting component, so that flow of gas from the nozzle is directed to a specific location on a skin surface when the supporting component is properly worn.
In some embodiments, at least one nozzle is rotatably attached to the supporting component, so that the direction of flow of gas from that nozzle can be changed by rotation of the nozzle by the actuator. In some embodiments, at least one nozzle is translatably attached to the supporting component so that the location on the skin surface at which a flow of gas from that nozzle is directed is changeable by translation of the nozzle by the actuator. Power Source
In some embodiments, a device further comprises a power source (typically a rechargeable battery or similar) for supplying the power required for operation of components of the device such as the controller and actuator. In some embodiments, such a power source is secured to the supporting component.
Gas and Gas Flow generating component
The flow of gas through the at least one nozzle is of any suitable gas. Typically, such a gas is inert and/or harmless and/or non-flammable, for example, air, CO2, helium, neon, or N2.
The flow of gas through the at least one nozzle is provided by any suitable component or device. In some embodiments, the device further comprises a gas-flow generating component to provide a flow of gas through the at least one nozzle. In some embodiments, the gas-flow generating component comprises a compressed gas container, e.g., of CO2, air, helium, neon or N2. In some embodiments, the gas-flow generating component comprises a gas-flow generator, e.g., a portable air pump, that generates a gas flow by drawing in and compressing ambient air.
Information acquirer
The information to be imparted is provided to the controller in any suitable fashion. In some embodiments, information is stored on the controller. In some embodiments, the controller comprises an information input channel and information to be imparted is provided from any suitable source, e.g., a computer, a gaming console, a mobile telephone or an image storage device.
In some embodiments, the device is configured to impart acquired information in real- time. Generally, in such embodiments, an information acquirer is functionally associated with the controller through an information input channel.
In some embodiments, a device further comprises, in communication with the controller, an information acquirer to acquire information to be imparted and to provide the acquired information to the controller, and the controller is configured to control the actuator to modulate the flow of gas in accordance with the acquired information in real time. In some embodiments, the information acquirer is in wired communication with the controller. In some embodiments, the information acquirer is in wireless communication with the controller, (e.g., using Bluetooth® technology).
Any suitable information acquirer can be used. In some embodiments an information acquirer is a component of the device.
In some embodiments, an information acquirer is an image acquirer, e.g., a camera.
In some embodiments, an image acquirer is a distance camera (also known as a range camera) that generates an image of distance as a function of x and y, using any suitable technology, e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect from Microsoft, Redmond, Washington), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture.
In some embodiments, an image acquirer is a monochrome or color camera, e.g., a visible light camera as known in the art of cellular telephony. In some embodiments, an image acquirer is an infrared camera or a thermal imaging camera.
In some embodiments, the information acquirer is configured to be (reversibly) worn by the subject. In some embodiments, the information acquirer is configured to be worn on the head of the subject, e.g., attached to a head band, a cap or the like.
In some embodiments, some device components are in wired communication with other device components.
In some embodiments, some device components are in wireless communication with other device components, for example using Bluetooth® technology.
For example, in some embodiments an image acquirer and a controller make up a single physical unit configured to be worn on the head of a subject while the actuator, at least one nozzle and gas-flow supply make up a single physical unit configured to be worn on the arm of a subject, where the controller and the actuator are in wireless communication. Such an embodiment allows wireless transmission of relatively modest volumes of data.
For example, in some embodiments an image acquirer makes up a single physical unit configured to be worn on the head of a subject while the controller, the actuator, at least one nozzle and gas-flow supply make up a single physical unit configured to be worn on the arm of a subject, where the image acquirer and the controller are in wireless communication. Such an embodiment allows a reduced size of components that are (uncomfortably) worn on the head. In Figures 1A and IB, an embodiment of a device in accordance with the teachings herein including two separate physical units, an information imparting (actuator) unit 10 configured to be worn on the forearm of a human subject and an image-data acquiring (generation) unit 30 configured to be worn on the head of a human subject, is schematically depicted.
Information-imparting unit 10, depicted in detail in Figure IB, comprises a plurality of nozzles 12, attached to a flexible supporting (backing surface) component 14 of silicone rubber, within a fixed-area rigid grid of polycarbonate to constitute a 5 by 5 nozzle array having a total of 25 nozzles. Supporting component 14 comprises a reversible attachment component 24 (straps with hook-and loop fasteners such as Velcro®) for securing information-imparting unit 10 over a skin surface of a subject, a forearm 16, such that nozzles 12 face the skin surface. Information-imparting unit 10 further comprises a reservoir 18 of compressed N2 gas in fluid communication with nozzles 12 through a manifold pipe (not apparent in the Figures).
An actuator 20 of the device comprises twenty- five individually- operable adjustable- aperture valves, each valve associated with a branch of the manifold pipe that directs nitrogen gas from reservoir 18 to a different nozzle 12. Each valve of actuator 20 is configured to independently prevent gas flow from an associated nozzle 12, or to allow a certain gas flow to exit from an associated nozzle 12. Specifically, actuator 20 is configured to independently change an intensity of a gas-flow from each of nozzles 12 (by opening or closing an associated valve), to independently change frequency of variation of intensity of a gas-flow from each of nozzles 12 (by changing the rate of opening/closing of an associated valve) and is configured to change a location of a skin surface at which at least a portion of the gas-flow is directed (by opening a valve associated with a given nozzle 12 to allow gas flow from that nozzle while closing a valve associated with a different nozzle 12 to stop gas flow from that nozzle).
Image-data acquiring component 30 comprises a distance-image acquirer (image- acquisition component) time-of-flight camera 32, attached to a removable headband 34, for positioning on the head of the user, and a controller 36 (a digital computational device similar to a smartphone) configured to function as an image-representation generator, for accepting image information (data) acquired by camera 32 as input and for controlling release of gas from reservoir 18 as a modulated gas-flow through nozzles 12, by controlling actuator 20 to modulate the flow of gas from nozzles 12 in accordance with information to be imparted. Specifically, controller 36 received an image from camera 32, pixelates the image to the resolution of the nozzle array (5 by 5) where the intensity of gas flow from a pixel is related to a distance to an object in the image (closer is more intense gas flow) and sends a series of commands in real time to control actuator 20 to modulate a gas-flow to constitute a representation of an image represented by the image data. The commands are wirelessly transmitted by controller 36 using Bluetooth® transceiver 38 to Bluetooth transceiver 22 located on actuator component 10, in communication with actuator 20.
In real time, actuator 20 controls the flow of gas from each nozzle 12 (e.g., by opening and closing valves) in such a way that the image acquired by camera 32 is represented by a modulated pixelated gas-flow from the array of nozzles 12 so that the pixelated gas-flow constitutes a tactile ly- sensible representation of the image acquired by camera 32.
Figure 2 presents an overview of an embodiment of a method of imparting information as described herein as a flow chart. Camera 32 acquires an image and generates image data representing the image. A controller 36 functioning as an image-representation generator accepts the image data as input and generates actuator commands that are sent to an actuator 20 through an actuator-control output. An actuator 20 follows the actuator commands, modulating the flow of gas through the array of nozzles 12 to form a modulated pixelated gas-flow directed towards a skin surface so that the pixelated gas-flow constitutes a tactile ly- sensible representation of the image acquired by camera 32.
As noted above, in some embodiments, information such as image data is provided to a controller 36 by some component other than an image-acquisition component, e.g., a component that provides stored or computer-generated image data.
In Figures 3A-3D, the representation of images with a pixelated flow of gas is schematically depicted.
Figure 3A is a schematic depiction of a distance image acquired by a distance camera such as 32. Figure 3B is a schematic depiction of a corresponding modulated pixelated gas flow generated by a nozzle array such as depicted in Figure IB. In Figure 3B, air flowing in pixels corresponding to a close object (person 42 at 1 meter) in the distance image Figure 3A are modulated by pulsing (changing the intensity of flow) at an 80 Hz frequency, air flowing in pixels corresponding to a further object (pole 44 at 4 meter) in the distance image is pulsed at 20 Hz frequency, and air flowing in pixels corresponding to objects at infinite (greater than 8 meter) in the distance image is stopped completely.
Figure 3C is a schematic depiction of a distance image similar to that of Figure 3 A, where the camera is further away from the objects in the image. Figure 3D is a schematic depiction of a corresponding modulated pixelated gas flow generated by a nozzle array such as depicted in Figure IB. In Figure 3D, air flowing in pixels corresponding to a close object (person 42 at 4 meter) in the distance image are modulated by pulsing at an 20 Hz frequency, air flowing in pixels corresponding to a further object (pole 44 at 7 meters) in the distance image is pulsed at 10 Hz frequency, and air flowing in pixels corresponding to objects at infinite (greater than 8 meter) in the distance image is stopped completely.
An additional embodiment of a device for imparting information as described herein, device 50, is schematically depicted in Figures 4A-4C. Device 50 is configured to acquire distance images and impart information from the acquired images to a subject using device 50 by modulating a flow of gas in accordance with the information to be imparted such that the modulated flow of gas constitutes a tactilely- sensible representation of information to be imparted.
Device 50 comprises two physically separate units: information- imparting unit 52 depicted in detail in Figures 4A and 4B configured to be worn on the forearm of a human subject and an image-data acquiring unit 54 depicted in detail in Figure 4C, configured to be worn on the head of a human subject.
Information-imparting unit 52 comprises a rigid supporting component 56, substantially a closed rigid box of polyethylene that acts to shield a selected skin surface from ambient stimulus and contains or supports other device components and includes straps 24 to allow information-imparting unit 52 to be reversibly worn on the forearm of a human subject.
Inside supporting component 56 are found:
a single nozzle 12 (hidden from view) configured to direct a flow of gas towards a skin surface of a subject properly wearing information-imparting unit 52;
an actuator 20 for modulating the flow of gas through nozzle 12;
a controller 36 for controlling actuator 20 to modulate the flow of gas in accordance with image information to be imparted to a subject such that the modulated flow of gas constitutes a tactilely- sensible representation of the information;
a portable air pump 58 as a gas-flow generator; and a Li-ion battery 60 as a power source for the other components.
Actuator 20 comprises a rigid square 4cm by 4 cm frame 62 immovably- contained inside supporting component 56. Two electrical step motors 64a and 64b (as known in the art of RC model airplanes) are immovably secured to two adjacent edges 62a and 62b of frame 62. To a rotor 66a and 66b of each respective motor 64 is attached a proximal end of a respective slotted bar 68a and 68b. Closer to the distal ends of the bars, slotted bars 68a and 68b overlap. In the hole defined by the overlapping slots of slotted bars 68a and 68b is slidingly secured a support pin 70 to which nozzle 12 (hidden from view) is secured.
As is clear to one skilled in the art, coordinated rotation of slotted bars 68a and 68b around rotors 66 with the use of step motors 64 allows support pin 70 and consequently nozzle 12 to be translated to any location inside the two-dimensional plane defined by rigid frame 62 using triangulation.
Nozzle 12 is a standard gas nozzle secured to support pin 70 so as to be directed perpendicularly to a skin surface when information-imparting unit 52 is properly worn by a human subject in fluid communication with portable air pump 58 through pipe 72. Along pipe 72 and controlling air flow therethrough is an additional component of actuator 20, valve 74. Valve 74 comprises two controllable modules, needle valve 76 with an associated electrical motor that allows precise control of the rate of gas flow from air pump 58 to nozzle 12 and rotatable perforated disk 78 with an associated electrical motor that allows control and variation of the frequency of the change of air flow intensity from nozzle 12. Specifically, when a perforation of disk 78 is located across the lumen of pipe 72, air flows to nozzle 12 but when a solid portion of disk 78 is located across the lumen of pipe 72, air is blocked from passing to nozzle 12.
Accordingly, actuator 20 is configured to change an intensity of a gas-flow from nozzle 12 (using valve 74) to change a frequency of variation of intensity of a gas-flow from nozzle (by changing a rate of rotation of disk 78 of valve 74) and to change a location of a skin surface at which gas-flow from nozzle 12 is directed (using motors 64a and 64b to translate nozzle 12 inside frame 62).
Controller 36 (a portable digital computer, similar to that used in an Iphone 4 by Apple Incorporated, Cupertino, California, USA) having a touch-screen (not depicted) and associated hardware and software components constituting a user-input channel for accepting user commands, an information-input channel 80 in wired communication with a Bluetooth® transceiver 38 for accepting image information as an image in a standard format (e.g., any format produced by an image acquirer, see below) and a processor 82 for accepting image information from information-input channel 80, processor 82 configured to convert the accepted image information to commands to actuator 20 to modulate a flow of gas to represent the received image. As detailed below, processor 82 and thus controller 36 is configured to control actuator 20 to modulate the flow of gas through nozzle 12 to constitute a tactile ly- sensible vector representation of image information. Image-data acquiring unit 54 , Figure 4C, includes a distance camera 84 (similar to a Kinect by Microsoft Corporation, Redmond, Washington, USA) as a distance-image acquirer attached to a head band 34, a Bluetooth® transceiver 38 to provide wireless communication between distance camera 84 and controller 36, specifically, to information input channel 80 of controller 36. Not depicted is a battery supplying power for the components of image-data acquiring unit 54.
For use, image acquiring unit 54 is worn on the head of a subject with the help of headband 34 and information-imparting unit 52 is worn on the forearm of a subject with the help of straps 24. The various components of device 50 are activated.
Distance camera 84 acquires a distance image in the usual way and transmits the acquired image using transceiver 38 to controller 36 through information-input channel 80 via transceiver 22.
In real time, processor 82 of controller 36 applies a vector image representation algorithm to convert the acquired image received from distance camera 84 to a vector image, where objects in the image are represented by a set of vectors in a plane, each such vector having an intensity value related to the distance to the represented object.
In real time, controller 36 then controls actuator 20 to modulate the flow of gas directed by nozzle 12 towards the forearm skin surface of the subject to constitute a tactilely- sensible representation of information to be imparted. Specifically, controller 36 translates each vector in the vector image to a linear motion of nozzle 12 having a direction and length matching the vector, and an intensity of air flow related to the distance to an object represented by the vector, where greater intensity corresponds to a closer object. In such a way, controller 36 uses actuator 20 and nozzle 12 to "draw" a vector representation of the acquired image on the skin surface.
Figure 5A is a schematic depiction of a modulated gas flow directed at a skin surface by device 50 that is substantially a vector image representation of the distance image depicted in Figure 3 A. The vertical line 44 at the left having a continuous low intensity air flow of 20 (arbitrary units) corresponds to a far object (pole 44 at 4 meter) while the three lines 42a, 42b and 42c having a continuous high intensity air flow of 80 (arbitrary units) correspond to a near object (person 42 at 1 meter).
Figure 5B is a schematic depiction of a modulated gas flow directed at a skin surface by device 50 that is substantially a vector image representation of the distance image depicted in Figure 3C. The vertical arrow 44 at the left having a continuous low intensity air flow of 10 (arbitrary units) corresponds to a far object (pole 44 at 7 meter) while line 42 having a continuous high intensity air flow of 20 (arbitrary units) corresponds to a near object (person 42 at 4 meter).
In principle, actuator 20 of device 50 is also configured to generate a pixelated flow of gas, requiring only appropriate configuration of controller 36 to control actuator 20 to modulate the flow of gas through nozzle 12 to constitute a tactile ly- sensible pixelated representation of image information. Specifically, processor 82 of controller 36 is configured to produce a pixelated representation of an image to be imparted to a subject and subsequently processor 82 controls actuator 20 to serially translate nozzle 12 to the locations inside frame 62 that correspond to pixels, and at each pixel direct a flow of gas from nozzle 12, the flow of gas having the appropriate intensity or frequency.
In some of the specific embodiments discussed above, imparting of image information is discussed in detail. As noted above, in some embodiments, other types of information are imparted to a user.
For example, in some embodiments, generally useful information is imparted to a user. For example, in some embodiments text information, is imparted to the user, for example as a series of symbols, letters or numerals "drawn" or "printed" on a skin surface with a modulated flow of gas as described herein. For example, in some embodiments the approach of a time limit (e.g., during playing a game, or when a specific action needs to be taken) is imparted, for example by a flow of gas having a periodic intensity, which intensity and/or frequency increase with the approach of the time limit.
For example sensory information (e.g., information usually imparted by one of the five senses) is imparted, such as visual information, i.e., information usually perceived by a human using vision. For example, a modulated flow of gas is used to impart information about the presence of an obstacle in front of a visually- imp aired person, not as an tactilely- sensible "image" of modulated gas, but as a symbolic "warning", for example a specifically modulated gas flow symbol. Such information can be, for example, of a specific color (a red, amber or green traffic light), of a specific shape (e.g., a stop sign) or movement (e.g., a vehicle approaching from one of the sides of the subject).
In some of the specific embodiments discussed above, imparting of information is imparted at one localized skin surface, specifically an area of skin on one forearm. In some embodiments, information is imparted concurrently (in some cases, simultaneously) at two different skin surfaces, e.g., two forearms, a forearm and the nape of the neck. In some embodiments, the information imparted at each of the two different surfaces is different, for example, one arm for image information and the other arm (or nape of the neck) for symbolic or complementary information. In some embodiments, the information imparted at the two different surfaces, taken together, represents, and is interpreted by the user as, a stereo image representation.
In some embodiments, the teachings herein impart information about visual stimuli in the surroundings to a human subject, in a form which can be understood by such a subject. The subject is provided with information regarding the size, shape and distance of an object within a detection range, without requiring actual contact with object. In some embodiments, the hands of the subject are left free, such that, for example, a visually- impaired subject may perform manual tasks while implementing the teachings herein, or optionally, use a white cane in addition to the teachings herein. Furthermore, the teachings herein do not rely on audio signals, such that the hearing of the subject is still available to obtain additional information about the surroundings, increasing the total amount of information potentially imparted and received by the subject.
Embodiments of the teachings herein allow the visually- imp aired to participate in heretofore unavailable entertainment such as video gaming, watching movies and the like.
Embodiments of the teachings herein may also be used, for example, by normally- sighted rescue workers in smoke-filled buildings, in order to help locate trapped individuals. In such circumstances, imparting visual information using prior art methods may be impossible due to difficulties in viewing a visual display of any kind in the presence of dense smoke, particularly where the face of the rescuer may be at least partially covered with a gas mask or other breathing device.
Embodiments of the teachings herein may be used for imparting information in an additional, yet-unused, mode. For example, in some embodiments, the teachings herein are implemented as an additional component of a game, for example a multimedia game.
In some embodiments, information is imparted in other fields, for example, in controlling a vehicle (e.g., driving) or other fields where it is desired to impart information.
It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination or as suitable in any other described embodiment of the invention. Certain features described in the context of various embodiments are not to be considered essential features of those embodiments, unless the embodiment is inoperative without those elements.
Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the scope of the appended claims.
Citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the invention.

Claims

CLAIMS:
1. A method for tactilely imparting information to a subject, comprising:
modulating a flow of gas in accordance with information to be imparted; and directing said modulated flow of gas towards a skin surface of the subject thereby allowing the subject to tactilely sense the modulation of the flow of gas and thus to perceive the imparted information.
2. The method of claim 1, wherein said modulating said flow of gas is such that said modulated flow of gas constitutes a representation of said information.
3. The method of any of claims 1 to 2, wherein said information is sensory information.
4. The method of any of claims 1 to 2, wherein said information is visual information.
5. The method of claim 3, wherein said information is image information, and said modulating of said flow of gas is such that said flow of gas constitutes a tactilely- sensible representation of said image.
6. The method of claim 5, wherein said image information is of a distance image
7. The method of claim 5, wherein said image information is of a monochrome image.
8. The method of claim 5, wherein said image information is of a color image.
9. The method of any of claims 1 to 8, wherein said information is real information.
10. The method of claim 9, further comprising:
acquiring said real information; and
modulating said flow of gas to impart said acquired information in real-time.
11. The method of any of claims 1 to 8, wherein said information is virtual information
12. The method of claim 11, further comprising:
generating said virtual information; and modulating said flow of gas to impart said generated virtual information in real-time.
13. The method of any of claims 9 or 11, wherein said information is stored, and further comprising:
retrieving said stored information; and
modulating said flow of gas to impart said retrieved information.
14. The method of any of claims 1 to 13, wherein said modulation comprises changing an intensity of at least a portion of the gas-flow.
15. The method of any of claims 1 to 14, wherein said modulation comprises changing a frequency of variation of intensity of at least a portion of the gas-flow.
16. The method of any of claims 1 to 15, wherein said modulation comprises changing a direction at which at least a portion of the gas flow reaches the skin surface.
17. The method of any of claims 1 to 16, wherein said modulation comprises a location of the skin surface which at least a portion of the gas flow reaches the skin surface.
18. The method of any of claims 16 to 17, where an intensity of said flow of gas is substantially constant during said changing of location or said change of direction.
19. The method of any of claims 1 to 18, wherein said flow of gas is pixelated.
20. A device for tactile ly imparting information to a subject, comprising:
a) at least one nozzle configured to direct a flow of gas towards a skin surface of a subject;
b) functionally associated with said at least one nozzle, an actuator for modulating said flow of gas; and
c) a controller for controlling said actuator to modulate said flow of gas in accordance with information to be imparted such that said modulated flow of gas constitutes a tactile ly- sensible representation of information to be imparted.
21. The device of claim 20, wherein said at least one nozzle is one nozzle.
22. The device of claim 20, wherein said at least one nozzle is at least two nozzles.
23. The device of claim 22, wherein said at least one nozzle comprises an array of nozzles.
24. The device of any of claims 22 to 23, wherein said at least one nozzle is at least two nozzles, and said actuator is configured to modulate a flow of gas from at least two said nozzles independently.
25. The device of any of claims 20 to 24, wherein said actuator is configured to change an intensity of a gas-flow from at least one said nozzle.
26. The device of any of claims 20 to 25, wherein said actuator is configured to change a frequency of variation of intensity of a gas-flow from at least one said nozzle.
27. The device of any of claims 20 to 26, wherein said actuator is configured to change a direction of a gas-flow from at least one said nozzle.
28. The device of any of claims 20 to 27, wherein said actuator is configured to change a location of a skin surface at which at least a portion of said gas-flow is directed.
29. The device of claim 28, wherein said actuator is configured to translate at least one said nozzle, thereby changing a location of a skin surface at which at least a portion of said gas-flow is directed.
30. The device of any of claims 28 to 29, wherein said actuator is configured to stop gas flow from at least one said nozzle and to allow gas flow from at least one other said nozzle, thereby changing a location of a skin surface at which at least a portion of said gas-flow is directed.
31. The device of any of claims 28 to 30, wherein said actuator is configured to change a direction of said gas-flow from at least one said nozzle, thereby changing a location of a skin surface at which at least a portion of said gas-flow is directed.
32. The device of any of claims 20 to 31, wherein said actuator is configured to generate a pixelated flow of gas.
33. The device of any of claims 20 to 32, said controller including an information- input channel to accept information to be imparted to a user.
34. The device of any of claims 20 to 33, wherein said information is sensory information.
35. The device of any of claims 20 to 34, wherein said information is visual information.
36. The device of any of claims 20 to 35, wherein said information is image information.
37. The device of any of claims 20 to 36, wherein said controller is configured to control said actuator to modulate a flow of gas to constitute a tactile ly- sensible pixelated representation of image information.
38. The device of any of claims 20 to 36, wherein said controller is configured to control said actuator to modulate a flow of gas to constitute a tactile ly- sensible vector representation of image information.
39. The device of any of claims 20 to 38, further comprising a supporting component to which said at least one said nozzle is attached, said supporting component configured to be worn on a portion of a human subject so that said at least one nozzle is positioned to direct said flow of gas towards a skin surface of a subject.
40. The device of claim 39, wherein at least one said nozzle is attached to said supporting component facing inwards, so that when said supporting component is properly worn by a subject, said at least one nozzle is directed at a skin surface of said subject.
41. The device of any of claims 39 to 40, wherein at least one said nozzle is immovably attached to said supporting component so that said flow of gas from said nozzle is directed to a specific location on a skin surface of said subject.
42. The device of any of claims 39 to 41 wherein at least one said nozzle is rotatably attached to said supporting component so that the direction of flow of gas from said nozzle is changeable by rotation of said nozzle by said actuator
43. The device of any of claims 39 to 42, wherein at least one said nozzle is translatably attached to said supporting component so that the location of a skin surface at which a flow of gas from said nozzle is directed is changeable by translation of said nozzle by said actuator.
44. The device of any of claims 20 to 43, further comprising a gas-flow generating component.
45. The device of any of claims 20 to 44, further comprising:
in communication with said controller, an information acquirer to acquire information to be imparted and provide said acquired information to said controller, and said controller configured to control said actuator to modulate said flow of gas in accordance with said acquired information in real time.
46. The device of claim 45, wherein said information acquirer is an image acquirer.
47. The device of claim 46, wherein said image acquirer is a camera.
PCT/IB2011/051157 2010-03-21 2011-03-20 Methods and devices for tactilely imparting information WO2011117794A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US31595510P 2010-03-21 2010-03-21
US61/315,955 2010-03-21

Publications (1)

Publication Number Publication Date
WO2011117794A1 true WO2011117794A1 (en) 2011-09-29

Family

ID=44262984

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2011/051157 WO2011117794A1 (en) 2010-03-21 2011-03-20 Methods and devices for tactilely imparting information

Country Status (1)

Country Link
WO (1) WO2011117794A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2487672A (en) * 2011-01-31 2012-08-01 Univ Sheffield Active sensory augmentation device
FR2999741A1 (en) * 2012-12-17 2014-06-20 Centre Nat Rech Scient HAPTIC SYSTEM FOR NON-CONTACT INTERACTING AT LEAST ONE PART OF THE BODY OF A USER WITH A VIRTUAL ENVIRONMENT
WO2014027228A3 (en) * 2012-08-16 2014-09-12 Uab Gaminu Apparatus for converting surroundings-related information into tactile depth map information
EP3413288A1 (en) * 2017-06-09 2018-12-12 Honda Research Institute Europe GmbH Method for assisting a person in acting in a dynamic environment and corresponding system
CN114639205A (en) * 2022-03-30 2022-06-17 中国工商银行股份有限公司 Password input device and method based on air injection induction and self-service equipment
WO2023019212A1 (en) * 2021-08-12 2023-02-16 Triton Systems, Inc Arm-mounted hands-free haptic display

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4337602A1 (en) * 1993-11-01 1995-05-04 Lehmann Hans Joachim Dynamic orientation aid and video simulator for the blind
DE19517649A1 (en) * 1995-05-17 1996-11-21 Thomas Rupp Communication device for information transfer from data processor to deaf and dumb people
US5636038A (en) * 1996-06-24 1997-06-03 Lynt; Ingrid H. Apparatus for converting visual images into tactile representations for use by a person who is visually impaired
US5942970A (en) * 1998-10-08 1999-08-24 Norman; Jim Image optical-to-tactile converter
US20030151597A1 (en) * 2002-01-03 2003-08-14 Roberts John W. Extended refreshable tactile graphic array for scanned tactile display
GB2390733A (en) * 2002-07-08 2004-01-14 Baback Asady Navigation and communication aid for the blind

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4337602A1 (en) * 1993-11-01 1995-05-04 Lehmann Hans Joachim Dynamic orientation aid and video simulator for the blind
DE19517649A1 (en) * 1995-05-17 1996-11-21 Thomas Rupp Communication device for information transfer from data processor to deaf and dumb people
US5636038A (en) * 1996-06-24 1997-06-03 Lynt; Ingrid H. Apparatus for converting visual images into tactile representations for use by a person who is visually impaired
US5942970A (en) * 1998-10-08 1999-08-24 Norman; Jim Image optical-to-tactile converter
US20030151597A1 (en) * 2002-01-03 2003-08-14 Roberts John W. Extended refreshable tactile graphic array for scanned tactile display
GB2390733A (en) * 2002-07-08 2004-01-14 Baback Asady Navigation and communication aid for the blind

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
TAKAFUMI ASAO ET AL: "A Study on Fundamental Information Transmission Characteristics of an Air-Jet Driven Tactile Display", 19 July 2009, HUMAN-COMPUTER INTERACTION. NOVEL INTERACTION METHODS AND TECHNIQUES, SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, PAGE(S) 397 - 406, ISBN: 978-3-642-02576-1, XP019122135 *
VISELL ET AL: "Tactile sensory substitution: Models for enaction in HCI", INTERACTING WITH COMPUTERS, BUTTERWORTH-HEINEMANN, GB, vol. 21, no. 1-2, 1 January 2009 (2009-01-01), pages 38 - 53, XP025939693, ISSN: 0953-5438, [retrieved on 20080830], DOI: DOI:10.1016/J.INTCOM.2008.08.004 *

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2487672A (en) * 2011-01-31 2012-08-01 Univ Sheffield Active sensory augmentation device
WO2014027228A3 (en) * 2012-08-16 2014-09-12 Uab Gaminu Apparatus for converting surroundings-related information into tactile depth map information
FR2999741A1 (en) * 2012-12-17 2014-06-20 Centre Nat Rech Scient HAPTIC SYSTEM FOR NON-CONTACT INTERACTING AT LEAST ONE PART OF THE BODY OF A USER WITH A VIRTUAL ENVIRONMENT
WO2014095765A3 (en) * 2012-12-17 2015-06-25 Centre National De La Recherche Scientifique Haptic system for establishing a contact free interaction between at least one part of a user's body and a virtual environment
US9760166B2 (en) 2012-12-17 2017-09-12 Centre National De La Recherche Scientifique Haptic system for establishing a contact free interaction between at least one part of a user's body and a virtual environment
EP3413288A1 (en) * 2017-06-09 2018-12-12 Honda Research Institute Europe GmbH Method for assisting a person in acting in a dynamic environment and corresponding system
US10475348B2 (en) 2017-06-09 2019-11-12 Honda Research Institute Europe Gmbh Method for assisting a person in acting in a dynamic environment and corresponding system
WO2023019212A1 (en) * 2021-08-12 2023-02-16 Triton Systems, Inc Arm-mounted hands-free haptic display
US20230053132A1 (en) * 2021-08-12 2023-02-16 Triton Systems, Inc. Arm-mounted hands-free haptic display
CN114639205A (en) * 2022-03-30 2022-06-17 中国工商银行股份有限公司 Password input device and method based on air injection induction and self-service equipment

Similar Documents

Publication Publication Date Title
EP3410264B1 (en) Image display device and image display method
WO2011117794A1 (en) Methods and devices for tactilely imparting information
US20170091535A1 (en) Head-mounted display with facial expression detecting capability
US20170186236A1 (en) Image display device, image display method, and computer program
JP5791082B2 (en) Image composition apparatus, image composition system, image composition method and program
WO2017161726A1 (en) Virtual reality device and virtual reality system
WO2017104320A1 (en) Image display device
US11907423B2 (en) Systems and methods for contextualized interactions with an environment
JPH10153946A (en) Sensation information presenting device
CN110770628A (en) Head-mounted display, display screen thereof, head-mounted support and video
TW202001787A (en) Guided virtual reality system for relaxing body and mind
KR20180006573A (en) The apparatus and method of forming a multi experience
CN104408702B (en) A kind of image processing method and device
CN105630145A (en) Virtual sense realization method and apparatus as well as glasses or helmet using same
US20190121515A1 (en) Information processing device and information processing method
KR20170101703A (en) System and apparatus for providing virtual experience
US20220100271A1 (en) Systems, Methods, and Graphical User Interfaces for Updating Display of a Device Relative to a User's Body
CN106842565A (en) A kind of wearable intelligent vision enhancing equipment of separate type
US9298010B2 (en) Wearable optical display with audio functionality
CN207611201U (en) A kind of VR helmets shell
US20200285310A1 (en) Information processing apparatus, information processing method, and program
US20230306695A1 (en) Devices, methods, and graphical user interfaces for three-dimensional user experience sessions in an extended reality environment
KR102561740B1 (en) Eye movement device for enlarging the viewing angle and the method of eye movement using it
WO2023183340A1 (en) Devices, methods, and graphical user interfaces for three-dimensional user experience sessions in an extended reality environment
JP2022108194A (en) Image projection method, image projection device, unmanned aircraft and image projection program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11717752

Country of ref document: EP

Kind code of ref document: A1

WPC Withdrawal of priority claims after completion of the technical preparations for international publication

Ref document number: 61/315,955

Country of ref document: US

Date of ref document: 20120830

Free format text: WITHDRAWN AFTER TECHNICAL PREPARATION FINISHED

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11717752

Country of ref document: EP

Kind code of ref document: A1