WO2011117794A1 - Methods and devices for tactilely imparting information - Google Patents
Methods and devices for tactilely imparting information Download PDFInfo
- Publication number
- WO2011117794A1 WO2011117794A1 PCT/IB2011/051157 IB2011051157W WO2011117794A1 WO 2011117794 A1 WO2011117794 A1 WO 2011117794A1 IB 2011051157 W IB2011051157 W IB 2011051157W WO 2011117794 A1 WO2011117794 A1 WO 2011117794A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- gas
- flow
- nozzle
- image
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/001—Teaching or communicating with blind persons
- G09B21/003—Teaching or communicating with blind persons using tactile presentation of the information, e.g. Braille displays
Definitions
- the invention in some embodiments, relates to the field of imparting information, for example image information, for example to the visually- impaired. More particularly, in some embodiments, the invention relates to methods and devices for imparting information, such as image information, as a modulated flow of gas to a skin surface of a subject.
- Visual perception is the ability to interpret information from visible light reaching the eye, achieved by the visual system.
- the lens of the eye focuses an image of the surroundings onto the photoreceptive cells of the retina, which detect the photons of light in the image and respond by producing neural impulses that represent the image.
- the neural impulses are sent as image information to the brain.
- Blindness is the condition of lacking visual perception due to physiological or neurological factors. The extent of vision loss may vary. Total blindness involves complete loss of perception of form and visual light. Legal blindness is defined in North America and most of Europe as visual acuity of 20/200 (6/6) or less in the better eye with best correction possible. Approximately ten percent of those deemed legally blind have no vision; the rest have some vision, from light perception alone to relatively good acuity.
- Visual perception enables humans to navigate or interact with the world, act fully independently, and be aware of events surrounding them.
- Visually- imp aired and blind people require techniques and tools that allow them to complete daily activities using their remaining senses.
- Such tools include a white cane, which is used to extend the user's range of touch sensation.
- a white cane is usually swung in a low sweeping motion across the intended path of travel, to detect obstacles.
- a white cane is only useful in detecting the presence of an object within the range of the cane.
- a white cane does not provide any information regarding the appearance (size, shape, etc.) of an object and does not provide a complete image of what is before the user.
- the invention in some embodiments, relates to methods and devices for imparting information to a human subject by directing a modulated flow of gas to a skin surface of the subject.
- a method for tactilely imparting information to a subject comprising:
- modulating a flow of gas in accordance with information to be imparted ; and directing the modulated flow of gas towards a skin surface of the subject thereby allowing the subject to tactilely sense the modulation of the flow of gas and thus to perceive the imparted information.
- a device for tactilely imparting information to a subject comprising:
- At least one nozzle configured to direct a flow of gas towards a skin surface of a subject
- a controller for controlling the actuator to modulate the flow of gas in accordance with information to be imparted such that the modulated flow of gas constitutes a tactilely- sensible representation of information to be imparted.
- the information is sensory information. In some embodiments, the information is visual information. In some embodiments, the information is image information, and the modulating of the flow of gas is such that the flow of gas constitutes a tactilely- sensible representation of the image.
- Embodiments of methods and/or devices of the invention may involve performing or completing selected tasks manually, automatically, or a combination thereof.
- Some embodiments of the invention are implemented with the use of components that comprise hardware, software, firmware or combinations thereof.
- some components are general-purpose components such as general-purpose computers or processors.
- some components are dedicated or custom components such as circuits, integrated circuits or software.
- some of an embodiment is implemented as a plurality of software instructions executed by a data processor, for example which is part of a general-purpose or custom computer.
- the data processor or computer comprises volatile memory for storing instructions and/or data and/or a non-volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data.
- implementation includes a network connection.
- implementation includes a user interface, generally comprising one or more of input devices (e.g., allowing input of commands and/or parameters) and output devices (e.g., allowing reporting parameters of operation and results.
- FIGS. 1A and IB schematically depict an embodiment of a device as described herein worn by a user
- FIG. 2 is a flow chart describing an embodiment of a method as described herein;
- FIGS. 3A-3D schematically depict distance images (3A and 3C) and corresponding modulated pixelated gas-flow representing the distance images (3B and 3D, respectively);
- FIGS. 4A-4C schematically depict an embodiment of a device as described herein; and
- FIGS. 5 A and 5B schematically depict vector image-representations (5 A and 5D) of distance images (3A and 3C, respectively).
- the invention in at least some embodiments, relates to methods and devices for imparting information tactilely, as a modulated flow of gas to a skin surface of a subject.
- a method for tactilely imparting information to a subject comprising:
- the subject is a human subject, in some embodiments a visually- impaired human subject.
- the modulating of the flow of gas is such that the modulated flow of gas constitutes a representation of the information.
- any suitable information is imparted using the method described herein.
- any suitable information is imparted using the method described herein, for example sensory information, such as visual information, i.e., information usually perceived by a human using vision.
- the visual information is image information and the modulating of the flow of gas is such that the flow of gas constitutes a tactile ly- sensible representation of the image.
- the flow of gas is modulated so as to constitute a representation of the information to be imparted that is tactilely perceivable by the subject, allowing the subject to "feel" the information on the skin, for example, when the information is image information, the subject 'feels' the image on the skin.
- the gas flow is modulated in order to provide information regarding the size and shape of an object, and the direction and distance of the object from the subject, as in an image.
- the information is image information of a monochrome image In some embodiments, the information is image information of a color image. In some embodiments, the information is image information of a distance image.
- information from any suitable source is imparted using the method described herein.
- the information is real information, e.g., real acquired images of the real world.
- the information is virtual information, e.g., generated images of a virtual world or reality.
- the information is imparted in real-time, that is to say, imparted substantially immediately with acquisition (real information) or generation (virtual information).
- the information is stored in a storage medium and is recovered from the storage medium and imparted when desired, at a time substantially different from when acquired (real information) or generated (virtual information).
- the information is real information and the method further comprises acquiring the real information, and (substantially immediately) modulating the flow of gas to impart the real information in real-time.
- the real information is acquired and imparted in real-time, that is to say, the real information is acquired and substantially immediately imparted to a user in accordance with the teachings herein.
- the information is real image information of an image that is acquired and the information is imparted to the subject substantially immediately by modulating the flow of gas in accordance with the acquired image such that the flow of gas constitutes a tactile ly- sensible representation of the acquired image.
- Such embodiments are useful, for example in allowing a visually-impaired person to "see" the real world and to navigate therein.
- an image is a distance image acquired using a distance camera (also known as a range camera) using any suitable technology, e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect, Microsoft Corporation, Redmond, Washington, USA), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture techniques.
- the image is a monochrome image or a color image acquired using a digital camera or image sensor as known in the art of digital photography and cellular telephony. Real-time imparting of virtual information
- the information is virtual information (e.g., image information of a computer-generated image in a virtual world or reality) and the method further comprises generating the information, and (substantially immediately) modulating the flow of gas to impart the virtual information in real-time.
- the information is generated and imparted in real-time, that is to say, the information is generated and substantially immediately imparted to a user in accordance with the teachings herein.
- the information is image information of an image that is generated (in the usual way, for example as known in the field of animation or in the field of video gaming) and the information is imparted to the subject substantially immediately by modulating the flow of gas in accordance with the generated image such that the flow of gas constitutes a tactile ly- sensible representation of the generated image.
- Such embodiments are useful, for example in allowing a visually-impaired person to "see” a virtual world, for example, allowing playing of an interactive video game or to consume animated video entertainment such as cartoons, stop-action films (e.g., Wallace and Gromit, Aardman Animations Ltd., Bristol, UK) or computer-generated films.
- the information is stored, and the method further comprises retrieving the stored information and then modulating the flow of gas to impart the retrieved information, e.g., pre-generated virtual information such as image information of an animated movie generated and stored on electronic storage media or recorded real information such as real image information acquired using a camera as described above, recorded and stored on electronic storage media.
- the stored information is retrieved and subsequently imparted to a user in accordance with the teachings herein.
- Such embodiments are useful, for example, in allowing a visually- impaired person to view a movie, an animated movie or to train in using the method. Modulation
- the modulation of the flow of gas may be any suitable modulation, that is to say, characteristics of the gas flow change as a function of time in a sensible way to impart information to the subject. It is important to note that it is known that a skin surface, especially root hair plexuses becomes desensitized to a constant stimulation such as a constant flow of air relatively relatively quickly but is very sensitive to a changing stimulation such as a fly or ant walking along the skin surface. Accordingly, modulation of a flow of gas for implementing the teachings herein changes to render the flow of gas tactilely sensible.
- modulation comprises changing an intensity (mass of gas per unit area per unit time) of at least a portion of the gas-flow.
- Typical usable intensities are similar to intensities of air exhaled by a person through the mouth.
- a nearby object is represented by a more intense (higher rate of flow) gas-flow than a farther object to impart image information related to a distance image; or a darker- colored object is represented by a more intense gas-flow than a lighter- colored object to impart image information related to a monochrome or color image; or higher wavelength- colored (redder) object is represented by a more intense gas-flow than a lower-wavelength colored (e.g., more blue/indigo/violet) object to impart image information related to a color image.
- modulation comprises changing a frequency of variation of intensity of at least a portion of the gas-flow.
- Typical usable frequencies are between about 1 Hz and about 60 Hz.
- a nearby object is represented by a more rapidly changing (higher frequency pulsed) gas-flow than a farther object to impart image information related to a distance image; or a darker-colored object is represented by a more rapidly changing gas-flow than a lighter- colored object to impart image information related to a monochrome image; or higher wavelength-colored (redder) object is represented by a more rapidly changing gas- flow than a lower- wavelength colored (more blue/indigo/violet) object to impart image information related to a color image.
- modulation comprises changing a direction at which at least a portion of the gas flow reaches the skin surface.
- changing a direction comprises changing an angle of incidence of the gas flow to the skin surface, typically between perpendicular (0°) to the skin surface and about 60° to the skin surface.
- changing a direction comprises changing an orientation, in analogy to compass directions.
- such gas flow is "pulsed", that is to say, there is a change of intensity as a function of time.
- the intensity of such gas flow is substantially constant during the change of direction. For example, information about an object in an image (distance, monochrome or color) is "traced" on the skin surface of the subject by changing the direction of at least a portion of the gas flow.
- modulation comprises changing a location of the skin surface at which at least a portion of the gas flow reaches the skin surface.
- such gas flow is "pulsed", that is to say, there is a change of intensity as a function of time.
- the intensity of such gas flow is substantially constant during the change of location. For example, an outline of an object in an image (distance, monochrome or color) is "traced" on the skin surface of the subject with at least a portion of the gas flow, optionally with a rate of change of intensity imparting distance information (e.g., from a distance image), intensity information (e.g., from a monochrome image) or color information (e.g., from a color image).
- distance information e.g., from a distance image
- intensity information e.g., from a monochrome image
- color information e.g., from a color image
- the flow of gas is pixelated, for example an object in an image (a distance, monochrome or color image) is "displayed" on the skin surface of the subject with a pixelated gas flow, optionally with a rate of change of intensity of each pixel imparting distance information (e.g., from a distance image), intensity information (e.g., from a monochrome image) or color information (e.g., from a color image).
- distance information e.g., from a distance image
- intensity information e.g., from a monochrome image
- color information e.g., from a color image
- the modulated flow of gas is directed to any suitable skin surface of a subject, preferably based on considerations of sensitivity to sensing modulation of a modulated air flow, interference with other tasks, aesthetic considerations and personal preference.
- a preferred suitable skin surface is a skin surface with higher such sensitivity due to, for example, greater concentrations of touch-sensitive nerve-endings or root hair plexuses (light-touch mechanoreceptors that detect bending of hairs, enabling movement of the hairs to be detected even if the skin is not touched directly). That said, it is also preferred that the suitable skin surface be selected so that implementation of the teachings herein does not substantially interfere with performing other tasks.
- a suitable skin surface is selected from the group consisting of a surface of an inner wrist, an arm, a nape of a neck, an earlobe, an ear canal, behind an ear, and especially a forearm.
- other suitable skin surfaces with high sensitivity include skin surfaces selected from the group consisting of a surface of a hand, a lip, a tongue, a face and inside of a nostril.
- skin surfaces selected from the group consisting of surfaces of a leg and of a torso.
- Embodiments of the methods described herein are implemented using any suitable device. In some embodiments, it is preferred to use a device as described herein for implementing such methods.
- a device for tactilely imparting information to a subject comprising:
- At least one nozzle configured to direct a flow of gas towards a skin surface of a subject
- a controller for controlling the actuator to modulate the flow of gas in accordance with information to be imparted such that a modulated flow of gas constitutes a tactilely- sensible representation of the information to be imparted.
- a device as disclosed herein comprises at least one nozzle configured to direct a flow of gas towards a skin surface of a subject.
- the at least one nozzle is one nozzle. In some embodiments, the at least one nozzle is at least two nozzles.
- the at least one nozzle comprises an array of nozzles, e.g., an x by y array of nozzles, where x and y are independently integers of at least 4.
- a nozzle array is useful in imparting image information as a pixelated flow of gas.
- the controller activates an actuator so that each nozzle of the nozzle array corresponds to a pixel of an imparted image, and each pixel is "displayed” by directing a flow of gas at a skin surface from the corresponding nozzle.
- an x by y array of pixels is "displayed" by an x by y array of nozzles. Conversion of image information to the pixel resolution of a nozzle array is easily performed using any suitable pixelization method or algorithm.
- a device as disclosed herein comprises an actuator functionally associated with the at least one nozzle, the actuator for modulating the flow of gas from the at least one nozzles.
- a typical actuator comprises one or more components for directing and changing an air flow from or through a nozzle such as valves, motors, step motors, bearings, moveable nozzle mounts, vanes, adjustable apertures, flaps and the like, for example, as known in the art of radio-controlled model aircraft.
- the at least one nozzle is at least two nozzles
- the actuator is configured to modulate a flow of gas from at least two of the nozzles independently, e.g., the at least one nozzle is a nozzle array and the actuator is configured to modulate the flow of gas from each of the nozzles of the nozzle array independently.
- the actuator is configured to change an intensity of the gas- flow from at least one nozzle.
- the actuator is configured to change a frequency of variation of intensity of the gas-flow from at least one nozzle.
- the actuator is configured to change a direction of the gas-flow from at least one nozzle (in some embodiments the angle and/or in some embodiments the orientation in analogy to a compass), thereby changing a direction at which at least a portion of the gas-flow (the portion from that nozzle) is directed at a skin surface.
- the actuator is configured to change a location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed. In some embodiments, the actuator is configured to translate at least one nozzle, thereby changing the location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed. In some embodiments, the actuator is configured to stop gas flow from at least one nozzle and to allow gas flow from at least one other nozzle, thereby changing the location of a skin surface at which at least a portion of the gas-flow is directed.
- the actuator is configured to change a direction of the gas-flow from a nozzle (in some embodiments the angle and/or in some embodiments the orientation in analogy to a compass), thereby changing a location of a skin surface at which at least a portion of the gas-flow (the portion from that nozzle) is directed.
- the actuator is configured to generate a pixelated flow of gas.
- the at least one nozzle comprises a matrix of nozzles, each nozzle corresponding to a pixel and the actuator configured to control the flow of gas from each nozzle independently.
- the at least one nozzle comprises a single nozzle and the actuator is configured to serially aim the nozzle (e.g., by rotating and/or translating the nozzle) to direct a flow of gas at different portions of a skin surface, each such portion constituting a pixel.
- a device as disclosed herein comprises a controller for controlling an actuator to modulate the flow of gas in accordance with information to be imparted.
- a controller is a component that, based on the information to be imparted, generates commands to the actuator that yield the appropriate modulation of the flow of gas such that the flow of gas constitutes a tactilely- sensible representation of the information.
- a controller typically comprises a suitable general-purpose or custom-made device, for example an appropriately- configured general purpose digital computer (such as known in the field of "smartphones", e.g., Iphone 4 by Apple Incorporated, Cupertino, California, USA) with a user-input channel, an actuator-control output, a memory and a processor configured to convert information to be imparted to actuator-control output.
- a suitable general-purpose or custom-made device for example an appropriately- configured general purpose digital computer (such as known in the field of "smartphones", e.g., Iphone 4 by Apple Incorporated, Cupertino, California, USA) with a user-input channel, an actuator-control output, a memory and a processor configured to convert information to be imparted to actuator-control output.
- a user-input channel for example implemented as a touch screen with a graphic-user interface (such as well-known in the field of "smartphones") is used to control various parameters and user preferences for operating the device.
- a graphic-user interface such as well-known in the field of "smartphones”
- a controller is also configured to accept the information to be imparted.
- a controller of a device further includes an information- input channel to accept information to be imparted to a user.
- Such an information input channel is typically implemented as an electronic input (wired or wireless) to accept information to be imparted to a user.
- the information is any information to be imparted as described hereinabove, including sensory information, visual information, and image information, in any suitable format, typical digital format.
- the information is image information (e.g., of a distance image, of a monochrome image, of a color image) in the form of a digital image data file as known in the art.
- a controller generally includes a processor configured to convert information to be imparted to actuator-control output, that is to say, commands for the actuator to modulate the flow of gas to represent the information to be imparted.
- the processor functions as an image-representation generator, to convert image information to actuator-control output for the actuator to modulate the flow of gas to represent the image information.
- the controller is configured to convert image information to actuator-control output for the actuator to modulate a flow of gas to constitute a tactilely- sensible pixelated representation of image information.
- the processor functions as an image-representation generator to convert image information to actuator-control output for the actuator to produce a pixelated flow of gas to impart image information, including by directing a flow of gas (with optional changes in intensity and/or direction) at a plurality of distinct locations on the skin surface to "print" a pixelated representation of the image information to be imparted. Any suitable known method of image pixelation may be used in implementing such embodiments.
- a plurality of pixels are "displayed" by a single nozzle, for example a single nozzle translatable in an x and y direction, or a linear array of nozzles moveable together in a single direction, or a rectangular array of fixed nozzles.
- the size, shape and distance of an object in an image is represented by modulation of a pixelated gas-flow.
- the image is represented such that portions (one or more pixels) of the gas flow representing closer objects change at higher frequency than portions (one or more pixels) of the gas flow representing farther objects.
- an image is represented by a gas-flow made up of a 20 x 20 pixel array in a 5 cm x 5 cm array.
- the controller is configured to convert image information to actuator-control output for the actuator to modulate a flow of gas to constitute a tactilely- sensible vector representation of image information.
- the processor functions as an image-representation generator to convert image information to actuator-control output for the actuator to produce a vector image representation to impart image information, including by translating a nozzle with a continuous flow of gas (with optional changes in intensity and/or direction) to "draw" a vector representation of the image on the skin surface. Any suitable known method of vector image representation may be used in implementing such embodiments.
- a device as disclosed herein further comprises a supporting component (in some embodiments also called a supporting backing component) to which the at least one nozzle is attached, the supporting component configured to be worn on a portion of a body of a subject so that the at least one nozzle is positioned to direct the flow of gas towards a skin surface of the subject.
- a supporting component in some embodiments also called a supporting backing component
- components in addition to the at least one nozzle such as the actuator and the controller are also attached to the supporting component.
- the supporting component comprises a frame or container, e.g., of aluminum or plastic.
- the supporting component includes a cover or shield to isolate a skin surface from external stimulus.
- the supporting component is configured to be worn appropriately so that the at least one nozzle is positioned to direct the flow of gas towards a selected skin surface, as discussed above.
- the skin surface is selected from the group consisting of a surface of an inner wrist, an arm, a nape of a neck, an earlobe, an ear canal, behind an ear, and especially of a forearm, of a hands, a tongue, a face and the inside of a nostril and in some embodiments, of a leg and of a torso.
- the supporting component is configured to be worn on any suitable portion of the body of a subject as described above.
- the supporting component typically includes (reversible) securing components to secure the supporting component in appropriate proximity and orientation to the selected skin surface, such as one or more of straps, snaps, ties, clips, hook-and-loop fasteners, and the like.
- At least one nozzle attached to the supporting component faces inwards, so when the supporting component is properly worn by a subject, the at least one nozzle is directed at a skin surface of the subject.
- At least one nozzle is immovably attached to the supporting component, so that flow of gas from the nozzle is directed to a specific location on a skin surface when the supporting component is properly worn.
- At least one nozzle is rotatably attached to the supporting component, so that the direction of flow of gas from that nozzle can be changed by rotation of the nozzle by the actuator. In some embodiments, at least one nozzle is translatably attached to the supporting component so that the location on the skin surface at which a flow of gas from that nozzle is directed is changeable by translation of the nozzle by the actuator.
- a device further comprises a power source (typically a rechargeable battery or similar) for supplying the power required for operation of components of the device such as the controller and actuator.
- a power source typically a rechargeable battery or similar
- such a power source is secured to the supporting component.
- the flow of gas through the at least one nozzle is of any suitable gas.
- a gas is inert and/or harmless and/or non-flammable, for example, air, CO2, helium, neon, or N 2 .
- the flow of gas through the at least one nozzle is provided by any suitable component or device.
- the device further comprises a gas-flow generating component to provide a flow of gas through the at least one nozzle.
- the gas-flow generating component comprises a compressed gas container, e.g., of CO2, air, helium, neon or N2.
- the gas-flow generating component comprises a gas-flow generator, e.g., a portable air pump, that generates a gas flow by drawing in and compressing ambient air.
- the information to be imparted is provided to the controller in any suitable fashion.
- information is stored on the controller.
- the controller comprises an information input channel and information to be imparted is provided from any suitable source, e.g., a computer, a gaming console, a mobile telephone or an image storage device.
- the device is configured to impart acquired information in real- time.
- an information acquirer is functionally associated with the controller through an information input channel.
- a device further comprises, in communication with the controller, an information acquirer to acquire information to be imparted and to provide the acquired information to the controller, and the controller is configured to control the actuator to modulate the flow of gas in accordance with the acquired information in real time.
- the information acquirer is in wired communication with the controller.
- the information acquirer is in wireless communication with the controller, (e.g., using Bluetooth® technology).
- an information acquirer is a component of the device.
- an information acquirer is an image acquirer, e.g., a camera.
- an image acquirer is a distance camera (also known as a range camera) that generates an image of distance as a function of x and y, using any suitable technology, e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect from Microsoft, Redmond, Washington), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture.
- a distance camera also known as a range camera
- any suitable technology e.g., stereo triangulation, sheet of light triangulation, structured light (e.g., Kinect from Microsoft, Redmond, Washington), time of flight (e.g., PMD Technologies, Siegen, Germany), interferometry and coded aperture.
- an image acquirer is a monochrome or color camera, e.g., a visible light camera as known in the art of cellular telephony. In some embodiments, an image acquirer is an infrared camera or a thermal imaging camera.
- the information acquirer is configured to be (reversibly) worn by the subject. In some embodiments, the information acquirer is configured to be worn on the head of the subject, e.g., attached to a head band, a cap or the like.
- some device components are in wired communication with other device components.
- some device components are in wireless communication with other device components, for example using Bluetooth® technology.
- an image acquirer and a controller make up a single physical unit configured to be worn on the head of a subject while the actuator, at least one nozzle and gas-flow supply make up a single physical unit configured to be worn on the arm of a subject, where the controller and the actuator are in wireless communication.
- the controller and the actuator are in wireless communication.
- an image acquirer makes up a single physical unit configured to be worn on the head of a subject while the controller, the actuator, at least one nozzle and gas-flow supply make up a single physical unit configured to be worn on the arm of a subject, where the image acquirer and the controller are in wireless communication.
- Such an embodiment allows a reduced size of components that are (uncomfortably) worn on the head.
- Figures 1A and IB an embodiment of a device in accordance with the teachings herein including two separate physical units, an information imparting (actuator) unit 10 configured to be worn on the forearm of a human subject and an image-data acquiring (generation) unit 30 configured to be worn on the head of a human subject, is schematically depicted.
- Information-imparting unit 10 comprises a plurality of nozzles 12, attached to a flexible supporting (backing surface) component 14 of silicone rubber, within a fixed-area rigid grid of polycarbonate to constitute a 5 by 5 nozzle array having a total of 25 nozzles.
- Supporting component 14 comprises a reversible attachment component 24 (straps with hook-and loop fasteners such as Velcro®) for securing information-imparting unit 10 over a skin surface of a subject, a forearm 16, such that nozzles 12 face the skin surface.
- Information-imparting unit 10 further comprises a reservoir 18 of compressed N2 gas in fluid communication with nozzles 12 through a manifold pipe (not apparent in the Figures).
- An actuator 20 of the device comprises twenty- five individually- operable adjustable- aperture valves, each valve associated with a branch of the manifold pipe that directs nitrogen gas from reservoir 18 to a different nozzle 12.
- Each valve of actuator 20 is configured to independently prevent gas flow from an associated nozzle 12, or to allow a certain gas flow to exit from an associated nozzle 12.
- actuator 20 is configured to independently change an intensity of a gas-flow from each of nozzles 12 (by opening or closing an associated valve), to independently change frequency of variation of intensity of a gas-flow from each of nozzles 12 (by changing the rate of opening/closing of an associated valve) and is configured to change a location of a skin surface at which at least a portion of the gas-flow is directed (by opening a valve associated with a given nozzle 12 to allow gas flow from that nozzle while closing a valve associated with a different nozzle 12 to stop gas flow from that nozzle).
- Image-data acquiring component 30 comprises a distance-image acquirer (image- acquisition component) time-of-flight camera 32, attached to a removable headband 34, for positioning on the head of the user, and a controller 36 (a digital computational device similar to a smartphone) configured to function as an image-representation generator, for accepting image information (data) acquired by camera 32 as input and for controlling release of gas from reservoir 18 as a modulated gas-flow through nozzles 12, by controlling actuator 20 to modulate the flow of gas from nozzles 12 in accordance with information to be imparted.
- a controller 36 a digital computational device similar to a smartphone
- controller 36 received an image from camera 32, pixelates the image to the resolution of the nozzle array (5 by 5) where the intensity of gas flow from a pixel is related to a distance to an object in the image (closer is more intense gas flow) and sends a series of commands in real time to control actuator 20 to modulate a gas-flow to constitute a representation of an image represented by the image data.
- the commands are wirelessly transmitted by controller 36 using Bluetooth® transceiver 38 to Bluetooth transceiver 22 located on actuator component 10, in communication with actuator 20.
- actuator 20 controls the flow of gas from each nozzle 12 (e.g., by opening and closing valves) in such a way that the image acquired by camera 32 is represented by a modulated pixelated gas-flow from the array of nozzles 12 so that the pixelated gas-flow constitutes a tactile ly- sensible representation of the image acquired by camera 32.
- FIG. 2 presents an overview of an embodiment of a method of imparting information as described herein as a flow chart.
- Camera 32 acquires an image and generates image data representing the image.
- a controller 36 functioning as an image-representation generator accepts the image data as input and generates actuator commands that are sent to an actuator 20 through an actuator-control output.
- An actuator 20 follows the actuator commands, modulating the flow of gas through the array of nozzles 12 to form a modulated pixelated gas-flow directed towards a skin surface so that the pixelated gas-flow constitutes a tactile ly- sensible representation of the image acquired by camera 32.
- information such as image data is provided to a controller 36 by some component other than an image-acquisition component, e.g., a component that provides stored or computer-generated image data.
- Figure 3A is a schematic depiction of a distance image acquired by a distance camera such as 32.
- Figure 3B is a schematic depiction of a corresponding modulated pixelated gas flow generated by a nozzle array such as depicted in Figure IB.
- Figure 3A are modulated by pulsing (changing the intensity of flow) at an 80 Hz frequency
- air flowing in pixels corresponding to a further object (pole 44 at 4 meter) in the distance image is pulsed at 20 Hz frequency
- air flowing in pixels corresponding to objects at infinite (greater than 8 meter) in the distance image is stopped completely.
- Figure 3C is a schematic depiction of a distance image similar to that of Figure 3 A, where the camera is further away from the objects in the image.
- Figure 3D is a schematic depiction of a corresponding modulated pixelated gas flow generated by a nozzle array such as depicted in Figure IB.
- air flowing in pixels corresponding to a close object (person 42 at 4 meter) in the distance image are modulated by pulsing at an 20 Hz frequency
- air flowing in pixels corresponding to a further object (pole 44 at 7 meters) in the distance image is pulsed at 10 Hz frequency
- air flowing in pixels corresponding to objects at infinite (greater than 8 meter) in the distance image is stopped completely.
- Device 50 is configured to acquire distance images and impart information from the acquired images to a subject using device 50 by modulating a flow of gas in accordance with the information to be imparted such that the modulated flow of gas constitutes a tactilely- sensible representation of information to be imparted.
- Device 50 comprises two physically separate units: information- imparting unit 52 depicted in detail in Figures 4A and 4B configured to be worn on the forearm of a human subject and an image-data acquiring unit 54 depicted in detail in Figure 4C, configured to be worn on the head of a human subject.
- Information-imparting unit 52 comprises a rigid supporting component 56, substantially a closed rigid box of polyethylene that acts to shield a selected skin surface from ambient stimulus and contains or supports other device components and includes straps 24 to allow information-imparting unit 52 to be reversibly worn on the forearm of a human subject.
- a single nozzle 12 (hidden from view) configured to direct a flow of gas towards a skin surface of a subject properly wearing information-imparting unit 52;
- controller 36 for controlling actuator 20 to modulate the flow of gas in accordance with image information to be imparted to a subject such that the modulated flow of gas constitutes a tactilely- sensible representation of the information
- a portable air pump 58 as a gas-flow generator
- a Li-ion battery 60 as a power source for the other components.
- Actuator 20 comprises a rigid square 4cm by 4 cm frame 62 immovably- contained inside supporting component 56.
- Two electrical step motors 64a and 64b (as known in the art of RC model airplanes) are immovably secured to two adjacent edges 62a and 62b of frame 62.
- To a rotor 66a and 66b of each respective motor 64 is attached a proximal end of a respective slotted bar 68a and 68b. Closer to the distal ends of the bars, slotted bars 68a and 68b overlap.
- a support pin 70 In the hole defined by the overlapping slots of slotted bars 68a and 68b is slidingly secured.
- Nozzle 12 is a standard gas nozzle secured to support pin 70 so as to be directed perpendicularly to a skin surface when information-imparting unit 52 is properly worn by a human subject in fluid communication with portable air pump 58 through pipe 72.
- Valve 74 comprises two controllable modules, needle valve 76 with an associated electrical motor that allows precise control of the rate of gas flow from air pump 58 to nozzle 12 and rotatable perforated disk 78 with an associated electrical motor that allows control and variation of the frequency of the change of air flow intensity from nozzle 12. Specifically, when a perforation of disk 78 is located across the lumen of pipe 72, air flows to nozzle 12 but when a solid portion of disk 78 is located across the lumen of pipe 72, air is blocked from passing to nozzle 12.
- actuator 20 is configured to change an intensity of a gas-flow from nozzle 12 (using valve 74) to change a frequency of variation of intensity of a gas-flow from nozzle (by changing a rate of rotation of disk 78 of valve 74) and to change a location of a skin surface at which gas-flow from nozzle 12 is directed (using motors 64a and 64b to translate nozzle 12 inside frame 62).
- Controller 36 (a portable digital computer, similar to that used in an Iphone 4 by Apple Incorporated, Cupertino, California, USA) having a touch-screen (not depicted) and associated hardware and software components constituting a user-input channel for accepting user commands, an information-input channel 80 in wired communication with a Bluetooth® transceiver 38 for accepting image information as an image in a standard format (e.g., any format produced by an image acquirer, see below) and a processor 82 for accepting image information from information-input channel 80, processor 82 configured to convert the accepted image information to commands to actuator 20 to modulate a flow of gas to represent the received image.
- a standard format e.g., any format produced by an image acquirer, see below
- Image-data acquiring unit 54 includes a distance camera 84 (similar to a Kinect by Microsoft Corporation, Redmond, Washington, USA) as a distance-image acquirer attached to a head band 34, a Bluetooth® transceiver 38 to provide wireless communication between distance camera 84 and controller 36, specifically, to information input channel 80 of controller 36. Not depicted is a battery supplying power for the components of image-data acquiring unit 54.
- image acquiring unit 54 is worn on the head of a subject with the help of headband 34 and information-imparting unit 52 is worn on the forearm of a subject with the help of straps 24.
- the various components of device 50 are activated.
- Distance camera 84 acquires a distance image in the usual way and transmits the acquired image using transceiver 38 to controller 36 through information-input channel 80 via transceiver 22.
- processor 82 of controller 36 applies a vector image representation algorithm to convert the acquired image received from distance camera 84 to a vector image, where objects in the image are represented by a set of vectors in a plane, each such vector having an intensity value related to the distance to the represented object.
- controller 36 then controls actuator 20 to modulate the flow of gas directed by nozzle 12 towards the forearm skin surface of the subject to constitute a tactilely- sensible representation of information to be imparted.
- controller 36 translates each vector in the vector image to a linear motion of nozzle 12 having a direction and length matching the vector, and an intensity of air flow related to the distance to an object represented by the vector, where greater intensity corresponds to a closer object.
- controller 36 uses actuator 20 and nozzle 12 to "draw" a vector representation of the acquired image on the skin surface.
- Figure 5A is a schematic depiction of a modulated gas flow directed at a skin surface by device 50 that is substantially a vector image representation of the distance image depicted in Figure 3 A.
- the vertical line 44 at the left having a continuous low intensity air flow of 20 (arbitrary units) corresponds to a far object (pole 44 at 4 meter) while the three lines 42a, 42b and 42c having a continuous high intensity air flow of 80 (arbitrary units) correspond to a near object (person 42 at 1 meter).
- Figure 5B is a schematic depiction of a modulated gas flow directed at a skin surface by device 50 that is substantially a vector image representation of the distance image depicted in Figure 3C.
- the vertical arrow 44 at the left having a continuous low intensity air flow of 10 (arbitrary units) corresponds to a far object (pole 44 at 7 meter) while line 42 having a continuous high intensity air flow of 20 (arbitrary units) corresponds to a near object (person 42 at 4 meter).
- actuator 20 of device 50 is also configured to generate a pixelated flow of gas, requiring only appropriate configuration of controller 36 to control actuator 20 to modulate the flow of gas through nozzle 12 to constitute a tactile ly- sensible pixelated representation of image information.
- processor 82 of controller 36 is configured to produce a pixelated representation of an image to be imparted to a subject and subsequently processor 82 controls actuator 20 to serially translate nozzle 12 to the locations inside frame 62 that correspond to pixels, and at each pixel direct a flow of gas from nozzle 12, the flow of gas having the appropriate intensity or frequency.
- imparting of image information is discussed in detail. As noted above, in some embodiments, other types of information are imparted to a user.
- generally useful information is imparted to a user.
- text information is imparted to the user, for example as a series of symbols, letters or numerals "drawn” or "printed” on a skin surface with a modulated flow of gas as described herein.
- the approach of a time limit e.g., during playing a game, or when a specific action needs to be taken
- a flow of gas having a periodic intensity, which intensity and/or frequency increase with the approach of the time limit.
- sensory information e.g., information usually imparted by one of the five senses
- visual information i.e., information usually perceived by a human using vision
- a modulated flow of gas is used to impart information about the presence of an obstacle in front of a visually- imp aired person, not as an tactilely- sensible "image” of modulated gas, but as a symbolic "warning", for example a specifically modulated gas flow symbol.
- Such information can be, for example, of a specific color (a red, amber or green traffic light), of a specific shape (e.g., a stop sign) or movement (e.g., a vehicle approaching from one of the sides of the subject).
- imparting of information is imparted at one localized skin surface, specifically an area of skin on one forearm.
- information is imparted concurrently (in some cases, simultaneously) at two different skin surfaces, e.g., two forearms, a forearm and the nape of the neck.
- the information imparted at each of the two different surfaces is different, for example, one arm for image information and the other arm (or nape of the neck) for symbolic or complementary information.
- the information imparted at the two different surfaces, taken together, represents, and is interpreted by the user as, a stereo image representation.
- the teachings herein impart information about visual stimuli in the surroundings to a human subject, in a form which can be understood by such a subject.
- the subject is provided with information regarding the size, shape and distance of an object within a detection range, without requiring actual contact with object.
- the hands of the subject are left free, such that, for example, a visually- impaired subject may perform manual tasks while implementing the teachings herein, or optionally, use a white cane in addition to the teachings herein.
- the teachings herein do not rely on audio signals, such that the hearing of the subject is still available to obtain additional information about the surroundings, increasing the total amount of information potentially imparted and received by the subject.
- Embodiments of the teachings herein allow the visually- imp aired to participate in heretofore unavailable entertainment such as video gaming, watching movies and the like.
- Embodiments of the teachings herein may also be used, for example, by normally- sighted rescue workers in smoke-filled buildings, in order to help locate trapped individuals.
- imparting visual information using prior art methods may be impossible due to difficulties in viewing a visual display of any kind in the presence of dense smoke, particularly where the face of the rescuer may be at least partially covered with a gas mask or other breathing device.
- Embodiments of the teachings herein may be used for imparting information in an additional, yet-unused, mode.
- the teachings herein are implemented as an additional component of a game, for example a multimedia game.
- information is imparted in other fields, for example, in controlling a vehicle (e.g., driving) or other fields where it is desired to impart information.
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US31595510P | 2010-03-21 | 2010-03-21 | |
US61/315,955 | 2010-03-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011117794A1 true WO2011117794A1 (en) | 2011-09-29 |
Family
ID=44262984
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2011/051157 WO2011117794A1 (en) | 2010-03-21 | 2011-03-20 | Methods and devices for tactilely imparting information |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2011117794A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2487672A (en) * | 2011-01-31 | 2012-08-01 | Univ Sheffield | Active sensory augmentation device |
FR2999741A1 (en) * | 2012-12-17 | 2014-06-20 | Centre Nat Rech Scient | HAPTIC SYSTEM FOR NON-CONTACT INTERACTING AT LEAST ONE PART OF THE BODY OF A USER WITH A VIRTUAL ENVIRONMENT |
WO2014027228A3 (en) * | 2012-08-16 | 2014-09-12 | Uab Gaminu | Apparatus for converting surroundings-related information into tactile depth map information |
EP3413288A1 (en) * | 2017-06-09 | 2018-12-12 | Honda Research Institute Europe GmbH | Method for assisting a person in acting in a dynamic environment and corresponding system |
CN114639205A (en) * | 2022-03-30 | 2022-06-17 | 中国工商银行股份有限公司 | Password input device and method based on air injection induction and self-service equipment |
WO2023019212A1 (en) * | 2021-08-12 | 2023-02-16 | Triton Systems, Inc | Arm-mounted hands-free haptic display |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE4337602A1 (en) * | 1993-11-01 | 1995-05-04 | Lehmann Hans Joachim | Dynamic orientation aid and video simulator for the blind |
DE19517649A1 (en) * | 1995-05-17 | 1996-11-21 | Thomas Rupp | Communication device for information transfer from data processor to deaf and dumb people |
US5636038A (en) * | 1996-06-24 | 1997-06-03 | Lynt; Ingrid H. | Apparatus for converting visual images into tactile representations for use by a person who is visually impaired |
US5942970A (en) * | 1998-10-08 | 1999-08-24 | Norman; Jim | Image optical-to-tactile converter |
US20030151597A1 (en) * | 2002-01-03 | 2003-08-14 | Roberts John W. | Extended refreshable tactile graphic array for scanned tactile display |
GB2390733A (en) * | 2002-07-08 | 2004-01-14 | Baback Asady | Navigation and communication aid for the blind |
-
2011
- 2011-03-20 WO PCT/IB2011/051157 patent/WO2011117794A1/en active Application Filing
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE4337602A1 (en) * | 1993-11-01 | 1995-05-04 | Lehmann Hans Joachim | Dynamic orientation aid and video simulator for the blind |
DE19517649A1 (en) * | 1995-05-17 | 1996-11-21 | Thomas Rupp | Communication device for information transfer from data processor to deaf and dumb people |
US5636038A (en) * | 1996-06-24 | 1997-06-03 | Lynt; Ingrid H. | Apparatus for converting visual images into tactile representations for use by a person who is visually impaired |
US5942970A (en) * | 1998-10-08 | 1999-08-24 | Norman; Jim | Image optical-to-tactile converter |
US20030151597A1 (en) * | 2002-01-03 | 2003-08-14 | Roberts John W. | Extended refreshable tactile graphic array for scanned tactile display |
GB2390733A (en) * | 2002-07-08 | 2004-01-14 | Baback Asady | Navigation and communication aid for the blind |
Non-Patent Citations (2)
Title |
---|
TAKAFUMI ASAO ET AL: "A Study on Fundamental Information Transmission Characteristics of an Air-Jet Driven Tactile Display", 19 July 2009, HUMAN-COMPUTER INTERACTION. NOVEL INTERACTION METHODS AND TECHNIQUES, SPRINGER BERLIN HEIDELBERG, BERLIN, HEIDELBERG, PAGE(S) 397 - 406, ISBN: 978-3-642-02576-1, XP019122135 * |
VISELL ET AL: "Tactile sensory substitution: Models for enaction in HCI", INTERACTING WITH COMPUTERS, BUTTERWORTH-HEINEMANN, GB, vol. 21, no. 1-2, 1 January 2009 (2009-01-01), pages 38 - 53, XP025939693, ISSN: 0953-5438, [retrieved on 20080830], DOI: DOI:10.1016/J.INTCOM.2008.08.004 * |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2487672A (en) * | 2011-01-31 | 2012-08-01 | Univ Sheffield | Active sensory augmentation device |
WO2014027228A3 (en) * | 2012-08-16 | 2014-09-12 | Uab Gaminu | Apparatus for converting surroundings-related information into tactile depth map information |
FR2999741A1 (en) * | 2012-12-17 | 2014-06-20 | Centre Nat Rech Scient | HAPTIC SYSTEM FOR NON-CONTACT INTERACTING AT LEAST ONE PART OF THE BODY OF A USER WITH A VIRTUAL ENVIRONMENT |
WO2014095765A3 (en) * | 2012-12-17 | 2015-06-25 | Centre National De La Recherche Scientifique | Haptic system for establishing a contact free interaction between at least one part of a user's body and a virtual environment |
US9760166B2 (en) | 2012-12-17 | 2017-09-12 | Centre National De La Recherche Scientifique | Haptic system for establishing a contact free interaction between at least one part of a user's body and a virtual environment |
EP3413288A1 (en) * | 2017-06-09 | 2018-12-12 | Honda Research Institute Europe GmbH | Method for assisting a person in acting in a dynamic environment and corresponding system |
US10475348B2 (en) | 2017-06-09 | 2019-11-12 | Honda Research Institute Europe Gmbh | Method for assisting a person in acting in a dynamic environment and corresponding system |
WO2023019212A1 (en) * | 2021-08-12 | 2023-02-16 | Triton Systems, Inc | Arm-mounted hands-free haptic display |
US20230053132A1 (en) * | 2021-08-12 | 2023-02-16 | Triton Systems, Inc. | Arm-mounted hands-free haptic display |
CN114639205A (en) * | 2022-03-30 | 2022-06-17 | 中国工商银行股份有限公司 | Password input device and method based on air injection induction and self-service equipment |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3410264B1 (en) | Image display device and image display method | |
WO2011117794A1 (en) | Methods and devices for tactilely imparting information | |
US20170091535A1 (en) | Head-mounted display with facial expression detecting capability | |
US20170186236A1 (en) | Image display device, image display method, and computer program | |
JP5791082B2 (en) | Image composition apparatus, image composition system, image composition method and program | |
WO2017161726A1 (en) | Virtual reality device and virtual reality system | |
WO2017104320A1 (en) | Image display device | |
US11907423B2 (en) | Systems and methods for contextualized interactions with an environment | |
JPH10153946A (en) | Sensation information presenting device | |
CN110770628A (en) | Head-mounted display, display screen thereof, head-mounted support and video | |
TW202001787A (en) | Guided virtual reality system for relaxing body and mind | |
KR20180006573A (en) | The apparatus and method of forming a multi experience | |
CN104408702B (en) | A kind of image processing method and device | |
CN105630145A (en) | Virtual sense realization method and apparatus as well as glasses or helmet using same | |
US20190121515A1 (en) | Information processing device and information processing method | |
KR20170101703A (en) | System and apparatus for providing virtual experience | |
US20220100271A1 (en) | Systems, Methods, and Graphical User Interfaces for Updating Display of a Device Relative to a User's Body | |
CN106842565A (en) | A kind of wearable intelligent vision enhancing equipment of separate type | |
US9298010B2 (en) | Wearable optical display with audio functionality | |
CN207611201U (en) | A kind of VR helmets shell | |
US20200285310A1 (en) | Information processing apparatus, information processing method, and program | |
US20230306695A1 (en) | Devices, methods, and graphical user interfaces for three-dimensional user experience sessions in an extended reality environment | |
KR102561740B1 (en) | Eye movement device for enlarging the viewing angle and the method of eye movement using it | |
WO2023183340A1 (en) | Devices, methods, and graphical user interfaces for three-dimensional user experience sessions in an extended reality environment | |
JP2022108194A (en) | Image projection method, image projection device, unmanned aircraft and image projection program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 11717752 Country of ref document: EP Kind code of ref document: A1 |
|
WPC | Withdrawal of priority claims after completion of the technical preparations for international publication |
Ref document number: 61/315,955 Country of ref document: US Date of ref document: 20120830 Free format text: WITHDRAWN AFTER TECHNICAL PREPARATION FINISHED |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 11717752 Country of ref document: EP Kind code of ref document: A1 |