GB2574630A - Reading aid - Google Patents

Abstract

A reading aid 1 is disclosed that fits on the finger 14 of a user and converts a tracked line of text into Braille symbols. The apparatus comprises a thimble-shaped housing 2 that includes a camera 20 disposed on an underside of the housing for capturing text or images, and a Braille display mechanism 46 disposed on an inner surface of the housing to be in contact with the user’s finger. A converter unit 30, converts text/image data into instructions enabling the display mechanism to display appropriate Braille symbols. The converter may be located within, or remote from, the finger mounted apparatus and may share a wired or wireless communication interface between the camera and display mechanism. The apparatus may also include a means for converting text data into an audio presentation. In a further embodiment, the converter unit may be run on a mobile device, such as a smart phone.

Description

The present invention relates to a reading aid, particularly for use by the blind and partially sighted. The invention provides a device that converts text and/or images, generally printed material, or material on a screen, into a form more accessible to those with sign impairment, in particular in to Braille or audio output.

Background to the Invention

There are 285 million blind and partially slighted people worldwide and this is set to double by 2050. 90% of blind people are unemployed leading to isolation and poverty.

Braille is a tactile language or writing system, used by people who are blind or visually impaired. Symbols in the form of raised dots are embossed onto paper, which can then be felt sensed by the reader.

Each pattern of dots represents a character, or a series of patterns, typically three or four, represents a word. The dots will be provided on a grid system; current grids being 2x3, i.e. two columns of three rows. For example, the letter a can be represented by a raised dot on the first row, first column of the grid, i.e.; b is represented by raised dots on the first and second row of the first column; and c is represented by raised dots on the first and second column, on the first row. The word “house” may be written in full as > λ’* , or in contracted form, namely “h|se”, as

While books and publications can be produced in Braille, less than 5% of titles are available in this format. Braille books are also very significantly larger than standard type books. This limits the availability of books to the partially sighted and blind population.

While Braille can be used in some public areas, for example on panels in lifts, or on some packaging and devices, much of the written work is unavailable to the blind and partially sighted population. For example, restaurant menus rarely include Braille.

-2Computers have been developed for use by the blind and partially sighted in which text is converted into Braille. Such devices typically use raisable pins to provide a line of a series of Braille characters next to each other, which updates once acknowledged as read by the user. Such devices are either specially designed and built computers or are adapted computers with additional attachments. As such these devices are large and not transportable, and are limited to reading text from a computer.

Devices are also available to allow a blind or partially sighted person to type text and read back what they have written on a line of Braille. Again using the raisable pins system, the text displayed in Braille changes as the user continues to type. Typically the typing will not be onto a qwerty keyboard but a series of finger pads relating to the pattern of dots in the Braille system. Again these devices do not enable a user to read printed text.

Computer programs exist that convert text into Braille. For example Brailletranslator.org converts text into a series of dots. However, these are not embossed and thus cannot be read by a blind or partially sighted person. In addition, optical character recognition programs are available to covert printed text, into editable text. It is understood that MIT have been developing a “cheap handheld device for translating Braille instantly”, see https://www.diqitahrends.com/cGol·· tech/taotite-mit--BraHte-transiatQr/ This device comprises a box that can be placed over a paragraph of text and contains a scanning mechanism to capture the image of the text and a computer program to recognize the text and translate it into Braille. The top of the box is provided with a series of blocks of pins which can be raised for form a long row of Braille symbols. It appears that the intention is to create a page of Braille appear above a page of text.

This device is also a large piece of equipment and thus not truly portable.

An aim of the present invention is therefore to provide a portable device for quickly and easily translating any text into Braille characters, including text on a page and text on a screen. Specifically, an aim is to provide a wearable device for reading text and providing Braille characters to a user.

Summary of the Invention

-3According to the invention there is provided apparatus to be worn on a user’s finger, comprising • a sensor unit that reads text and/or images and outputs text and/or image data;

• a converter unit that converts the text and/or image data into a tactile presentation of the text and/or images; and • a display unit that displays the tactile presentation to the user’s finger.

In use, a blind or partially sighted person, or someone wishing to learn Braille or the like, places the device on their finger, typically their forefinger, and runs it slowly over a line of text. As the device passes over the text, the sensor reads the text, sending image data to the converter unit. The converter unit converts this data to instructions for creating a textured surface, generally Braille. These instructions are followed by the display, which creates a tactile presentation, generally in the form of a Braille symbol or a combination of such symbols, under the pad of the user’s finger. The user is able to detect and read the symbol(s). As the device reads each letter or group of letters it presents a new symbol or symbols to the user. The faster the user runs the device over the text, the faster the user will be presented with symbols.

Typically the sensor unit comprises an image capturing device, preferably a camera, mounted on an exterior surface of the apparatus. Preferably the senor is adapted to read text that has been printed or electronically displayed.

The sensor may comprise a transmitter for wireless transmission of the output data to the converter unit. Additionally or alternatively, the sensor may comprise a cable for transmission of the output data by direct connection to the converter unit.

Usually the converter comprises a corresponding receiver for receiving data transmitted from the sensor wirelessly. Additionally or alternatively, the converter may comprise a receiver for receiving data transmitted from the sensor via cable.

Preferably, the converter converts the text and/or image data into a code that corresponds to a tactile presentation of text and/or images. Advantageously, the converter unit may also convert the text and/or image data into an audio presentation of the text and/or images. This enables the user to hear the converted text and/or image information additionally or alternatively to sensing the tactile presentation.

-4This is particularly advantageous for users who are not familiar with tactile presentation of information or who wish to learn this format.

Preferably the tactile presentation will be Braille. This may be Braille in Grade 1 format, where symbols representing individual letters and numbers are presented, or Grade 2 where symbols representing compressed text, are presented. Alternatively the tactile presentation may be another tactile presentation that is known to and understood by users.

Where the presentation is Braille, the Braille may be in the language of the text. This may be any language that is appropriate to the user. Alternatively, the converter may also be adapted to translate the text into a different language e.g. reading the text in one language and presenting Braille via another. Braille is different in every

Commonly, a given text word, say “house”, is presented in tactile manner as a series of symbols. Prior art devices have multiple symbol presentation areas in a line next to each other and present the series of symbols adjacent each other in a linear array and the user reading the symbols rums a finger or fingers over these adjacent symbols. The symbols may remain presented via pins or other projections in a line until dismissed as read. Another series of symbols is presented in a line for the next word or phrase.

Preferred apparatus of the invention are for attachment to or wearing on just one finger and more preferably provide a single symbol presentation area/platform to that one finger. Preferably, in the invention, the apparatus provides an output comprising a sequence of tactile presentations spaced apart temporally. Thus the user detects a sequence of symbols, presented one after the other, at the same location, meaning at the same symbol presentation area/platform. The symbols are spaced apart in time. Usually they are separated by a period during which no symbol is presented. The duration between symbols can vary e.g. corresponding to the speed at which the user moves the device over the text and/or images. The duration may be predetermined, to provide a reasonable pause between symbols. The duration may also be programmable by the user according to reading speed or other preference

-5Accordingly, the display unit is preferably adapted to read the code provided by the converter unit and display the tactile presentation in the same place with symbols separated in time. The display unit may include a receiver to receive the code from the converter unit wirelessly. Additionally or alternatively, the display unit may include a receiver to receive the code from the converter via cable.

Typically, the display unit is adapted to form a variable and uneven surface for presentation to the user’s finger. Preferably, the uneven surface is created by projections in grid pattern. This is suitable for forming Braille symbols.

The Braille symbols may be formed in a number of different ways. In one preferred option, the projections may be raisable pins. Various means for raising and lowering the pins are known. For example, the pins may be raised using the piezo-electric effect. Alternatively the pins may be raised and lowered using a nickel-titanium alloy that is known to expand or contract which an electric current is applied. Another means for raising and lowing the pins include bubble cells or micro-fluids. Further alternatives include actuators, including nano-actuators, solenoids and heat actuators. In a further alternative, the projections may be inflatable bladders. Generally, the manner of forming the Braille symbols is not an element of the invention and many variants for achieving this are suitable.

Preferably the projections are in the form of a 2 x 3 grid as it is the more traditional form of arid and is used in most printed braille documents. This makes it easier for the user when compared to 2 X 4. Alternatively, other grid sizes may be used, for example 2x4 which is more recent development just for tactile displays and dgital use. or other dimensions. British standard Braille currently uses 2x3 grids in printed text , while 2x4 grids are used in digital tactile devices. Preferably, the projections may be raised and lowered by 0.4 - 1mm. While a projection having a height of 0 4 mm can be detected by a trained user, typical Braille projections are 0.5 - 0.8mm high. Thus more preferably, the projections may be raised and lowered by 0.5 0.8mm. Typically the projections have a diameter of 1.0 - 1,5mm. Most preferably the projections have a diameter of 1,2mm as this is a common standard and used as e.g. the British standard. The projections in a grid may have a spacing of 2 - 3 mm, preferably 2.5mm. This a grid of 2 x 3 pins, measured from the outside edge of one pin to the outside edge of the other pin, may be 3.7mm x 6.2mm. This is suitable for detection by the pad of a finger of a user. The braille cell in the device will be 2 X 3

-6Preferably the apparatus may comprise a housing on the outside of which is mounted the sensor unit, and on the inside of which is mounted the display unit. The converter unit may comprise software adapted to be run on a CPU held on the housing. This has the advantage of the provision of a single, stand alone, unit which carries out the sensing, conversion and presentation. However, a CPU on which software runs would currently add significant weight and bulk to the unit. As miniaturization of CPUs continues this option will become more preferable. In accordance with a preferred option, the converter unit may be provided on a separate device. Preferably, the converter unit comprises software adapted to be run on a mobile device, in the form of an app. As a significant proportion of the world’s population now owns or has access to a smart phone tablet or similar device, and this proportion is increasing, such devices can be used to carry out the conversion. Such devices are adapted to communication with a separate device, namely the apparatus of the invention, wirelessly or by a wired connection. Alternatively, the converter unit may be software adapted to be run on a separate stand-alone unit, connected to the apparatus wirelessly or by wired connection. This option is advantageous for user’s who do not possess or have access to a mobile device, or for user’s who wish to keep these items separate.

Standard software is available for converting analogue text (e.g. on a page or screen) to digital format, and this is generally known as optical character recognition (OCR software). The software of the conversion unit may include standard OCR software. Software is also known for converting a digital signal to Braille symbols, such as Arduino software. The software in the conversion unit may also include such software.

A user may user more than one apparatus, worn on different fingers.

Preferably the reading aid comprises:

a housing sized to fit over the end of a finger, the housing comprising;

an image capturing device mounted to capture images as the housing is moved over text and/or images;

-7an interior surface on which a pad of a finger rests in use, the surface adapted to form a sequence of individual textured symbols; and means for forming the symbols;

means for converting the captured images to instructions for forming the symbols.

According to a further aspect of the invention there is provided a method of converting text to a sequence of individual textured symbols comprising:providing an image capturing device adapted to be held beneath a finger of a user and to capture the image or text traversed by the finger;

converting the captured image to signals representing a sequence of textured symbols; and forming the textured symbols for detection by the finger of the user.

According to another aspect of the invention there is provided a method of converting text to Braille, comprising reading the text and presentation the text as a series of Braille symbols at the same location spaced temporally.

The user experiences a series of Braille symbols presented to him that are spaced apart by time, and separated by a period during which no symbol is presented.

Preferably the method includes the user moving an image capturing device over text, while being presented with the sequence of Braille symbols. As such the user’s finger does not move relative to the presentation of the symbols, but the symbols are formed and removed as the text is read and presented to the user.

To help understanding of the invention, specific embodiments thereof will now be described by way of example and with reference to the accompanying drawings, in which:

Figure 1 is a view from one end of a thimble according to a first embodiment of the invention;

Figure 2 is a cross-sectional view on line AA of the thimble of Figure 1;

Figure 3 is a top view of a grid of pins in the thimble of Figure 1;

Figure 4 is a perspective view showings some of the pins raised to form a Braille symbol;

-8Figure 5 is a perspective view of a second embodiment of the invention, including a thimble and a mobile phone on which the software of the converter unit is installed; and

Figure 6 is a perspective view of a third embodiment of the invention, including a thimble, which is connected by a wire to the separate converter unit, which is held on the wrist of a user.

Examples

Referring to Figures 1 and 2, apparatus 1 of a first embodiment of the invention is provided in the form of a thimble 2 sized to fit the finger 4 of a user. The thimble is injection molded from acrylonitrile butadiene styrene. However, the thimble may be made from other materials, including other plastics, for example polyethylene or polyethylene terephthalate, rubber, metal, ceramic, composite materials or even wood. However, generally it will be made from plastic material as this is easy to manufacture and will produce a light weight but durable structure.

The thimble has a body 8 which surrounds the circumference of the user’s finger such that the user can readily move the thimble as they move their finger. As shown the thimble is open at a front end 10, to allow a tip 14 of the finger 4 to protrude. In some designs (not shown) the thimble may be closed at its front end, enclosing the end of the finger. The thimble also includes a battery 18 to provide power to the device. This can be positioned anywhere on the device, but will typically be provided on the top 16 of the device for balance.

Attached to an underside 6 of the thimble is provided a digital camera 20, positioned to capture an image of text and/or images beneath the thimble. When passing over text or images, the camera 20 captures the images and provides and output of the image data. The digital camera can be of any type that is able to read text and/or an image positioned directly beneath it, and is sized to fit under a thimble that fits on a finger. Typically the camera will read an image of a word or a group of letters and form image data.

The image data is processed by a converter unit 30, as shown mounted on the top 16 of the thimble. The converter unit 30 comprises a CPU 32 containing software designed to convert the image data to instructions to form Braille symbols.

-9The software may include Processing software that converts the image data provided by the camera to a computer language to be understood by other programs. The CPU may include an Arduino uno microchip. Arduino software may also be used to convert the information provided by the Processing software to create instructions to produce a sequence of Braille symbols. The Arduino software uses the information provided by the Processing software to form a series of binary on and off (0 and 1) for display to the user. Alternatively, the Arduino software can also be used as a stand-alone software that is directly connected to the camera sensor. This way the sensor will directly send the information to the microchip which will process the information and translate it accordingly.

The software may also convert the image data to audio instructions, that when connected to a speaker or headphones, creates an audio signal which the user can hear. The thimble 2 also includes an audio socket 19 for connection of a speaker or headphones. This is particularly advantageous for users who cannot read Braille or who are learning Braille.

The instructions created by the software are transmitted to a display 40, positioned on an inside 18 of the thimble, directly beneath and in contact with the pad of the user’s finger 4 in use. The display consists of a grid of pins 42 mounted in a surface 44. When the pins are not extended, they are flush with the surface 44. As shown the pins 42 form a 2 x 3 grid. This enables British Standard Braille symbols to be formed. However, other sizes of grid can be used, for example 2x4, which enables old Braille symbols to be formed, or even other sizes of grid can be formed for other forms of Braille or the equivalent.

The display also includes a mechanism 46 for raising and lowering the pins 42 in response to the instructions provided by the converter unit 30. As show the pins are raised and lowered using a piezoelectric mechanism, whereby electric signals stimulate piezoelectric bimorphs to bend up or down, raising and lowering the pins.

A piezoelectric unit, such as is available from Metec Consulting Engineers (http://www.metec.ie/), is attached to each pins. The pins are of the British standard size to allow ease of readability via touch for majority of users. Thus the pins have a diameter of 1.5mm, and a 2.5mm centre to centre spacing. Other sixes of pins and spacing can be used, but typically this will be a standard arrangement. Other system

- 10that can be used to raise and collapse pins are wires, solenoids, squiggly motors and wax motor.

Figure 3 shows the grid of pins 42 in the surface 44 that can be raised and lowered to form Braille symbols and Figure 4 shown a Braille symbol created from the pins.

In use, the users places the device over their finger and places their finger on a piece of text. The camera 20 captures an image of one or more symbols of the text and sends image data relating that that one or more symbols to the converter unit 30. Software in the converter unit converts the image data into instructions to form one of more Braille symbols relating to the original text. These instructions are transmitted to the display unit 40, the mechanism 46 of which raises the appropriate pins to form the first Braille symbol, lowers the pins and then raises the appropriate pins for form a subsequent Braille symbol, if more than one is required.

This process occurs almost instantaneously, such that when the image data has been captured the user is presented with one or more Braille symbols.

As the user moves his or her finger along a line of text, the camera 20 captures a series of images of the text, all of which are processed by the converter unit 30. The converter unit sends a series of instructions to the display unit 40 to produce a series of Braille images, one after another, by raising and lowering the appropriate pins. The faster the user moves his or her finger across the text, the faster the series of Braille symbols will be presented.

As the device is sized to fit on the finger of a user, it is portable and can travel with the user. The camera 20 is capable to capturing images of text that has been printed, or text that is on a screen, for example on a computer. All image data can be processed by the converter unit 30 to a series of instructions to the display unit 40 to create a series of Braille symbols that a presented one after another to the finger of the user.

Turning now to Figure 5, the device thereshown is very similar to the device of Figures 1 - 4. However, whereas in the first embodiment, the converter unit was held on the thimble, in this second embodiment, the converter unit is provided in a separate device.

-11 The thimble 101 is provided with a camera 120 and a display unit 140 in a similar way to the first embodiment. However, the converter unit 130 is provided in a separate device 132. As shown the separate device is a mobile phone 134, which will typically be owned by the user. Software in the form of an app can be placed on the phone to carry out the conversion process. The app will be available in Windows, IOS and Android formats and acts as a library that the user can store documents to read on as well as save previously read documents. The application being available in all three platforms means that it is not only usable on a mobile phone but also on tablets and laptops. When connected to the app, the user also has an option to have audio feedback on the test the thimble is sliding over. The user can connect a speaker or headphones to the phone 134, in which a socket is provided 136, tablet or laptop to receive the audio signal.

The thimble 101 is provided with a transmitter/receiver 150 to wirelessly transmit image data provided by the camera 120 to the phone 134, and receive instructions for the display unit 140 from the phone.

The thimble 101 is also provided with a socket 152 for attachment of a cable 154, which also connects to the phone 134, which is provided with an appropriate socket 138 (for example as usually used for charging). This also enables the image data to be transmitted to the phone or mobile device, and instructions to the display unit from the software, to be received by the display unit. The cable will be directly connected to the PCB of the transmitter and receiver (thimble). The wireless version will have usb type c female sockets on both, the transmitter and the thimble. This will allow the user an option to connect the two devices via cable as well as charge the device.

While the wireless transmitter/receiver and cable communication are alternatives and would not be used together, they may both be provided on the thimble to provide options for the user. Alternatively, in other embodiments, one or the other may be provided.

Where no wireless transmitter/receiver is provided, the thimble 101 may not include a battery 119, as power may be provided by the phone or mobile device to which it is attached by wired connection.

- 12 Other aspects of the device, including the functioning of the software, are as discussed in respect of the first embodiment.

Now turning to Figure 6, which shows a third embodiment according to the invention. This embodiments includes a thimble 201, which is essentially that described with respect to the second embodiment.

In this embodiment, the converter unit 230 is provided on a separate stand-alone device. The unit 230 is provided with a transmitter/receiver 232 for wireless reception and transmission of data from and instructions to the thimble 201. The unit 230 also includes a socket 234 for connection by a wire 236. In other embodiments, not shown, the unit may be adapted for either wireless connection or wired connection to the thimble, but not necessarily both. In this embodiment, the unit 230 is shown attached to the thimble by a wire.

The unit 230 may also be worn on the wrist of the user, for convenience, using a strap 238. Alternatively, the user may place the device in his or her pocket (not shown). Thus unit may also include a socket 219 for the attachment of headphones or a speaker where the converter unit is able to convert the image data to an audio signal in addition to the instructions to the display unit.

As in the second embodiment, the thimble 201 includes a transmitter/receiver 250 to wirelessly transmit image data provided by the camera 220 to the converter unit 230, and receive instructions for the display unit 240 from the converter unit 230.

The thimble 201 is also provided with a socket 252 for attachment of a cable 236, which also connects to the converter unit 230 as described.

As shown the thimble 201 also includes a battery unit 218. However, as the converter unit 230 also includes a battery 239, if the thimble will be permanently connected to the converter unit 230, the thimble may not incorporate a battery 218.

The invention is not intended to be restricted to the details of the above-described embodiments. For example the apparatus may be in the form of a computer-style mouse, with the sensor unit on an underside of the mouse (as in the tracking unit if current mice), the display unit on the top of the mouse, and the processing unit and

-13battery being provided within the mouse. The processing unit for such a mouse could alternatively be provided as an app for a mobile device, such as a phone, tablet or laptop, connected wireless or by wired connection.

Claims (25)

1. Apparatus to be worn on a user’s finger, comprising • a sensor unit that reads text and/or images and outputs text and/or image data;

• a converter unit that converts the text and/or image data into a tactile presentation of the text and/or images; and • a display unit that displays the tactile presentation to the user’s finger.

2. Apparatus according to claim 1, wherein the sensor unit comprises a camera mounted on an exterior surface of the apparatus.

3. Apparatus according to claim 1 or claim 2, wherein the sensor comprises a transmitter for wireless transmission of the output data to the converter unit.

4. Apparatus according to any preceding claim, wherein the sensor comprises a cable for transmission of the output data by direct connection to the converter unit.

5. Apparatus according to any preceding claim wherein the sensor is adapted to read text that has been printed or electronically displayed.

6. Apparatus according to any preceding claim as dependent on claim 3, wherein the converter unit comprises a receiver for receiving data transmitted from the sensor wirelessly or via cable.

7. Apparatus according to any preceding claim, wherein the converter unit convers the text and/or image data into a code that corresponds to a tactile presentation of text and/or images

8. Apparatus according to any preceding claim, wherein the converter unit additionally converts the text and/or image data into an audio presentation of the text and/or images.

9. Apparatus according to any preceding claim, wherein the tactile presentation is Braille.

10. Apparatus according to any preceding claim, wherein the tactile presentation comprises a sequence of tactile presentations spaced apart temporally.

11. Apparatus according to any preceding claim, wherein the tactile presentation comprises a sequence of tactile presentations presented in series at the same location.

12. Apparatus according to any preceding claim, wherein the tactile presentation comprises a sequence of tactile presentations separated by a period during which no symbol is presented.

13. Apparatus according to claim 7 or any one of claims 8 - 12 as dependent on claim 8, wherein the display unit is adapted to read the code provided by the converter unit and display the tactile presentation.

14. Apparatus according to claim 7 or any one of claims 8 - 13 as dependent on claim 8, wherein the display unit includes a receiver to receive the code from the converter unit wirelessly or via cable.

15. Apparatus according to any preceding claim, wherein the display unit is adapted to form an uneven surface for presentation to the user’s finger.

16. Apparatus according to any preceding claim as dependent on claim 9, wherein the uneven surface is created by projections in grid pattern.

17. Apparatus according to claim 16, wherein the projections are raisable pins.

18. Apparatus according to any one of claims 16 or 1, wherein the grid is a 2 x 3 grid.

19. Apparatus according to any preceding claim, wherein the apparatus comprises a housing on the outside of which is mounted the sensor unit, and on the inside of which is mounted the display unit.

20. Apparatus according to claim 19, wherein the converter unit is provided on the housing.

21. Apparatus according to claim 19, wherein the converter unit is provided on a separate device.

22. Apparatus according to any preceding claim, wherein the converter unit comprises software adapted to be run on a mobile device.

23.Apparatus according to any preceding claim, wherein the converter unit comprises software adapted to be run on a CPU held on the housing.

5

24. A reading aid comprising:

a housing sized to fit over the end of a finger, the housing comprising; an image capturing device mounted to capture images as the housing is moved over text and/or images;

an interior surface on which a pad of a finger rests in use, the surface 10 adapted to form a sequence of individual textured symbols; and means for forming the symbols;

means for converting the captured images to instructions for forming the symbols.

15

25.A method of converting text to Braille, comprising reading the text and presentation the text as a series of Braille symbols at the same location spaced temporally.