GB2445603A - Audio description of controls - Google Patents

Abstract

A method is described for audio-descriptions to enable the control of a programme to retrieve data in audio form without the need of a visual display. When a control is selected its function is described via audio together with any additional instructions required. E.g. if a function for selecting the voice to be used as audio output the audio output will state that different voices may be previewed by pressing that function and that to select a specific voice the "ENTER" button must be pressed.

Description

Methods to Add Audio Descriptions to Controls

Background

Written text is not readily accessible to numerous people. This is due to partial vision, blindness or reduced reading ability.

The spoken word is understood by a larger population because it is the most usual form of everyday communication; conversation, radio, television, speeches, lectures etc. Many articles and books are available in the spoken word form as audio tapes or CD's, MP3 files, ebooks etc for playing on domestic equipments, Daisy equipments, or computers.

However, it is necessary to have some training and expertise to use the interactive features.

In many cases it is necessary to be able to read a display, or to see icons on a screen, or to have a detailed knowledge of key functions. Pointer devices rely on the ability to co-ordinate pointer movement by seeing a small cursor on the screen and then to keep the pointer still whilst performing other actions; this is difficult for many disabled and elderly users.

The visual display or screen requires a high proportion of a unit's power requirement, particularly when back-lighting is required.

The interactive menus on the computer screen seldom produce any audio output.

Accessibility options are not present as a default, it is therefore difficult for a visually impaired person to use a computer. A problem with screen reader software is that the screen layout is designed for sighted persons who can scan the whole page in a second to decide which part they want to read in detail. If they do not want to read any part of the page then they can quickly go to another page; a blind reader on the other hand has to listen to the whole contents of a page before knowing which parts are of interest and then repeat the page again to re-access those sections. The invention relates to pages' and controls that are designed to be accessible without any vision.

Statement of Invention

The invention relates to methods by which a user can identify the functions of controls on an equipment ny means of audio prompts. It enables even those with serious vision impairment to operate the equipment. According to the invention the control will indicate its function to the user. In some cases, like a microwave cooker, or television set, the function will always be the same eg "set microwave mode", "timer is turned to iminute", "start', "set to channel 1". In a more interactive application the function of the control will be explained more fully according to the current condition of the controls: the microwave START/OK button may announce "start cooking, you have set microwave mode, 850watts power, cooking time 3minutes". Or, when applied to a television set, pressing the PRESET+ button may announce "Switching to next channel, number 10, UKHistory, Blue Planet 5:30 to 6:30, present time is 5:35 pm".

Another application of this invention relates to a method for enabling a user to navigate through an electronic text document on a computer, without the need of a display. In many cases just three pushbuttons and earphones will be all that are required for full interactive enjoyment and comprehension.

A data store contains digital coding of spoken texts and sounds, some relating to user instructions (control instruction data) and some to the text to be read, music to be performed, etc. (programme data). The equipment has operator controls, means for reading the data storage device, a microprocessor programme for converting each text to an audio output on earphones, loudspeaker etc. The microprocessor programme includes control of which data is to be output, according to which pushbuttons are pressed and to the interaction required by the actual data currently being processed. The invention relates to the methods by which the embedded user instructions enables the most inexperienced or disabled user to use the equipment.

The methods of the invention can be applied to television sets, video recorders, dvd equipments, mp3 equipments, ipods and their clones, computers, industrial equipment, CD equipments, record equipments, radio sets, washing machines, cookers, mobile phones, cameras, electric kettles, alarm clocks, electronic games, learning consoles, driving simulators, vehicle controls, telephones, alarm systems, storage systems, filing cabinets, door entry controls, musical instruments, readers of electronic books, optical scanners, toys, medical equipment, remote control units for any of the above, in fact any item of equipment having pushbutton or rotary controls Advantages Storage devices are output to audio earphones or a loudspeaker by means of an adapted equipment device. Audio instructions are given to the user, to tell the user which key to use in order to play the data in any required order. The method for interspersing control instructions in audio form enables the equipment to be operated without any need for a video display and the equipment requires a reduced number of operating keys. This enables equipments to be manufactured at a reduced price and size. Since there is a reduced power consumption equipments of this type can be run from batteries, or a hand generator, for long periods of time When used for a screenless computer with an operator wearing headphones there is less eyestrain and there is no screen showing data that can be seen by others When used with a screen the screen can be used to assist in teaching applications, where the student may see examples at the same time as appropriate texts or music are played audibly. A computer adapted for control using a small number of keys, and audible instructions at each point in a programme does not require a pointing device such as a mouse.

The invention also has many advantages for the vision impaired, and for companies and authorities required to submit data to them in audio form. Bank statements, utility bills, public announcements, calendars of events and advertisements would be much more accessible to the visually impaired if they could be navigated through instead of having to listen to the entire text. Websites, designed with the same method would enable the disabled person to use a computer on the internet. An introduction and setup programme such as that described below would enable the user to access the internet without having to see the computer screen at all

Introduction to drawings

Since the methods of the invention relate to audio output and not visible output, drawings are shown for a hypothetical screen, for a typical implementation using javascripts embedded in the Adobe Acrobat 7 programme in computer memory, optical disc, memory card etc. Balloon 1 represents the audio output when the equipment is started.

Balloon 2 represents the audio output when an Onward key is pressed.

Balloon 3 represents the audio output when an Action key is pressed.

The programme proceeds to other type 2 balloons each time the Onward key is pressed.

Other type 3 balloons are attached to each type 2 balloon and programme operation may proceed to another type 2 balloon, back to the attached type 2 balloon, or to any data content on the equipment, on an external memory, or on a networked device, including the internet.

Balloon 4 is the last of a section of programme and speaks the actions if the Action and Onward pushbuttons are pressed.

The arrow lines between type 2 balloons represent the action of each Onward keystroke. A Backward key moves programme execution in the opposite direction as shown by the dotted line arrows.

The Onward key uses the standard TAB key, the Action uses the ENTER key and Backward is achieved by pressing SHIFT and TAB.

Detailed Description

Most electronic equipment with an audio output can be adapted to use a method of embedded instructions. The data storage could be incorporated with existing devices. Thus, an audio disc could have each of its tracks prefixed with its track number and title and its first track could have its list of contents. The author has made such CD's That method enables a user to skip unwanted tracks and to play selected tracks. The disadvantage is that if there are a large number of tracks then the user has difficulty in remembering which number track he requires In the case of books with several hundred pages it is time consuming to navigate to particular pages The example now to be described is of an improved method uses a programme written for use with a computer The function of the programme is to acquaint the user with synthetic/synthesized voices, enable the user to choose the voice and to adjust the attributes, and then to go to an audio menu page for an audio English dictionary. Each item on the menu page can direct the programme to a new sub-menu page according to the first letter or sound of the first syllable of a word. The sub-menu page then breaks down to further menus according to the second letter or sound of a word, Further sub-menus would enable a breakdown to a dictionary page where each item speaks a word and its spelling with the option of hearing examples of the use of the word or links to other associated words.

An example dictionary in phonetic order has three pages', the voice setup page, part of the main menu page, a sub-menu page based on the or' sound as in the words or' or autumn', and a dictionary data page of words commencing with the sound or' Several words appear on the data page and the word ornate' is expanded to explain its meaning and usage.

Whereas the normal alphabetic order is abcdefghijklmnopqrstuvwxyz the phonetic order requires more sounds. for example the a may be pronounced in several ways, these can be denoted al a2 a3 a4 etc Some letters are not voiced in words. Thus a phonetic spelling of are' could be a4, the same as the English pronunciation of the capital letter R'. Phonetic ordering of words is then achieved by the alphabetic sorting methods using the order 123456789abcdefgh z. In order to sort words in rhyming order the symbols of the phonetic spelling are reversed in order, thus begin' could be represented by a phonetic spelling bilgilni having a reverse phonetic order nhilgilb. Standard sorting of words in reverse phonetic order will produce a rhyming order The programme is written as an Adobe Acrobat version 7 file and the embedded data is text which is read out using the computers speech synthesizer engine (typically text-to-speech' or Us engines running voices specifications SAPI4 or SAPI5 included with Microsoft Windows XP).

On opening the programme (it opens automatically on an autostart CD on a computer), the computer screen text indicates that you must listen for instructions and use the TAB, ENTER and SHIFT+TAB keys whilst a speech (1) gives the title of the programme, the name of the first page' and the instruction to press TAB.

The first press of TAB results in the current computer speech (2a) voice announcing its name, demonstrating alternative voices and then instructing the user to press TAB to change voice settings or ENTER to go to the Contents.

Pressing TAB activates speech 2b. The speech tells the user that different voices can be selected by pressing Enter a number of times Each time that ENTER is pressed the programme in 3b selects a different voice and speaks with that voice "you have selected speaker. If ENTER is pressed a sufficient number of times then any of the available voices can be chosen.

Pressing TAB takes programme execution to 2c and the speech instructs that the pitch of the voice can be altered by pressing ENTER. The programme 3c increases the pitch with each press and announces the newly set pitch. When the highest pitch has been reached the programme selects the lowest pitch so that any required pitch can be chosen by pressing the ENTER a number of times.

Pressing TAB takes programme execution to 2d and the speech instructs that the reading speed can be altered by pressing ENTER. The programme 3d increases the speed with each press and announces the newly set speed, reading at that speed. When the highest speed has been reached the programme selects the lowest speed so that any required speed can be chosen by pressing the ENTER a number of times.

If it is required to go back to a previous step then a third key can be used. In the example above the SHIFT key is pressed and then the TAB key. Other speech attributes such as volume could be adjusted in the same way by the addition of more steps 2e, 3e etc. A final TAB on the page (4) can announce that the setting up is complete and that the next page of the programme can be accessed by pressing TAB the speech (4) can also announce the effect of the ENTER, possibly to link to an Internet location, to start another programme, or to exit. When TAB is pressed the new page opens with its own introductory speech (1) so that the user knows exactly where he is in the programme The layout described can be duplicated on other pages of the document' When there are many type 2 balloons on a page its speech can include a reference for example paragraph 10 of 30, on page 136. In a straightforward novel most of the type 3 balloons will not be required Type 3 balloons can be used to enlarge on the speech of its adjacent type 2 balloon, or for directing the programme to a word-processor document, another programme on the computer, or to an email or Internet access programme and reference. Type 3 balloons may be used as a reference to the readers position in the document; for example the type 2 balloons may speak the words of each paragraph of a story whilst the associated type 3 balloon would speak the page number and paragraph number. Type 3 balloons can be used to demonstrate the same speech as in its associated Type 2 balloon except in a different language, voice or accent. A set-up page could include the selection of a different language for the document as a whole. A document using the methods described can be used as audio instructions for teaching how to use an item of equipment or a computer programme.

A document of the type described can be used to enable a user to fill in forms relating to questionaires, purchase orders etc, all without the need for a screen display.

A programme could be provided having the Type 2 balloons including a displayed text and the Type 3 balloons speaking the text out loud each time that the Action key is pressed. Such a programme would enable a user to practice reading with confirmation of the pronunciation instantly available. A document of this type could include user-entry for the user to enter text or speech.

Additional implementations According to the invention the operation of the equipment may include speech output explaining the operation of the equipment and of the options for playing the particular data programme. The control programme for this may be embedded in the equipment unit, or in the same memory device holding the data. It may even be present in both, enabling the memory device to override the normal functionality of the equipment when used with data programmes according to this invention. For example, the equipment may have particular text-to-speech voices whilst the data programme may have its own text-to-speech facilities The speech output may be entirely synthetic speech, or entirely human voice, or a combination of the two. Operating instructions can then be a different characteristic to the speech of the data programme. The data programme may have both synthetic speech and human voice speech -this is particularly important for dramatic effect when a particular actor can be chosen to speak important pieces of dialogue, or when a digital recording of applause, sound effects or background sound may be inserted or superimposed.

The speech output may have provision for the user to select different voices and attributes The voice used for speech output may be chosen at random by the equipment at particular points so that when a speech is played the voice may change. Thus, in the case of a dictionary or phrase book, the speech attributes could be changed each time it is accessed, enabling the user to become acquainted with male and female voices, English and American accents etc. In the case of a stereo equipment the left and right tracks can be used for playing the instruction speech and programme data (speech or music) separately.

The instruction speeches may include several different languages The programme data speeches may include several different languages The speech output may include several different dialects or accents, for example, a dictionary, tourist guide or phrase book could be played to output speech in different regional accents.

In the case where the data programme is in the form of spoken text using synthetic speech, the data programme may include instructions so that different voices and attributes may be applied during the playing, such as in the case of a dialogue where different characters may be given different voices -especially important in the case of an audio performance of a play such as Romeo and Juliet When the method of the invention is used in relation to a music equipment, or a reference with many audio items to be accessed, means can be provided for the user to choose the navigation sorting method In the case of music the navigation could include contents listed according to genre and sub-listed in alphabetic order of title, of artists name, of release date etc operating in the manner of an audio output to a spreadsheet/database programme The invention may be used to play long texts or music pieces or many texts or many music pieces, in which case use of the single-step ONWARD and BACKWARD pushbuttons would be laborious. The programme can be arranged such that multiple steps (such as skipping pages of text or tracks/sections of music) are performed if the pushbuttons are held operated for certain times; for example less than I second to step once, from ito 3 seconds to step 10 and for over 3 seconds to step 100: these functions being spoken as part of the introductory speech step when the equipment is first started The ONWARD operation would then stop at the last step of the programme (last piece of music/last page of text), and the BACKWARD operation would stop at the first/start up state of the programme.

A PAUSE/RESUME function may be implemented with an extra pushbutton or by the simultaneous pressing of two pushbuttons (preferably the ONWARD and BACKWARD). The pressing of the buttons would have to take place within, say, I second of each other.

Preferably the unit would speak confirmation of the action (for example "equipment is now paused" or "Play has now been resumed") The pushbuttons of the equipment may be difficult to identify for the vision impaired user.

Each pushbutton may be provided with means for outputting an audio message to the user relating to its use in the particular programme application. A first method is for the equipment to speak the instructions for use of the pushbuttons on first switching the equipment on. A second method is for the pushbuttons to speak their purpose if the pushbutton is pressed for a short time only, the speech commencing on the early release of the button and terminating either at the end of the speech or when any pushbutton is pressed. A third method is for the pushbuttons to be replaced by three position switches where the positions relate to off, speak switch function and operate switch function. After learning the operation of the equipment the user may no longer need the audio prompts; in this case a switch could be provided to switch the prompts off, leaving the normal programme audio only Another means for making control prompts optional is to have the prompts only activated after the control has been released, having been pressed for a short time only whereas the actual control function is activated when the control is pressed for a longer time before being released; thus enabling the user to test' the function of a control at any time, prior to activating it, or to activate the control without using the control prompt.

The pushbutton functions may be performed by any physical switching device including toggle switches, membrane switches, proximity switches, multi-position switches, magnetically operated switches, capacitance, inductance or resistance sensing switches or transducers, movement sensors, light operated switches, radiation operated switches and any switching method in current or future use.

The pushbuttons could be replaced by light sensors which would operate when lit by a directional laser beam from a unit that could be mounted on the head of a disabled person for

example

The pushbutton functions could be replaced by a microphone receiving sound signals by the human voice or other source, the signal then being processed to determine and execute the intended action.

In a multi-listener situation the audio data programme output may be routed through a loop system as used for the hard of hearing or by low-energy radio signals, while the operator hears both the control speeches and the data speeches. Such applications could arise where a single equipment unit is operated by a teacher to choose exercises for a class of students or for a guide to select speeches explaining features of interest to a group of tourists.

In the case of a vehicle control system, or simulator, the addition of a speech output would be particularly useful, both for confirmation to the driver and for reference by an instructor or voice recorder. For example if the left rudder bar was pressed the audio output could speak "Left rudder 4" and then, when the periodic response of the vehicle was received, it could include "heading 125, turning to port, rate of turn 10 degrees per minute' for example. The same principle applies to radio-controlled systems for land vehicles, aircraft, water-craft and spacecraft and to control of cameras, guns and missile launchers.

Since each control can speak its function a written user manual should be unnecessary. In order for the user to practice operation of the equipment a specific user manual programme could be provided in which the actual control steps are made without actually operating any physical device. Preferably the word simulation' would be inserted in each speech so that the user is reminded that the programme is just a practice for operating the microwave cooker, washing machine, etc.

Claims (18)

1. Claims 1. A method for setting the speech output of a computer

   programme whereby the operator receives audio instructions as to which pushbuttons should be pressed in order to select the voice settings to be used by the text-to-speech engine and receives audio messages demonstrating the settings as each adjustment is made.
2. 2. A method for enabling the functions of each control of an equipment to be output in an audio speech form.
3. 3. A method by which a user manual can be provided for an item of equipment in audio form, for output by the equipment itself in a simulation mode.
4. 4. A method by which a visual indicator or display on an item of equipment may be substituted for a unit having audio output.
5. A method by which audio output may be paused at any time and resumed later by pressing two pushbuttons simultaneously in each case.
6. 6 A method for finding data in an equipment whereby the operator receives instructions as to which pushbuttons should be pressed in order to access a particular piece of data for audible output and how to then select adjacent data or go back to an audio list of contents. As each piece of data is selected an audio response provides a data position reference.
7. 7 A method for finding words in an audio dictionary whereby the operator receives an audio explanation of the action of each control pushbutton by means of three pushbuttons. The order for finding and listing adjacent words being in normal alphabetic order.
8. 8. A method for finding words in an audio dictionary whereby the operator receives an audio explanation of the action of each control pushbutton by means of three pushbuttons. The order for finding and listing adjacent words being in a phonetic alphabetic order
9. 9. A method for finding words in an audio dictionary whereby the operator receives an audio explanation of the action of each control pushbutton by means of three pushbuttons. The order for finding and listing adjacent words being in a phonetic rhyming alphabetic order
10. 10. A method by which the output language of any of the above claims may be chosen from a number of different languages.
11. 11. A method by which any of the above audio outputs in one language can be repeated in another language.
12. 12. Any of the above claims where the pushbutton is replaced by any physical switching device including toggle switches, membrane switches, proximity switches, movement sensors, light operated switches, radiation operated switches and any switching method yet to be discovered
13. 13. Any of the above claims where the pushbutton functions are replaced by devices operated by sound or by the human voice.

    14. Any of the above claims where the equipment is used as an aid to learning a language 15. Any of the above claims where the equipment is to be used for reading text out loud.
14. 14. Any of the above claims where the pushbutton functions are performed by a remote controller.
15. 15. Any of the above claims where the audio output is broadcast by means of radio waves to remote listeners with radio receivers.
16. 16. Any of the above claims where the audio output is broadcast by means of a loop system to listeners with adapted heanng aids or headsets.
17. 17. A Radio Control unit whereby the control instructions implemented by the operator are output as speech and/or control signals.
18. 18. Any vehicle controls whereby the control actions are confirmed by speech output.

    19 Any of the above claims having a mobile telephone as the equipment unit