WO2011011225A1

WO2011011225A1 - Methods for enhancing the communication rate of a speech generation device

Info

Publication number: WO2011011225A1
Application number: PCT/US2010/041807
Authority: WO
Inventors: Bob Cunningham; Linnea Mcafoose
Original assignee: Dynavox Systems, Llc
Priority date: 2009-07-24
Filing date: 2010-07-13
Publication date: 2011-01-27

Abstract

A method for enhancing the communication rate of a user of a speech generation device is disclosed. The method generally includes electronically creating a textual phrase or sentence including a slot, displaying the textual phrase or sentence on a display device of the speech generation device, receiving an input selecting the slot and electronically inserting a slot filler into the slot.

Description

TITLE OF THE INVENTION

METHODS FOR ENHANCING THE COMMUNICATION RATE OF A SPEECH

GENERATION DEVICE PRIORITY CLAIM

[0001] This application claims the benefit of previously filed U.S. Provisional Patent Application entitled "HAND-HELD SPEECH GENERATION DEVICE," assigned USSN 61/228,256, filed July 24, 2009, and which is fully incorporated herein by reference for all purposes,

BACKGROUND OF THE INVENTION

[0002] The present invention generally pertains to speech generation devices (SGDs), and more particularly to methods for enhancing the communication rate of an SGD.

[0003] Various debilitating physical conditions, whether resulting from disease or injuries, can deprive the afflicted person of the ability to communicate audibly with persons or devices in one's environment in real time. Injuries suffered during armed combat, whether by domestic police officers or by soldiers engaged in battle zones in foreign theaters, are swelling the population of potential users. Persons lacking the ability to communicate audibly can compensate for this deficiency by the use of speech generation devices. Some conventional speech generation devices are desktop devices, while others are rendered portable by being mounted on vehicles such as wheel chairs, and still other conventional portable speech generation devices can be carried by user as one would carry a brief case.

[0004] It is estimated that less than 10% of the potential users of speech generation devices currently is being served by conventional speech generation devices. This population is highly variable from the standpoint of a range in ages from preschool children through elderly adults and a variety of lifestyles, geographic locations, educational attainments, language sophistication, and available physical motor skills to operate the speech generation device.

[0005] Communicating with others using a conventional speech generation device can often be frustrating for a user due to the significant amount of time required to compose a message. In particular, many SGDs require that a user manually enter or type an entire message before it can be communicated through the SGD. Other SGDs may include pre-stored phrases and/or sentences that may be selected by the user and communicated through the SGD. However, the scope of the user's communication is limited to the number of phrases and/or sentences stored on the SGD. In other words, for the user to communicate a message that is not pre-stored on the SGD, the user must typically create the message one word at a time, which wastes the user's vaulable time and energy. Accordingly, a need exists for a method that enables a user to quicky compose dynamic messages within an SGD.

[0006] In light of the various design concerns in the field of speech generation devices, a need continues to exist for refinements and improvements to address such concerns. While various implementations of speech generation devices and associated features have been developed, no design has emerged that is known to generally encompass all of the desired characteristics hereafter presented in accordance with aspects of the subject technology.

BRIEF SUMMARY OF THE INVENTION

[0007] In general, the present subject matter is directed to an improved speech generation device (SGD) having electronic components that enable the SGD to transmit and receive messages to assist a user in communicating with others. For example, the SGD may correspond to a particular special-purpose electronic device that permits a user to communicate with others by producing digitized or synthesized speech based on configured messages. Such messages may be preconfigured and/or selected and/or composed by a user within a message window provided as part of the speech generation device user interface. A variety of physical input devices and software interface features may be provided to facilitate the capture of user input to define what information should be displayed in a message window and ultimately communicated to others as spoken output or other outgoing communication.

[0008] Features that may be included in exemplary SGD embodiments of the present subject matter concern computer readable medium that, when executed by a processor, instruct the SGD to perform specific functionality. For example, exemplary SGD embodiments may include executable instructions stored on a computer-readable medium that implement a unique method of communication rate enhancement that combines sentences and phrases with slot fillers. In general, implementation of a "slots" feature triggers software provision of mechanisms to insert a "slot" or placeholder in a textual phrase or sentence. Each one of these slots or placeholders is associated with a list of "slot fillers" that can potentially fill in this place in the text. In one embodiment the list of slots and their associated fillers are stored in a database associated with other computer readable medium storing computer executable instructions for implementing the rate enhancement functionality.

[0009] Generally, the technical effect of the disclosed computer-implemented methods is to enhance the operation and functionality of the speech generation device of the present subject matter.

[0010] Additional objects and advantages of the invention will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate at least one presently preferred embodiment of the invention as well as some alternative embodiments. These drawings, together with the description, serve to explain the principles of the invention but by no means are intended to be exhaustive of all of the possible manifestations of the invention.

[0012] Fig. 1 provides a perspective view of an embodiment of a speech generation device in accordance with an aspect of the present invention; [0013] Fig. 2 provides another perspective view of the embodiment of the speech generation device illustrated in Fig. 1 in accordance with an aspect of the present invention;

[0014] Fig. 3 provides an exploded view of the embodiment of the speech generation device illustrated in Figs. 1 and 2 in accordance with an aspect of the present invention;

[0015] Fig. 4 provides a perspective view of another embodiment of a speech generation device in accordance with an aspect of the present invention;

[0016] Fig. 5 provides another perspective view of the embodiment of the speech generation device illustrated in Fig. 4 in accordance with an aspect of the present invention;

[0017] Fig. 6 provides an exploded view of the embodiment of the speech generation device illustrated in Figs. 4 and 5 in accordance with an aspect of the present invention;

[0018] Fig. 7 provides a schematic diagram of exemplary hardware

components for use within a speech generation device in accordance with an aspect of the present invention;

[0019] Fig. 8 provides an exemplary embodiment of a graphical user interface menu for displaying an embedded vocabulary list in a vocabulary list box;

[0020] Fig. 9 provides a flow chart of an exemplary method for implementing a method of enhancing the communication rate of a speech generation device in accordance with aspects of the present subject matter;

[0021] Fig. 10 provides a first view of an embodiment of a graphical user interface menu provided via software features for implementing slots and fillers; and

[0022] Fig. 11 provides a second view of an embodiment of a graphical user interface menu provided via software features for implementing slots and fillers.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Reference now will be made in detail to the presently preferred embodiments of the invention, one or more examples of which are illustrated in the accompanying drawings. Each example is provided by way of explanation of the invention, which is not restricted to the specifics of the examples. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one

embodiment, can be used on another embodiment to yield a still further

embodiment. Thus, it is intended that the present invention cover such

modifications and variations as come within the scope of the appended claims and their equivalents. The same numerals are assigned to the same components throughout the drawings and description.

[0024] Referring now to the drawings, Figs. 1-3 illustrate the various structural components of a first exemplary embodiment of a speech generation device (SGD)

100 in accordance with an aspect of the present invention. Structural components of a second exemplary SGD embodiment 400 are shown in and discussed with reference to Figs. 4-6. The internal hardware and software components and related functionality of both SGDs 100 and 400 may be as described with reference to Figs. 7-11. It should be appreciated that specific embodiments of an

SGD in accordance with the disclosed technology should not be limited by the complete spectrum of hardware and software components described herein.

Instead, an SGD may selectively incorporate one or more of such disclosed features and steps and others to create additional and alternative embodiments.

[0025] As shown in Figs. 1-3, a first exemplary SGD 100 corresponds to a hand-held device that is both portable and easy for a user to hold in his hands and operate to generate speech and accomplish additional functions as described herein. SGD 100 may include one or more outer housing, or casing components, such as a top casing component 102, a first bottom casing component 104 and a second bottom casing component 106. In one embodiment, a battery is stored between the first and second bottom casing components 104 and 106, with or without the inclusion of a metal plating cover to help secure the battery therein. The top casing component 102 and second bottom casing component 106, together with speaker grille 108 collectively form the outer shell for SGD 100.

Second bottom casing component 106 forms the bottommost surface of the outer shell.

[0026] Various features of the outer shell of SGD 100 are provided to

accommodate portability of SGD 100 within a user's hands. First and second opposing sides 110 and 112 of the outer shell formed by casing components 102- 106 are generally shaped in a convex fashion and with a predetermined thickness to fit within the palms of a user's hands. First and second generally convex extensions 114, 116 extend from the second bottom casing component 106 to provide an ergonomic configuration for facilitated location and grip of a user's fingers. First and second convex extensions 114 and 116 generally extend along second bottom casing component 106 from the speaker grille 108 forming a front surface of SGD 100 towards a rear surface of SGD 100. Extensions 114 and 116 may taper towards the rear surface of SGD 100. First extension 114 may be provided generally adjacent or near to the first side 110 of second bottom casing component 106, while second extension 116 may be provided generally adjacent or near to the second side 112 of the second bottom casing component 106.

When a user positions his fingers/hands around the casing components of the SGD device, thus operating the SGD as a hand-held device, the first and second extensions 114 and 116 may be positioned on the bottom surface of the SGD and the outer speaker grille 108 may be positioned to face outward from the user so that audio output from the SGD radiates outwardly to others within a user's environment.

[0027] Referring still to the outer shell formed by the casing components of SGD 100, top casing component 102 may include a recessed area 118 or other feature for accommodating an outer touch screen 120. Top casing component 102 may also be formed to define a relatively large opening 122 therethrough to provide access to a display panel 124 which may be viewed through touch screen 120. Touch screen 120 may generally serve as an input feature for SGD 100, and is described later in more detail. Display pane! 124 may generally serve as an output feature for SGD 100 and is also described later in more detail.

[0028] The casing components forming the outer shell of SGD 100 may also be formed to define additional openings to accommodate data input and output. For example, an opening 126 within top casing component 102 may be provided through which an LED or other light source can provide a device power indicator light. An opening 128 can provide a location for a button by which a user can toggle the power for SGD 100 in an "on" or "off" position. Another opening 130 formed within one or more casing components can provide a location for data input/output ports. Speaker grille 108 may be formed to define a first opening 132 for accommodating a volume control knob to control the volume of speakers associated with SGD 100, as well as a second opening 134 to provide a location for a USB port through which various peripheral devices may be coupled to SGD 100.

[0029] One or more printed circuit boards (PCBs) may also be provided as structural components within SGD 100, and may be housed between top casing component 102 and first bottom casing component 104. For example, a PCB may be provided as a mounting surface or motherboard for such hardware and/or electronics components as a computer processing unit, hard drive(s) and/or other associated memory or media device(s). One or more PCBs may also serve as a mounting surface for radio or wireless communications modules, antennas, data buses, related integrated circuits and passive devices, and the like. First and second speakers 136 may also be positioned among and/or securely mounted to the PCBs and/or casing components. As shown in Fig. 3, speakers 136 may be respectively coupled via a first connection to first bottom casing component 104 and via a second connection to an associated PCB.

[0030] Various connectors may be used to establish relative positioning among the various casing components and secure the components to form an integrated assembly. For example, mating holes, pins and standoffs may be used to orient adjacent casing components to one another. Snap-fit features, threaded screws and/or other connectors may be used to secure and connect the casing

components together.

[0031] A second exemplary embodiment of a speech generation device is shown in Figs. 4-6, respectively. From a functional perspective, such embodiment

400 may include some or all of the same or similar internal hardware and software features as the first exemplary embodiment of Fig. 1 -3. The overall size and dimension of SGD 400 compared to SGD 100 may vary. For example, one exemplary embodiment of SGD 100 corresponds to a size with dimensions 6.15" x 4" x 1.9". One exemplary embodiment of SGD 400 corresponds to a size with dimensions 10.8" x 8.5" x 1.9". The SGD can incorporate one or more of the foregoing features and yet weigh less than 2 pounds when including a 5 1 /4-inch diagonal display and less than 3 pounds when incorporating at least a 10-inch display screen,

[0032] Referring now to Figs. 4-6, SGD 400 generally includes several structural components, including a display lens 402, a top casing component 404, an internal casing component 406, a bottom casing component 408, a speaker grille 410 and a battery door 412. Display lens 402 may be a protective covering or may correspond to the outer surface of a touch-screen LCD or other display device for use with SGD 400, as described later. Top casing component 404 is formed to define an opening therein for accepting the display lens 402 and for providing viewing access to a display device. An upper peripheral surface of top casing component 404 may be formed with a material {e.g., chrome) selected to highlight the outer perimeter of SGD 400 as viewed by a user and to place emphasis on the relatively thin edge profile of SGD 400. Internal casing

component 406 provides a structural support and backer for the device display. Interna! hardware components separate from the display may be mounted and secured between the internal casing component 406 and the bottom casing component 408. Bottom casing component 408 may be formed to define a first opening 414 to access data input/output (I/O) ports. Opening 414 may be secured with a data door that can be flush with the surrounding surface of bottom casing component 408. Bottom casing component 408 may also be formed with a second opening by which a user may access a battery associated with SGD 400. The battery door 412 with latch feature is provided for securing the battery within second opening.

[0033] Similar to the generally ergonomic features of SGD 100, first and second opposing sides 430 and 432 of the outer shell formed by casing components 404 and 408 are generally shaped in a convex fashion and with a predetermined thickness to fit within the palms of a user's hands. Additionally, the bottom casing component 408 of SGD 400 may also include first and second generally convex extensions 418 and 420 projecting from a lower surface of bottom casing component 408. First and second convex extensions 418 and 420 generally extend along second bottom casing component 408 from the speaker grille 410 forming a front surface of SGD 400 towards a rear surface of SGD 400.

Extensions 418 and 420 may taper towards the rear surface of SGD 400. First extension 418 may be provided generally adjacent or near to the first side of bottom casing component 408, while second extension 420 may be provided generally adjacent or near to the second side of the bottom casing component 408. Further, the first and second extensions 418 and 420 may define generally concavely shaped recessed areas 434 and 436, which may also serve as ergonomic features of the SGD 400. For example, the concave recessed areas 434 and 436 may be configured to facilitate location and grip of a user's fingers as they are wrapped around the first and second extensions 418 and 420.

[0034] The outer casing components used in first and second exemplary SGD embodiments 100 and 400 may be molded from any substantially rigid and lightweight material. In one embodiment, one or more outer casing components are made from a material such as but not limited to plastic, thermoplastic, polymer, polyethylene, or resin material. In another embodiment, one or more outer casing components (particularly the bottom casing components) are made from

magnesium or an alloy thereof. When magnesium is used in the casing

components, it provides several advantages for an SGD and enables the case to be an active functional component and not just a protective component.

[0035] First, a magnesium casing provides conductive and radiated immunity to the components mounted within the magnesium case without requiring separate shielding. The magnesium prevents intrusion of radiated conducted emissions that otherwise might effect the electronic components of the SGD disposed within the case. The physical properties of the magnesium also provide a superior shielding from electromagnetic interference (EMI) signals. Typically plastic enclosures need a metal shield or shielding directly on components on the PCB to provide protection from radiated and conducted emissions.

[0036] A magnesium case also provides heat dissipation features for an SGD. In particular, the SGD's microprocessor may include a direct thermal interface thermally coupled to the magnesium case so that the magnesium case functions as a heat sink. Accordingly, the SGD can eliminate the need for cooling fans, which conserves the power demands on the SGD's lithium polymer battery, reduces the SGD's background noise level, which is especially desirably for a speaking device, and eliminates moving parts of the fan that otherwise are subject to breakage upon dropping of the SGD and therefore improves the overall durability of the SGD.

[0037] Additionally, the magnesium case provides greater structural integrity with low weight and high strength than conventional plastic casings. In particular, it permits the walls of the case to be thinner than would be the situation in a fixotropic molded plastic case in a conventional SGD. The strength to weight ratio of the magnesium case thus is also greater than might otherwise be if molded from fixotropic plastic requiring thicker walls in order to achieve the same durability or even less durability than is possible with the magnesium case of the present invention. Injection molding magnesium produces better complex net shape requiring less machining and secondary operations. Bosses can be drilled and tapped thus eliminating the need for inserts or self-tapping screws which allow the case to be vibration and resonance free, thus improving the sound quality.

[0038] It should be appreicated that the SGD configurations illustrated in Figs. 1-6 are included as exemplary embodiments of the present subject matter.

However, in general, the methods disclosed herein may be utilized with any speech generation device known in the art. Thus, one of ordinary skill in the art should appreciate that the scope of the present subject matter need not be limited to the specific embodiments described above.

[0039J Electronic components intended for selective use with a speech generation device in accordance with aspects of the present invention are shown in Fig. 7. The electronic components may include a combination of hardware, software and/or firmware elements, all of which either correspond to physical tangible apparatuses or which are embedded as instructions on a physical and tangible apparatus such as a computer-readable storage medium,

[0040] In general, the electronic components of an SGD enable the device to transmit and receive messages to assist a user in communicating with others. For example, the SGD may correspond to a particular special-purpose electronic device that permits a user to communicate with others by producing digitized or synthesized speech based on configured messages. Such messages may be preconfigured and/or selected and/or composed by a user within a message window provided as part of the speech generation device user interface. As will be described in more detail below, a variety of physical input devices and software interface features may be provided to facilitate the capture of user input to define what information should be displayed in a message window and ultimately communicated to others as spoken output or other outgoing communication.

[0041] Referring now to Fig. 7, central computing device 700 may include a variety of internal and/or peripheral components. Power to such devices may be provided from a battery 703, such as but not limited to a lithium polymer battery or other rechargeable energy source. A power switch or button 705 may be provided as an interface to toggle the power connection between the battery 703 and the other hardware components. In addition to the specific devices discussed herein, it should be appreciated that any peripheral hardware device 707 may be provided and interfaced to the speech generation device via a USB port 709 or other communicative coupling. It should be further appreciated that the components shown in Fig. 7 may be provided in different configurations and may be provided with different arrangements of direct and/or indirect physical and communicative links to perform the desired functionality of such components.

[0042] Referring more particularly to the exemplary hardware shown in Fig. 7, a central computing device 700 is provided to function as the central controller within a SGD and may generally include such components as at least one memory/media element or database for storing data and software instructions as well as at least one processor. In the particular example of Fig. 7, one or more processors) 702 and associated memory/media devices 704a and 704b are configured to perform a variety of computer-implemented functions (i.e., software-based data services). One or more processor(s) 702 within computing device 700 may be configured for operation with any predetermined operating systems, such as, but not limited to, Windows XP, and thus is an open system that is capable of running any

application that can be run on Windows XP. Other possible operating systems include BSD UNIX, Darwin {Mac OS X), Linux, SunOS (Solaris/OpenSolaris), and Windows NT (XPΛ/ista/7).

[0043] At least one memory/media device (e.g., device 704a in Fig. 7) is dedicated to storing software and/or firmware in the form of computer-readable and executable instructions that will be implemented by the one or more

processors) 702. Other memory/media devices (e.g., memory/media devices 704b) are used to store data which will also be accessible by the processors) 702 and which will be acted on per the software instructions stored in memory/media device 704a. The various memory/media devices of Fig. 7 may be provided as a single or multiple portions of one or more varieties of computer-readable media, such as, but not limited to, any combination of volatile memory (e.g., random access memory (RAM, such as DRAM, SRAM, etc.)) and nonvolatile memory

(e.g., ROM, flash, hard drives, magnetic tapes, CD-ROM, DVD-ROM, etc.) or any other memory devices including diskettes, drives, other magnetic-based storage media, optical storage media and others. In some embodiments, at least one memory device corresponds to an electromechanical hard drive and/or or a solid state drive (e.g., a flash drive) that easily withstands shocks, for example that may occur if the SGD is dropped. Although Fig. 7 shows two separate memory/media devices 704a and 704b, the content dedicated to such devices may actually be stored in one memory/media device or in multiple devices. Any such possible variations and other variations of data storage will be appreciated by one of ordinary skill in the art.

[0044] In one particular embodiment of the present subject matter, a first portion of memory/media device 704b is configured to store input data received from a user for performing the desired functional steps associated with a speech generation device. For example, data in memory 704b may include inputs received from one or more peripheral devices, including but not limited to touch screen 706, microphone 708 and other peripheral devices 710, which indicate a user's selections of text to be spoken by the SGD or other related actions.

Memory device 704a includes computer-executable software instructions that can be read and executed by processors) 702 to act on the data stored in

memory/media device 704b to create new output data (e.g., audio signals, display signals, RF communication signals and the like) for temporary or permanent storage in one of the memory/media devices. Such output data may be

communicated to a peripheral output device, such as display device 712, speakers 714, cellular phone or RF device 716, wireless network adapter 718, or as control signals to still further components. Computing/processing device(s) 702 may be adapted to operate as a special-purpose machine by executing the software instructions rendered in a computer-readable form stored in memory/media element 704a. When software is used, any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein. In other embodiments, the methods disclosed herein may alternatively be implemented by hard-wired logic or other circuitry, including, but not limited to application-specific circuits. The steps, features and algorithms described later, including algorithms for impiementing slots and fillers and the like, are provided as executable software instructions stored in the computer-readable medium of elements 704a and/or 704b or others.

[0045] Referring still to Fig. 7, various input devices may be part of an SGD and coupled to the computing device 700. For example, a touch screen 706 may be provided to capture user inputs directed to a display location by a user hand or stylus. A microphone 708, for example a surface mount CMOS/MEMS silicon- based microphone such as offered for sale by Knowles Acoustics, a division of Knowles Electronics of Itasca, Illinois, may be provided to capture user audio inputs. Other exemplary input devices (e.g., peripheral device 710) may include but are not limited to a peripheral keyboard, peripheral touch-screen monitor, peripheral microphone, mouse and the like.

[0046] In one particular example, touch screen 706 includes a projective capacitive touchscreen instead of a keyboard, a mouse or a pressure sensitive touchscreen. A capacitive touchscreen panel is a sensor typically made of glass coated with a transparent conductor such as indium tin oxide (ITO). This type of sensor is basically a capacitor in which the plates are the overlapping areas between the horizontal and vertical axes in a grid pattern. Since the human body also conducts electricity, a touch on the surface of the sensor will affect the electric field and create a measurable change in the capacitance of the device. Thus, a transparent rigid panel covers the display screen and prevents false activations of the SGD when the transparent panel inadvertently is subject to pressure. Each of the icons on the display of the SGD is a capacitive switch, which is activated by the capacitance of the user's body rather than by pressure against the screen. One particular example of a touch screen for use in accordance with the subject SGD corresponds to a capacitive touch screen such as offered for sale by Touch

International of Austin, Texas.

[0047] When employed in accordance with an aspect of the present invention, the incorporation of projective capacitive technology into a dedicated speech generating device has several advantages, especially relative to a typical resistive touch panel, A primary advantage stems from the fact that active sensing material can be applied to the back of any transparent material, in this case a super thin (0.043") chemically strengthened glass. The overall thinness and light weight are crucial in a lightweight, handheld ambulatory design. The top glass surface can be sealed to a bezel associated with a front casing component, providing ingress protection to moisture and dirt as well as a flush mounting surface so there is no recess for debris or liquids to accumulate. The active electronics and graphics are applied to the back surface of the glass which makes the top scratch resistant and increases the durability of the panel. High resolution devices are generally available, such as on the order of 10,000 dpi (dots per square inch). Capacitive touch screens also don't typically require recalibrations, as may be needed with resistive or pressure-sensitive touch screens.

[0048] Capacitive touch screens also provide advantages for conventional users of a speech generation device. A capacitive touch panel requires no activation force but only a slight contact, which is an advantage for a user who may have motor control limitations. Typical resistive sensors have a predetermined activation force required to physically short two conductive layers together. These top layers are external and can suffer failure due to scratching or physical breakdown of the spacing layer that separates the two conductive surfaces.

Capacitive touch screens also accommodate multi-touch applications (i.e., a set of interaction techniques which allow a user to control graphical applications with several fingers) as well as scrolling.

[0049] In general, the different types of input devices (including optional peripheral devices) are configured with software instructions to accept user inputs in accordance with one or more access methods, including the following: a "Touch Enter", Touch Exit", "Touch Auto Zoom", "Scanning", "Joystick", "Audio Touch", "Mouse Pause/Headtrackers", "Morse Code" and/or "Eye Tracking" access methods. In a "Touch Enter" access method, selection is made upon contact with the touch screen, with highlight and bold options to visually indicate selection. In a

"Touch Exit" method, selection is made upon release as a user moves from selection to selection by dragging a finger as a stylus across the screen. In a "Touch Auto Zoom" method, a portion of the screen that was selected is automatically enlarged for better visual recognition by a user. In a "Scanning" mode, highlighting is used in a specific pattern so that individuals can use a switch (or other device) to make a selection when the desired object is highlighted.

Selection can be made with a variety of customization options such as a 1 -switch autoscan, 2-switch directed scan, 2-switch directed scan, 1 -switch directed scan with dwell, inverse scanning, and auditory scanning, in a "Joystick" mode, selection is made with a button on the joystick, which is used as a pointer and moved around the touch screen. Users can receive audio feedback while navigating with the joystick. In an "Audio Touch" mode, the speed of directed selection is combined with auditory cues used in the "Scanning" mode. In the

"Mouse Pause/Headtrackers" mode, selection is made by pausing on an object for a specified amount of time with a computer mouse or track ball that moves the cursor on the touch screen. An external switch exists for individuals who have the physical ability to direct a cursor with a mouse, but cannot press down on the mouse button to make selections. A "Morse Code" option is used to support one or two switches with visual and audio feedback. In "Eye Tracking" modes, selections are made simply by gazing at the device screen when outfitted with eye controller features and implementing selection based on dwell time, eye blinking or external switch activation.

[0050] SGD hardware components may also include one or more integrated output devices, such as but not limited to a display device 712 and speakers 714. Display device 712 may correspond to suitable monitor, screen or other output device for visual presentation of to a user. Suitable examples, include but are not limited to a light-emitting diode (LED) display, electroluminescent display (ELD), plasma display panel (PDP), and liquid crystal display (LCD). One particular embodiment corresponds to an LCD such as an LW500AC9001 component offered for sale by Chi Mei Optoelectronics (CMO) of Taiwan. Speakers 714 may generally correspond to any compact high power audio output device, for example, a GC0401 low profile paper and myiar speaker such as offered for sale by Amaoto Industrial Co., Ltd. (A.I. C.) of Taiwan.

[0051] Speakers 714 may function as an audible interface for the speech generation device when computer processor(s) 702 utilize text-to-speech functionality. In accordance with general functionality of a speech generation device, a user provides text, symbols corresponding to text, and/or related or additional information in a "Message Window" which may then be interpreted by a text-to-speech engine and provided as audio output via the speakers 714. In one embodiment, The SGD also includes an e-book reader that can be controlled by the user to read the e-book and have the e-book speak the words being read to the user. Speech output may be generated in accordance with one or more preconfigured text-to-speech generation tools in male or female and adult or child voices, such as but not limited to such products as offered for sale by Cepstral, HQ Voices offered by Acapela, Flexvoice offered by Mindmaker, DECtalk offered by Fonix, Loquendo products, VoiceText offered by NeoSpeech, products by, AT&T's

Natural Voices offered by Wizzard, Microsoft Voices, digitized voice (digitally recorded voice clips) or others. A volume control module 722 may be controlled by one or more scrolling switches or touch-screen buttons.

[0052] SGD hardware components may also include various communications devices and/or modules, such as but not limited to an antenna 715, cellular phone or RF device 716 and wireless network adapter 718. Antenna 715 can support one or more of a variety of RF communications protocols. A cellular phone or other RF device 716 may be provided to enable the user to make phone calls directly and speak during the phone conversation using the SGD, thereby eliminating the need for a separate telephone device. A wireless network adapter

718 may be provided to enable access to a network, such as but not limited to a dial-in network, a local area network (LAN), wide area network (WAN), public switched telephone network (PSTN), the Internet, intranet or ethernet type networks or others. These communications modules (or others, such as but not limited to an infrared (IR) transceiver) may be provided to function as a universal remote control for the SGD that can operate devices in the user's environment, for example including TV, DVD player, and CD player.

[0053] When different wireless communication devices are included within an SGD, a dedicated communications interface module 720 may be provided within central computing device 700 to provide a software interface from the processing components of computer 700 to the communication device(s). In one

embodiment, communications interface module 720 includes computer instructions stored on a computer-readable medium as previously described that instruct the communications devices how to send and receive communicated wireless or data signals.

[0054] Antenna 715 may be provided to facilitate wireless communications with other devices in accordance with one or more wireless communications protocols, including but not limited to BLUETOOTH, WI-FI (802.11 b/g) and ZIGBEE wireless communication protocols. In one example, the antenna 715 enables a user to use the SGD with a BLUETOOTH headset for making phone calls or otherwise providing audio input to the SGD. The SGD can also generate BLUETOOTH radio signals that can be used to control a desktop computer, which appears on the SGD's display as a mouse and keyboard. An example of a suitable antenna for such devices is a surface mount Rufa brand antenna such as offered for sale by Antenova Ltd. of Cambridge, United Kingdom.

[0055] Integrated BLUETOOTH features afford a user the opportunity to take advantage of several optional BLUETOOTH accessories. For example, a switch may be provided for users to mechanically actuate a selection on the SGD and then communicate that selection via BLUETOOTH protocols. Switching is often used when an SGD operates in a user-input mode where choices are scanned across the display as visual options or sequenced within an audio output and a user can then select one of the scanned options upon selection by switch.

Scanning users often rely on switches located wherever the user has consistent and reliable motor control. Switches may be located on a head rest, seat, leg support, etc. Many conventional switches are hard-wired and include cables that are routed from the device around a wheelchair or other mounting location to the accessibility points. Provision of a BLUETOOTH-communicating input switch eliminates the potential for wire tangling, thus providing a more convenient and safer environment for the user in a wheelchair with moving parts.

[0056] Another option afforded by BLUETOOTH communications features involves the benefits of a BLUETOOTH audio pathway. Many users utilize an option of auditory scanning to operate their SGD. A user can choose to use a BLUETOOTHenabled headphone to listen to the scanning, thus affording a more private listening environment that eliminates or reduces potential disturbance in a classroom environment without public broadcasting of a user's communications. A BLUETOOTH (or other wirelessly configured headset) can provide advantages over traditional wired headsets, again by overcoming the cumbersome nature of the traditional headsets and their associated wires.

[0057] It should be appreciated that words, phrases, symbols, images and other information stored on a SGD can be grouped and used in preconfigured or customized compilations referred to herein as vocabulary lists or vocabulary list boxes. Vocabulary list boxes enable a user to have a wide variety of words and phrases immediately available. By listing groups of related words and phrases, vocabulary list boxes enable a user to quickly search through a wide range of text options when composing a message. For example, a user can select a particular group of words and/or phrases and associate all selected items into a new vocabulary list, which may be named and optionally assigned a unique symbol to visually represent the vocabulary list. Setting options characterizing each vocabulary list may also be defined, for example, as an indication of whether the vocabulary iist should accept new words and phrases from sources available on the Internet. Once a vocabulary list is made, new words or phrases can be added to the vocabulary list and/or existing words or phrases may bθ edited or deleted. Features may also be provided such that existing vocabulary lists can be embedded into other vocabulary lists, thus providing a hierarchy of related sets and subsets of vocabulary lists. Features may be provided to add vocabulary searches into a vocabulary list or to sort and display the contents of a vocabulary list according to selected parameters such as alphabetization and frequency of use in messages. Features may also be provided to trigger actions performed by the SGD upon selection of an item from a vocabulary list, for example, to automatically "speak" or provide as audio output the words/phrases from a vocabulary list box immediately as it is selected by a user, or to send the words/phrases from the vocabulary list box to the Message Window as it is selected by a user.

[0058] When embedding one vocabulary list in another, a user can get better access to more specific vocabulary on a related subject. For example, as shown in Fig. 8, software instructions may be configured to display an embedded vocabulary list in a similar fashion (i.e., as an icon) as any other item in a given vocabulary list box. The example in Fig. 8 shows a user interface displaying a vocabulary list box with the vocabulary list "Animals" having the "Animal Sounds" vocabulary iist embedded within it. When an embedded vocabulary list is selected by a user, its contents will then be displayed in the vocabulary list box. All the words in the original vocabulary list "Animals" (e.g., aardvark, alligator, animal, ant, anteater, antelope, ape, armadillo, etc.) as well as the words in the embedded vocabulary list "Animal Sounds" (e.g., bark, bleat, buzz, chirp, duck, croak, gobble, growl, hiss, etc.) are then available for a user to use in a message.

[0059] When embedding a vocabulary search within a vocabulary list, a search for words and phrases will automatically scan through a dictionary of words and phrases provided on the SGD and display all words and phrases that meet the user-defined search criteria. For example, suppose that a user adds twelve words to an SGD dictionary and marks them all with the "Clothing" category. The next time a user opens a page with a vocabulary list containing a vocabulary search on "Clothing", those twelve new words will automatically appear in the vocabulary list box. By inserting a search itself to a vocabulary list, a search can be performed dynamically. As such, when other items are added to a dictionary that matches a user's search criteria, a user does not have to go back and separately add the new words to different preexisting vocabulary lists. The results of an embedded vocabulary search can be intermixed with the other vocabulary items within the vocabulary list. They will appear side by side with the individual vocabulary items, and they can be sorted based on the settings of the vocabulary list box

(alphabetically, as is, or by frequency of use).

[0060] In exemplary embodiments of the present subject matter, the SGD may further include executable instructions stored on a computer-readable medium that implement a method of communication rate enhancement that combines

sentences and/or phrases with slots and slot fillers. Generally, the disclosed method enables a user to create dynamic phrases and/or sentences by providing to the user the ability to quickly and easily vary the words contained in otherwise static text. In particular, the slots and fillers methodoiogy may provide a user with access to common words that can be used to complete a message in a variety of different settings and situations. For example, in the phrase: "Can we have dinner now?", a slot may be created that includes the slot filler "dinner". Other commonly- used words like "breakfast" or "coffee" may be used as different slot fillers that can be inserted interchangeably into the slot to complete similar messages like: "Can we have breakfast now?". Thus, rather than build a dynamic message one word at a time, a user can create phrases and/or sentences that may contain slots in key locations. As such, when a phrase and/or sentence containing a slot is added to a Message Window or other graphical user interface of the SGD, the user may select the slot and replace the slot fillers to create an entirely new phrase and/or sentence. In other words, the disclosed method may generally provide a variety of vocabulary options while reducing the number of selections that a user must make to create a whole message, thereby increasing the efficiency of message composition and saving the user valuable time and energy. Moreover, the reduction in the number of input selections that must be made by the user may also help to conserve space on the touch screen of the SGD by reducing the number of buttons, menus and other graphical user interfaces that must be displayed to the user.

[0061] In general, implementation of the disclosed method triggers software provision of mechanisms to insert or create a "slot" or placeholder in a textual phrase or sentence. Each slot or placeholder may be associated with a list of "slot fillers" that can potentially be used to fill-in the slot contained within the text. For example, a list of slots and their associated slot fillers may be stored in an internal database of the SGD. Thus, in one embodiment, when a user inserts text including a slot into a document, the slot may be intially filled with a default slot filler from the list of associated of slot fillers. In such an embodiment, the default slot filler may be underlined or otherwise emphasized within the text in order to identify its status as a slot filler and provide the user with a visual indication of the location of the slot. Alternatively, a slot contained with a textual phrase or sentence may be initially unfilled and, thus, may require the user to subsequently fill in the slot with an appropriate slot filler.

[0062] The computer-readable instructions implementing the slots and fillers methodology may aiso provide a user various options for replacing a default filler contained within a slot or for adding a slot filler into an unfilled slot. For example, a user may be able to select a slot contained within a phrase or sentence and, upon such selection, the user may be presented with an entire list of slot fillers associated with that particular slot. The user may then choose a slot filler from the list, at which point the selected slot filler may be inserted into the slot. In one embodiment, the newly inserted slot filler may be underlined within the text so that it can be easily identified and/or changed at a later time.

[0063] In addition, or as an alternative, to selecting a slot filler from a list of fillers associated with a particular slot, it should be appreicated that slots may also be filled using a text input option. In particular, a user may be provided with the option of typing a slot filler directly into a selected slot. As such, the flexibility of the language output available to the user can be significantly enhanced. Thus, when a user selects a slot contained withn a textual phrase or sentence, the user may be presented with both a list of fillers associated with the selected slot and also an option to manually type a slot filler into the slot. For example, a keyboard may be displayed through the display device to enable the user to type a slot filler into the selected slot using the touch screen of the SGD. Once inserted, the typed slot filler may be visually indicated within the text, such as by being underlined, so that the slot filler may be easily identified and/or changed again in the future.

Additionally, in the event that the typed slot filler was not contained within the display list of slot fillers, the new slot filler may be optionally added to the list of fillers associated with the particular slot to permit later use of the typed slot filler without the necessity of re-typing the filler into the SGD. It should be appreciated that, in one embodiment, the text input option may be augmented with word prediction by matching any partially typed text to words stored in an internal dictionary of the speech generation device. As such, the ease and efficiency of inserting slot fillers using the text input option can be greatly enhanced. In a particular embodiment of the present subject matter, the word prediction utilized with the text input option may be biased to give slot fillers contained in an existing list of fillers a higher priority rating than other words generally available to the user

(e.g., words stored in an internal dictionary of the SGD).

[0064] Another variation on the slots and fillers methodology allows a user to declare to speak a textual phrase or sentence containing one or more slots. In doing so, the user may be prompted to specify slot fillers for each slot contained within the phrase or sentence, such as by selecting a slot filler from an associated list of fillers or by typing a slot filler into the slot. The SGD may then generate a text-to-speech output corresponding to the textual phrase or sentence, including the inserted slot fillers chosen, through its speakers. [0065] It should be appreicated that slot fillers may comprise literal terms, such as words or phrases, vocabulary lists and related features as described above and/or embedded vocabulary searches. Slot fillers may also contain embedded recursive slots (i.e., slots fillers that, themsevles, contain slots). As such, the slots and slot fillers framework can be provided with an unlimited number of recursive embeddable structures so as to mirror the complexity of natural language. In particular, slot fillers containing embedded slots may enable various linguistic structures to be made available to a user such that the efficient assembly of complex language structure can be achieved. For example, embedded slots may provide for the consistent availability of numerous language types, such as conjunctions, disjunctions, dependent prepositional clauses, and the like.

[0066] Additionally, it should be appreciated that a "slot" placeholder may be included in button text, button labels and phrases. In other words, specific interface options may be provided by the SGD for a user to create a phrase containing slots, adding slots to buttons, and otherwise work with slots in a

Message Window of the SGD. As such, it should be appreciated that slots and slots fillers can be added and/or deleted from a textual phrase or sentence by the user utilizing the various interface options. Further, to faciliate the selection of slot fillers for use within a textual phrase or sentence, it should be appreciated that slot fillers may optionally have pictures associated with them in order to augment the display of such fillers to the user.

[0067] Referring now to Fig. 9, exemplary steps that may be implemented to enhance the communication rate of a speech generation device are presented. In step 1110, at least one slot is electronically created within a textual phrase or sentence. For example, a user may be provided with specific interface options, such as buttons, menus and the like, to enable the creation of phrases and sentences within the SGD that include slots. In step 1112, the textual phrase or sentence including the at least one slot is displayed to the user on the display device of the SGD. It should be appreciated that slot fillers may generally be emphasized or otherwise visually indicated within the text. For instance, as desribed above, a slot filler may be underlined within the associated phrase or sentence. Alternatively, various other text formats, such as bold, italics, highlighting, font coloring, text style, text size and the like, may be used to indicate the location of a slot filler within the text.

[0068] In step 1114, an input is received selecting the at least one slot. A slot may generally be selected utilizing any suitable input method and any suitable input device. For example, when the SGD of the present subject matter includes a touch screen associated with the display device, user input selecting a slot may be provided by simply contacting the touch screen in the location in which the slot is displayed. It should be appreciated that a slot may be selected by selecting the actual "slot" contained within the phrase and/or sentence or by selecting the slot filler, such as a default filler, included within the slot.

[0069] In step 1116, at least one slot filler is electronically inserted into the at least one slot. For instance, in one embodiment, a slot filler may be automatically inserted into a slot when an input is received selecting the slot filler from a list of fillers associated with that particular slot. Alternatively, a text input may be received from the user such that a typed slot filler is inerted into the slot. It should be appreciated that step 1116 may include both embodiments in which a preexisting filler (e.g., a default filler) is contained within the slot and a different slot filler is being inserted into the slot as a replacement and embodiments in which the selected slot does not contain a slot filler and an initial slot filler is being added into the slot.

[0070] Referring now to the examples provided in Figs. 10 and 11 , the text included in the illustrated Message Window represents the label text for an input button of the SGD. As shown, the first slot contained within the text is associated with a "breakfast" concept and the second slot is associated with a "fruit" concept. Thus, when a user selects each of the slots, the user may choose to replace the slot filler that is currently filling the slot with another slot filler from the same concept or can alternatively type a new slot filler into the corresponding slot. As such, the example phrase "I want oatmeal and a banana for breakfast" can quickly and easily be changed to read as follows: "I want toast and a nectarine for breakfast". Further, as indicated above, slot fillers may also include embedded slots. Thus, if a user enjoys oatmeal having different types of ingredients for breakfast, the first slot associated with the "breakfast" concept and including the slot filer "oatmeal" can be modified to include an embedded slot, such as "oatmeal with ," to enable the creation of more complex sentence structures.

Accordingly, the example phrase "I want oatmeal and a banana for breakfast" can be modified to read "I want oatmeal with cinnamon and a grapefruit for breakfast" or "I want oatmeal with sugar and a peach for breakfast." Various other examples utilizing the disclosed slots and fillers method, including slot fillers having any number of embedded slots, should be apparent to those of ordinary skill in the art.

[0071] Further, as described above, when a user selects a slot contained within a textual phrase and/or sentence, the user may be presented with a list of slot fillers associated with that particular slot. For example, if a particular slot is associated with a "drink" concept, a user, upon selection of the slot, may be presented with a list of specific drink-related slot fillers. Specifically, the list may include an extensive catalog of known drinks and beverages, such as water, orange juice, grape juice, tea, apple juice, milk, beer and the like, through which the user may scan in order to find the desired slot filler to be used to fill-in the selected slot. In addition to, or as an alternative to, the list of specific drink-related slot fillers, the list presented to the user may also include generalized slot filler types. In particular, the SGD of the present subject matter may include suitable software instructions that permit the creation of generalized slot filler types which may be embedded within an existing slot filler list. Thus, upon selection of the slot associated with the "drink" concept, the user may be presented with a list of generalized drink-related slot filler types. For instance, the slot filler types may include groups such as hot drinks, cold drinks, fruit drinks, soft drinks and the like. Moreover, it should be appreciated that further slot filler sub-types may be embedded within the generalized slot filler types. For example, specific sub-types of soft drinks, such as diet soft drinks, caffeine-free soft drinks and regular soft drinks, may be embedded within the "soft drink" slot filler type. Accordingly, the SGD may provide slot filler lists having a hierarchy of related types and sub-types to enable a user to quickly and efficiently fill in slots with desired slot fillers,

[0072] It should be appreciated that a user may also be able to add slots to user-created phrases contained within a customized phrase database stored within the SGD. Adding slots to stored phrases is one way to maximize the potential of both a slot/filler rate enhancement feature as well as a phrase prediction rate enhancement feature. This technique provides a user with rapid access to complete statements, while still enabling the user to vary what he is going to say. For example, if a user tells an assistant what he wants to wear every morning, he may want to create a phrase to say "I want to wear my jeans today." The user may then simply turn the word "jeans" into a slot that accesses the "clothing" concept, Every morning the user can quickly access the same phrase, no matter what page or popup is active, and say "I want to wear my sweater today." or "I want to wear my boots today." The user may also add further slots into the phrase, such as one that accesses the "colors" concept or the "textures" concept, to expand the description provided in such statements.

[0073] Additionally, in accordance with phrase prediction rate enhancement functionality, one or more areas of a user display (e.g., buttons, menus or windows) may be filled with phrases as a user enters text into a document. For example, as a user enters text, the SGD can match the partially entered text to an internal database of stored phrases and present those phrases whose starting characteristics match the partially entered text. Each phrase may also be assigned a priority rating (e.g., based on frequency of use or other characterizing feature). In the case where more phrases are found that match the partially entered text than there are buttons to fill, those phrases with the highest priority ratings are shown. Of course, it should be appreciated that any internal phrase database stored on the SGD may be added to or deleted from by a user.

Additionally, stored phrases may optionally have pictures associated with them such that the pictures may be used to augment the display of the phrases.

Further, as indicated above, the stored phrases may also contain slots with their associated fillers so that the entire phrase, including any slots and/or slot fillers, may be quickly accessed by the user.

[0074] While at least one presently preferred embodiment of the invention has been described using specific terms, such description is for illustrative purposes only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the present invention.

Claims

WHAT IS CLAIMED IS:

1. A method for electronically enhancing the communication rate of a speech generation device, the method comprising:

electronically creating at least one slot in a textual phrase or sentence; displaying the textual phrase or sentence including the at least one slot on an electronic display device of the speech generation device;

receiving an electronic input selecting the at least one slot; and

electronically inserting at least one slot filler into the at least one slot.

2. The method of claim 1 , further comprising communicating the textual phrase or sentence having the at least one slot filled with the at least one slot filler as electronic audio signals outputted by the speech generation device.

3. The method of claim 1 , further comprising displaying on the electronic display device a list of slot fillers associated with the at least one slot.

4. The method of claim 3, further comprising receiving an electronic input selecting the at least one slot filler from the list of slot fillers.

5. The method of claim 3, wherein the at least one slot and the list of associated slot fillers are stored in an internal database of the speech generation device.

6. The method of claim 5, wherein an image displayed on the electronic display device is associated with the at least one slot filler, the image being stored in an interna! database of the speech generation device.

7. The method of claim 3, wherein at least one slot filler type is embedded within the list of associated slot fillers.

8. The method of claim 1 , further comprising receiving a text input corresponding to a typed entry of at least a portion of the at least one slot filler.

9. The method of claim 8, wherein the text input is augmented with word prediction by matching the at least a portion of the at least one slot filler to words or phrases stored in an internal dictionary of the speech generation device.

10. The method of claim 9, wherein the word prediction is biased towards words or phrases contained in an existing list of slot fillers stored in an internal database of the speech generation device.

11. The method of claim 8, wherein the text input is provided through a touch screen of the speech generation device.

12. The method of claim 8, wherein the at least one slot filler

corresponding to the text input is added to a list of slot fillers associated with the at least one slot.

13. The method of claim 1 , wherein the textual phrase or sentence is stored in an internal database of the speech generation device.

14. The method of claim 13, wherein an image displayed on the electronic display device is associated with the textual phrase or sentence, the image being stored in an internal database of the speech generation device.

15. The method of claim 1 , further comprising electronically creating the textual phrase or sentence by receiving a text input corresponding to the textual phrase or sentence.

16. The method of claim 15, wherein the text input is augmented with phrase prediction by matching an initial portion of the text input to phrases stored in an internal dictionary of the speech generation device.

17. The method of claim 1 , wherein electronically creating at least one slot in a textual phrase or sentence comprises receiving an input identifying at least one location within the textual phrase or sentence at which to place the at least one slot.

18. The method of claim 1 , wherein the at least one slot is initially filled by a default slot filler.

19. The method of claim 18, wherein inserting the at least one slot filler into the at least one slot comprises replacing the default slot filler with the at least one slot filler.

20. The method of claim 1 , wherein the at least one slot filler is displayed on the electronic display device using a text format that differs from the text format of any surrounding text of the textual phrase or sentence.

21. The method of claim 1 , wherein the at least one slot filler comprises at least one embedded slot.

22. The method of claim 1 , wherein the at least one slot filler comprises a word, a phrase, a vocabulary list or an embedded search.

23. A speech generation device, comprising:

a display device for electronically displaying images to a viewer;

one or more speakers for generating electronic audio outputs; a processor communicatively coupled to the display device and the one or more speakers; and

a computer-readable medium for storing instructions executable by the processor;

wherein the instructions stored on the computer-readable medium configure the electronic device to implement the method of claim 1.

23. A computer-readable medium comprising computer-readable and executable instructions configured to control a processing device to implement the method of claim 1.