Speech synthesis aid for the vocally handicapped
This invention concerns a speech synthesis aid for the vocally handicapped which takes advantage of recent advances in the field of electronic speech synthesis to provide a relatively low-cost portable device exhibiting substantial flexibility of application ranging from a communication aid operable by a person incapable of speech to enable speech utterances selected from a preselected range to be made, to a device useful for example as a learning aid which couples synthesised speech utterances with the playing of a simple game or the performance of simple routine operations.
In recent years speech synthesiser circuits have been developed to a point where devices are available which are capable of selective operation to output signals which when applied to drive an acoustic transducer produce clear and intelligible speech. Such devices commonly employ more or less complicated microelectronic circuitry including read-only-memories (ROM's) pre-programmed with data information selectively accessible by a microcontroller for outputting the requisite signals for speech generation. Such circuits
have found application in electronic calculators, instruments and computer-type peripherals where a spoken word output is desired in addition to for example a visual display, and it has been proposed to incorporate speech synthesis systems into operating routines such as for example are involved in pre-flight checking of aircraft. The present invention is not concerned with the actual circuits and methods employed for synthesising speech, rather the invention is concerned with applications for known speech synthesis devices.
Hereinafter described, in accordance with the present invention is a speech synthesis communication aid for the vocally handicapped which employs a small microcomputer to monitor control inputs and appropriately drive a visual display and a speech synthesiser unit. The aim is to provide a portable and easily used aid for persons with speech handicaps. By means of simple controls, the invention allows the user to communicate via synthesised speech, independent of natural voice function. Since a broad vodabulary is required to cover the range of application, a modular approach employing interchangeable repertoire modules and associated word-selector overlays is used, the repertoire modules being designed so that the user may plug them into the speech aid as required. Associated with each repertoire module is a translucent overlay sheet on which appropriate symbols or words are printed, the overlay sheet corresponding to the currently installed repertoire module being adapted to be placed over a selector panel on the speech aid. This panel is comprised of a rectangular array of lamps any one of which may be selected by the user to be illuminated under microcomputer control, the arrangement being such that when an overlay is placed on the
selector panel, the words or symbols of the overlay are positioned to locate over the lamps in the array and may be selectively highlighted to indicate selection of the corresponding speech output. Selector controls are provided for changing, via the microcomputer, the lamp or lamps currently illuminated in the selector panel. Use of these controls allows the apparent movement of an index light behind the overlay sheet, permitting the choice of desired words or symbols printed thereon. Having selected a word or symbol, other controls may be used to cause the invention to utter a spoken equivalent to that word, or to store it. In the latter case, a string of utterances may be constructed by repeatedly selecting and storing words or symbols. When the desired assembly has been created it may be synthetically spoken by operating the appropriate control. The capabilities to repeat and extend existing utterance strings are provided. Further to basic speech-aid capabilities, the invention includes options for extra therapeutic functions. By means of external connection facilities and appropriate programming, units may be interconnected or linked to a master console. These modes are intended for teaching applications. The latter mode allows a teacher to monitor and guide pupils in the style of a language laboratory. Sentence construction exercises might be conducted through such a network equipped with suitable overlay sheets and vocabulary modules. Provision is made for data to flow bidirectionally between speech-aid units and the control console. In a standalone therapeutic mode, the aid may function as a game or puzzle, simple concept-ordering problems being provided in pre-programmed form associated with specific overlay sheets and vocabulary modules and the vocal
output of the unit being used to guide and encourage the user.
The invention enables an unlimited repertoire, a simple means of utterance selection and outstanding therapeutic capabilities. A lack of restriction on repertoire size is obtained by virtue of the use of interchangeable vocabulary modules and overlay sheets. The back-illuminated selector panel with uncomplicated controls (joystick or slider, or push-buttons, for example) facilitates utterance selection. Due to its microcomputer-based design, the invention is amenable to functional expansions including network and games modes.
The invention in its various aspects, and features and advantages thereof, will become fully apparent to those possessed of appropriate skills from consideration of the following descriptions of two exemplary embodiments of the invention, the first embodiment being described in outline and the second in detail, which are illustrated in the accompanying drawings wherein:-
Fig. 1 is a top plan view of the operating panel of the first embodiment;
Fig. 2 shows the circuit diagram (in block form) of the embodiment of Fig. 1;
Fig. 3 shows the circuit diagram of an input . switch sub-system of the circuit of Fig. 2;
Fig. 4 shows the circuit diagram of a display sub-system of the circuit of Fig. 2; and Figs. 5 and 6 together show the basic system software flow-chart of the first embodiment; and wherein:-
Fig. 7 shows a schematic circuit diagram of a microcomputer section of the second embodiment; Fig. 8 shows a schematic circuit diagram of
display and keyboard sections of the second embodiment;
Fig. 9 shows a schematic circuit diagram of speech synthesis and control key sections of the second embodiment; Fig. 10 shows a schematic circuit diagram of a power supply section of the second embodiment;
Fig. 11 shows input/output port allocation in the second embodiment; and
Fig. 12 shows a simplified system software flow-chart of the second embodiment. Fig. 1 shows the front face appearance of the first described embodiment which may be comparable in size and weight with a portable cassette tape recorder. The controls include an 8 x 8 cell individually and selectively back-illuminated selector panel 1 with slider controls 3 and 4 to allow choice of the cell illuminated on the selector panel, the controls selecting columns and rows respectively. Translucent word- defining overlay sheets (not shown) are laid on panel 1. Power to the unit is switched by 2, whilst keys 5 and 6 control "speak" and "store" functions of the unit, the "speak" key being used to initiate the device output and for repetition of utterances and the "store" key being used when sequentially selected utterances are to be strung together.
The device of Fig. 1 is adapted to provide readily intelligible synthesised speech from a Tele- sensory Systems Inc. TSI-S2 large scale integrated two chip synthesiser with interchangeable vocabulary ROMS embodied as plug-in modules. The device operates under microcomputer control, a device from the Intel MCS-48 single-chip microcomputer family for example being selected for this purpose. The 8748 unit of this family might by selected, this unit. being UV erasable and reprogrammable and providing 1K x 8 bits of control
memory, 64 x 8 bits of read/write memory and 27 input/ output (I/O) lines. Such a device is capable of around one minute of speech output per vocabulary module, divided among 64 selectable utterances. In the embodiment described only the speech output mode is provided. Symbols selected by the user are uttered by the invention. Utterances may be stored to produce strings of words and may be repeated. Expansion to include the network and therapeutic puzzle modes mentioned hereinbefore is readily available by appropriate reprogramming of the microcomputer control memory and the addition of a simple hardware interface.
The system may be considered as five sections, namely:-
(i) a single-chip microcomputer with support components; (ii) input switches and associated bus-sharing components; (iii) the speech synthesiser chip set and associated audio-path components; (iv) decoders controlling the selector panel and system enable lines; and (v) the selector panel driver and lamp array sub-system.
Referring to Figs. 2, 3 and 4 of the accompanying drawings, an Intel 8748-8 microcomputer 15 (Fig. 2) operating with a cycle time of 4.17μs is the central controlling element of the system, ports 1 and 2 of the microcomputer being used for output and mixed I/O respectively. Output port 1 provides a six bit select bus to speech synthesiser integrated circuit 10 and to display decoders 8 and 11. The speech synthesiser 10 comprises for example a Telesensory Systems Inc. device no. S2. Bits 6 and 7 remaining from Port 1
are decoded to provide four system module enable lines via decoder 13. These lines are used to control the speech synthesiser 10 and the input subsystem constituted.by selection switches 16, 17 and function keys l8. The low order four bits of Port 2 are not used to control intrinsic system functions, but are dedicated to the role of providing the top four bits of addresses to external program memory in conduction with 8 lines from address latch 19. These low order address bits are strobed from the data bus into the address latch during ALE cycles.
Bits 4 to 7 of Port 2 are used to input data from the. select and function switches 16, 17 and 18. As shown in Fig. 3 , all three switch modules share the bus and are selected onto it by pulling switch common lines to ground via transistors 21, 23 and 26. These devices are controlled by system enable lines. The structure of the 8748 Ports 1 and 2 is such that, when configured as inputs, lines are internally pulled high by 50k resistors, thereby obviating the need for external pull-ups. The diodes 20, 22, 24 and 25 associated with the switches prevent the state of disabled switches from effecting valid data.
Fig. 2 shows the speech synthesiser LS1, 10 interfaced to the microcomputer 15 via eight lines; the previously mentioned six-bit utterance select input, and "start" and "busy" lines. A level-shifting network is provided as shown in Fig. 2 on appropriate synthesiser 1/0 lines to allow interfacing of the TTL and PMOS logic signals. "Start" is an input and is controlled by a system enable line. "Busy" is a status output indicating synthesiser non-availability, and is monitored by a single-bit testable input, TO. Connections are also made to a vocabulary ROM 7, housed in a plug-in module. This contains the parameters associated with the
utterance set from which the synthesiser reconstructs speech. Resulting audio signals are low-pass filtered by filter circuit 9 to remove sampling frequency components associated with digital signal processing. They are then fed, after amplification by a small monolithic audio amplifier 12, to a loudspeaker 14.
The circuit diagram of the eight by eight incandescent lamp display matric constituting the selector panel is represented in Fig. 4 which is a simplified diagram showing only four element lamps and associated drivers, the full system comprising eight such row and column drivers connected to 64 incandescent lamps each with an associated diode. Lamps 30 etc. are driven by row and column PNP transistor buffers 27 and 28 respectively. Eight transistors make up each buffer. These devices are activated by low- true signals derived from two 74LS138three-to-eight line decoders (8 and 11 in Fig. 2) controlled by bits 0 to 5 of Port 1 of microcomputer 15. Diodes 29 etc. are employed in the array to prevent spurious current paths through unselected lamps. System expansion facilities have been provided by incorporating an address bus latch (Fig. 2, 19) and by making PSEN (program Store Enable), RD (Read), WR (Write) and the data and address busses externally accessible. Memory mapped interface devices and up to
3K bytes of program memory and 190 bytes of data memory may be added to the system via these expansion facilities. System software is illustrated by the flowchart of Figs. 5 and 6 wherein the following terminology is employed:- UTT ADDR is the synthesiser and selector panel control address; and UTT PTR is a pointer to the current UTT ADDR storage location in data memory.
Referring to Figs. 5 and 6, the majority of CPU time is spent in an idle loop in which the selector panel controls and function keys are scanned. Display refreshing is done in this loop, illuminating the current lamp for 20 ms per cycle. After reading the row and column selector switches, the data obtained is translated from Grey code (produced by the slide logic switches) to an UTT ADDR. This is then compared with the UTT ADDR obtained in the last scan. A repeat flag, FO is set if a change is detected.
When the "Store" key is actuated, the current UTT ADDR is stored at the UTT PTR location, the FO flag is cleared and the UTT PTR is incremented. Thus a string of utterances may be stored by alternately setting the selector panel controls and depressing the "Store" key.
Actuation of the "Speak" key causes the FO flag to be tested. If it is set, the selector panel controls have been altered since the last speak or store operation, and a new item is assumed. Thus the UTT PTR and FO flag are reset before the selected utterance is output. If the F0 flag is not set, repeat mode is assumed and the current stored string is output, leaving the UTT PTR unchanged, ready for a repeated output if the "Speak" key is re-depressed. If the
"Store" key is actuated at this point, new utterances may be added to the end of the existing string.
It will be appreciated that the flow diagrams (Figs. 5 and 6) are at a high level and do not show the structure of specified operations.
The function key input subroutine is edgesensitive and includes debounce delays to avoid spurious data entries. The "speak string" subroutine outputs to both the speech synthesiser and selector panel. Data is output from an UTT ADDR buffer starting
at the buffer base and ending on the last position of the UTT PTR. BUSY, generated by the speech synthesiser, is tested within the subroutine, allowing correct output timing. Functional enhancements are attainable within the IK byte on-chip program memory space. External additional program and data memory may be added to further expand functional capabilities.
Having thus described, in outline, a first embodiment of the invention, a second embodiment will hereinafter be described in detail with reference to Figs. 7 to 12 of the accompanying drawings.
Whilst a single-chip microcomputer controller would be employed in production versions of the speech aid hereinafter described to minimise parts cost, the described embodiment uses a Z8θ microprocessor with associated support components, this approach having been dictated by the availability to the inventors of ZδO software development tools. The embodiment hereinafter described comprises four readily identifiable sections which are illustrated in Figs. 7, to 10, namely a microcomputer controller (Fig. 7) an integrated display/keyboard (Fig. 8), aspeech synthesiser unit with vocabulary ROMs and audio output components (Fig. 9) and a logic-controlled power supply (Fig. 10).
Referring to Fig. 7, the microcomputer controller is based on a Z8θ microprocessor 50, and employs a memory-mapped architecture with partially decoded linear select device addressing. Program memory consists of 2k bytes of PROM space in a single 2716 device 51 residing between hexadecimal addresses 0000 and 07FF in the system memory space. By virtue of its erasable/reprogrammable nature, the 2716 was a logical choice for program development. Read/write memory is
provided in the form of two 2114 RAM chips 52 offering Ik bytes of temporary workspace mapped between hexadecimal addresses 3000 and 33FF.
Communication with display, keyboard, speech synthesiser and power supply control lines is furnished via two 8255A programmable peripheral interface (PPI) devices 53 each offering three eight-bit I/O ports, A, B and C. These are configured under software control to give four output and two input ports. Mapped in memory space, the PPIs have hexadecimal addresses of
5000 to 5003 and 9000 to 9003. respectively. The former device controls display rows, display columns and various enables through its three ports. The latter strobes keyboard rows and provides speech synthesiser utterance addresses through its A port, whilst its B and C ports, configured as inputs, monitor keyboard columns and miscellaneous system status bits.
System support functions (clock and reset generation and device selection) are performed by the remaining microcomputer components. A discrete crystal-controlled transistor oscillator 54 operating at 1MHz generates the system clock signal. This is buffered by a TTL schmitt inverter 55 before being fed to the microprocessor clock terminal. An inverted version of the same signal is used to synchronise, via a D type flip-flop 56, an asynchronous reset line derived from a schmitt inverter-based power-on reset circuit 57. The synchronised reset signal is fed in inverted and non-inverted forms to the reset terminals of the microprocessor 50 and PPIs 53 respectively. An array of two-input NAND gates 58 each fed by an inverted version of the Z8θ MREQ. (memory request) signal together with levels derived from the top four bits of the address bus provide device enable lines. Following a system reset, the Z80 program counter commences
addressing memory from hexadecimal 0000, hence address bit A12 is initially low. This bit is inverted before being routed to the PROM enable NAND gate, thereby allowing program memory access following system reset. The other device-enabling gates are fed directly by address lines A13, A14 and A15 and enable RAM and PPIs 1 and 2 respectively. As is generally the case with linear select systems, only one address selecting bit may be true at any one time, hence A12 A13 A12 and A15 are considered mutually exclusive in the system software. The RAM enable signal is further gated with RD and WR (read and write) signals by gates 59 before being routed to the 2114 chip select terminals. This prevents data bus contention during write cycles when the RAM enable signal is true before a write pulse. Reference should be made to the Zilog Z80 Technical Manual for a detailed description of Z80 system timing.
Referring now to Fig. 8, the display/keyboard circuit is schematically represented. Physically, this assembly comprises a set of 64 back-illuminated switches 60 (only some of which are shown) realised via membrane switch technology. This allows cells of the flexible translucent keyboard overlays, associated with particular vocabulary repertoires to be selectively high-lighted under software control.
The display consists of 64 T1 incandescent lamps 61 (used because of their "white" light output) isolated against spurious back currents by series diodes 62 aiid driven by 16 bipolar transistors 63 arranged as eight row and eight column drivers, the former being configured as emitter followers while the latter operate in the saturating common emitter mode. As with the lamps 61, only some of transistors 63 are shown in Fig. 8. Both driver sets are arranged to be enabled
by high-true signals. A set of eight 4K7 resistors 64 pull up the row driver inputs which are connected to port A of PPI 1 (Fig. 7). Eight 390Ω. resistors 65 limit drive current sourced by port B of PPI 1 to the bases of the column driver transistors 63.
The keyboard may be based on a matrix of 8 x 8 thin silver contact strips laid down on two stable plastic films. A thin perforated non-conducting membrane separates the films which are arranged so that the conductor sets lie at right angles to each other. The perforations are aligned with the crosspoints of the conductor-carrying films so that external pressure (e.g. from a finger) causes deformation of the film through a perforation and consequential contact closure. Keyboard rows are strobed by port A of PPI 2 (Fig. 7) which, under software control, sequentially pulls row lines .low. Column lines, pulled high by 47K resistoι»s 66 are montored by port B of PPI 2. If a coincidence of row strobe and contact closure occurs, the associated column line is pulled low. Software then decodes the location of the depressed key.
The speech synthesiser and control key subsection is shown in Fig. 9 . A Telesensory Systems Inc. S2B mini speech synthesis board 70 is used in the embodiment, this circuit having been modified to include a vocabulary ROM containing a "standard" set of 64 utterances mounted "off-board" as a plug-in module. Different plug-in modules will contain repertoires appropriate to the diverse needs of the speech handicapped. The computer interface to the board 70 comprises six address lines SA 0-5 (to select one of 64 utterances), a "start" line ST, a ROM power down control line VR0M, and a "busy" output. The S2B requires its inputs to be pulled high via resistors and its "busy" output to be level-shifted and buffered
when interfaced to TTL circuitry.
Utterance address and ROM power down signals derived from port A, PPI 2 and a "start" signal from port C, PPI 1 are buffered by a 74LS244 device 71 and pulled high via a 4K7 resistor package 72 before being fed to the appropriate terminals of the S2B. The "busy" signal from the synthesiser 70 is level- shifted and buffered by a discrete transistor inverter 73, then fed to port C of PPI 2 along with low true "speak" and "clear" control key lines. The analogue output AAS of the S2B is processed by an amplifier 74 and filter 75 before being routed to a loudspeaker 76. Filter characteristics are variable by means of a preset Baxandall circuit to allow intellegibility to be optimised. The basic filter response is bandpass and, with Baxandall flat, has corner frequencies at 400Hz and 1KHz. Roll-on and roll-off are at approximately 6dB per octave. Filter and amplifier are realised via 3l4θ operational amplifier chips. The output level is adjustable by a gain trimmer in the amplifier stage. A complementary emitter follower buffer 77 using germanium transistors for low base-emitter voltage drop drives the loudspeaker from the filter output. A red/green light emitting diode (LED) indicator 78, driven by a pair of TTL inverters 79 enabled by signals from port C of PPI 1 is employed to show system status. Typically green will be lit to show the normal "on" state. Red might indicate low battery conditions. In order to meet certain special functional requirements of the speech aid, the power supply (Fig. 10) incorporates several unusual features. It is required that the aid be portable and independent of external power sources, hence internal batteries are dictated. Most internal circuitry requires a +5 volt
supply, hence three 2-volt Cyclon accumulators 80 were chosen for the power source. These 2.5 Ah devices yield a series EMF of 6 volts, allowing an accompanying regulating circuit only a one volt drop. In addition it was desired that a low battery condition be signalled to the microcomputer to allow an orderly automatic shutdown procedure. Further, a requirement for toggle on/ off action and automatic time-out in standby mode was specified. Finally the S2B speech synthesiser requires a minus 10 volt rail which must be derived from the 6 volt power source.
The requirement of low regulator dropout voltage dictated a germanium saturable series pass element 8l. This transistor is driven by the collector current of one side of a long-tailed pair 82 which operates as a comparator between a zener derived reference potential and a proportion of the output voltage. Emitter current for the long-tailed pair 82 is sunk by a resistor 83 in series with a transistor switch 84. When the transistor 84 is disabled, base current to the series pass transistor 8l is cut off and the regulator output falls to zero. A keep-alive current is fed to the reference zener 85 under these conditions. This facilitates orderly power-up behaviour. A PNP transistor 86 with baseemitter connected via a current limiting resistor 87 across the series pass transistor 8l monitors the input- output differential voltage and produces, via a CMOS schmitt trigger gate 88 , a "battery low" logic signal which is routed to port C of PPI 2. Under powered down conditions the regulator draws less than one milliamp, and hence may be permanently connected to the six volt supply. The base of the long-tailed pair emitter current switch transistor 84 is fed via a current limiting resistor 89 from a set/reset flip-flop 90 configured
using CMOS schmitt NAND gates, permanently powered from the six volt supply. Flip-flop 90 may be triggered "on" by the momentary closure of the "on/ off" key switch 91. A differentiating network 92 between the switch and gate ensures a short ( 10 microsecond) trigger pulse. The set/reset flip-flop 90 is triggered "off" by a high to low transition of a line from port C of PPI 1. Under conditions of low battery voltage, a "lock-out" circuit 93 prevents the flip-flop 90 from adopting a stable "on" state. This is achieved by resetting the flip-flop 90 with a suitably delayed version of the "battery low" logic signal. Automatic power-down on low battery voltage is disabled after a few milliseconds by a delayed output from the flip-flop 90 derived by CR network 94. After this initial automatic power-down time "window", only a logic signal from the microcomputer can switch off the system. To this end an output (low true) is derived from the "on/off" control key 91 and routed to port C of PPI 2, thereby enabling the microcomputer to detect when the user wishes to power-down.
In order to cater for the -10 volt rail requirements of the S2B, a flyback type converter is included in the power supply sub-system. A 1:1 ferrite pulse transformer 95 is driven by a transistor 96 in series with a current sensing resistor 97 from the six volt supply. Transformer primary current is monitored by the inverting input of a 3140 operational amplifier 98 c-onfigured as a schmitt trigger with threshold varied by feedback from the transformer secondary circuit. Two paths provide both AC positive feedback ensuring high switching efficiency, and negative DC feedback regulating the output voltage. Secondary current is rectified and smoothed to form the minus 10 volt output. Logic controlled on/off switching is effected by virtue of the
power supply to operational amplifier 98 being derived from the +5 volt rail. When this is disabled, no base current flows in the switching transistor 96, hence no current is drawn from the six volt supply. Having thus described the construction and organization of the second embodiment, reference may now be had to Fig. 12 which is a software flowchart of the basic system function together with the following functional description of the machine operation. On power-up (depression of ON/OFF key) LED flashes red during settling time and until key is released, then lights green to indicate "ready" state. If battery voltage is initially low, power-up is inhibited by hardware. To enter an utterance or string of utterances, display areas are pressed in the order that utterances are required. As areas are pressed they back-illuminate to show current store contents.
Pressing the SPEAK key allows the stored utt erance string to be verbally output in the order of entry. As each utterance is made the associated display area blanks. At the end of the string all previously selected display areas are re-illuminated.
Pressing the CLEAR key during any user entry phase clears the utterance store and blanks the display. Entering an utterance already stored clears that utterance from memory and blanks the associated display area.
If more than l6 utterances are entered, LED flashes red/green and further entries are ignored; control key functions continue to operate normally.
If battery volts drop below the "battery low" threshold during operation and the machine is speaking, the LED displays red and the machine automatically powers-down after the current utterance is spoken.
Under standby conditions, low battery voltage is indicated by red LED illumination for 5 seconds during which time all control and entry functions except ON/ OFF are locked out. At the end of the time-out period automatic power-down is invoked.
If the machine is left "on" for greater than 10 minutes without any entries being made, the LED flashes green for 10 seconds to warn of incipient power-down. Any entries made during this time restore normal operation. If no entry is made, the machine powers itself down.
There follows appended hereto a complete software listing in respect of the operations and external and internal control functions in respect of the embodiment abovedescribed with reference to Figs. 7 to 12 of the accompanying drawings.
Whilst there have thus been described two embodiments of the invention, it will be well appreciated by those possessed of appropriate skills that many variations, alterations and modifications, could be made without departure from the spirit and scope of the invention. The foregoing is therefore intended to be read as exemplifying the invention and not in a limitative sense.