AU620221B2 - A vlsi formant speech synthesiser - Google Patents

Description

i :ti i iI

L

1 I IA620221 COMMONWEALTH OF AUSTRAL PATENTS ACT 1952 COMPLETE SPECIFICATION

(ORIGINAL)

FOR OFFICE USE Form Short Title: Int. Cl: Application Number: Lodged: PJ 3090 7 March 1989 00 o o a0 0 00 0 0 00 0 0 0 0 0 0 00 0 0 0 000 0 Complete Specification-Lodged: Accepted: Lapsed: Published: Priority: Related Art: 0000 0 0 0 00 0 0000 0 0 :o 0 0 0 0.

00 0 0 000 TO BE COMPLETED BY APPLICANT Name of Applicant: Address of Applicant: Actual Inventor: Address for Service: OVERSEAS TELECOMMUNICATIONS COMMISSION AUSTRALIA 231 Elizabeth Street, Sydney, New South Wales, 2000, Australia CLIVE SUMMERFIELD GRIFFITH HACK CO 71 YORK STREET SYDNEY NSW 2000 Complete Specification for the invention entitled: "A VLSI FORMANT SPEECH SYNTHESISER" The following statement is a full description of this invention, including the best method of performing it known to me/us:- S0i3497 07 03 5332A.EM A VLSI FORMANT SPEECH SYNTHESISER This invention relates to a formant speech synthesiser and, in particular, to a central formant filter structure which facilitates very large scale integration (VLSI) implementation of the synthesiser in a single chip; e.g. implementation in an Application Specific Integrated Circuit (ASIC).

Formant speech synthesis has become an established and widely used method of generating synthetic speech 10 alongside the rival LPC techniques. With the rapid expansion of information technology that is projected for t: the next decade, it is predicted that multi-access speech response services will play an increasingly important role •r in the dissemination and communication of information, particularly in the telecommunication industry. The present invention flows from an endeavour made to develop an Sa4 o if. efficient ASIC structure which has sufficient acoustical and processing performance to provide for high quality, SGo.t, intelligible multi-channel speech production.

As a progression from earlier developments made in respect of a parallel formant synthesiser structure and in a o an attempt to achieve a regularised parallel architecture, this being necessary for efficient ASIC implementation, a formant filter design has been produced which comprises a network with a parallel connection of six formant filters.

For design details and an analysis of this structure, reference may be made to a paper entitled "VLSI Structures for the Implementation of a Formant Synthesiser", by 6867S/EM 2 L 1 i C. D. Summerfield, in the transactions of the European Ccnference on Speech Technology, 1987, at pages 393-396. A further paper, entitled "VLSI Structures for the Implementation of a IHigh Performance Formant Speech Synthesiser" has been published as part of the proceedings of the first IASTED International Symposium on Signal Processing and its Applications, 1987.

The complete formant filter structure detailed in these papers contains six formant filter channels, 10 designated F1 to F5 and FN. Channels Fl to F5 model the a time varying acoustical transfer characteristics of the oral cavity. Formant channel FN, which comprises a supplementary 0 d att resonator, models nasal cavity resonance and provides a mechanism for controlling the low frequency response of the synthesiser. Five fixed filters are used at the output of formant channels F1 to F5 to modify the response of the 04°4 resonators and allow correct mixing of the formant channels.

4444 4 44In the following text, "Functional Operator" is a module or unit containing a structure of arithmetic elements 4 o which perform a particular algorithmic function; and the term "primitive" as in multiplier primitive) is used 4 to mean an element which is incorporated in a semiconductor chip to perform an operator or algorithmic task.

A functional operator structure has been developed to implement the parallel formant synthesiser, the structure having five functional operators which perform excitation mixer-gain control, coefficient generation, resonance filtering, fixed filtering and output combining. As will 6867S/EM -3 Si: I4 hereinafter be described in more detail, a pipeline arrangement of six multiplier-primitives implement the mixer-gain, coefficient generator and resonance filter operators in the previously developed structure. However, because of the large number of multiplier-primitives used, this prior structure has proved to be excessively large and It impractical to fabricate as a single ASIC device using current VLSI fabrication technology, i The present invention seeks to diminish this problem by providing a formant filter structure which is arranged to o oo a effect mixer-gain, co-efficient generation and resonance o0 0 9 filter operations in a single very large scale integrated 0 0 S circuit, comprising: 0 0 o0o two concurrently operable bit-serial multiplier- 000 0 primitives, a bank of multiplexers for controlling bit serial inputs to the multipliers and which are arranged to goo schedule the multiplier operations, and an arrangement of S bit serial adder/subtractor-primitives connected to the :00. multipliers to perform algebraic summing and differencing of successive outputs from the multipliers.

The formant filter structure is characterised by the 00 use of two only concurrently operable bit-serial multiplierprimitives. This reduction in multiplier-primitives, and thereby the number of interconnections between them, for the same functionality permits a practical ASIC realisation of a six filter formant speech synthesiser, in that it permits a reduction in the order of 60% in the size and complexity of the previously developed circuit. The invention also permits 6867S/EM- 4 a reduction of the wire overhead and allows the use of automatic place and route algorithms provided by ASIC design tools to achieve an efficient and practical design.

The invention additionally permits multiple independent speech channels to be synthesized on a single

ASIC.

The invention will be more fully understood from the following description of a preferred embodiment of a formant filter structure, the description being provided with reference to the accompanying drawings in which: 8 8 FIG. 1 shows a representation of a complete parallel 0 0 formant speech synthesiser at a system behavioural- S0 S functional level; So°. FIG. 2 shows a functional operator structure which embodies the functions of the synthesiser of FIG. 1; and FIG. 3 shows a representation of an architecture o":o embodying the present invention which uses logic-primitives and mathematic operator primitives and which is arranged to o o effect the mixer-gain, filter coefficient generation and resonance filter operations in the structure of FIG. 2.

The 6 channel, parallel formant speech synthesiser 0oo which is shown in FIG. 1 comprises six resonance filters Fl to F5 and FN connected in parallel. Each of the filters F1 to F5 simulate resonances of the oral tract and are followed by the fixed filters FF1 to FF5 necessary for high quality speech. In the Fl channel, the fixed filter function consists of a single order filter containing both a pole and zeros, whilst all the other fixed filters consist of a 6867S/EM L simple differential filter with the zero at the origin. The additional filter FN is included to simulate the effect of the nasal cavity. The resonator gains are controlled by the mixer circuits Ml to M5 and MN, which also perform the mixing of the outputs of the voice source VS and the noise source NS; the voice and noise sources VS and NS produce the differential vocal fold vibration and fricative noise wave forms respectively. Other inputs are the voice excitation gains, the noise (fricative) excitation gains, and the 10 centre frequencies and bandwidths of the resonance filters.

00 0 0A final output functional operator, combiner C, 0 #4 combines the outputs from each of the parallel channel filters in alternate polarity to produce the output speech So, waveform.

Soo 6 The functional operator structure which is shown in FIG. 2 and which embodies the synthesiser of FIG. 1 oo00 implements the parallel formant synthesiser filtering oo0 0° functions as shown in FIG. 1. The structure contains five S 00oo0 functional operators which perform the excitation mixer-gain 0 0 o controls MGC, coefficient generation CG, resonance filtering F, the fixed filtering FF1 and the output combining OC, 00 respectively.

At the functional operator level, the complete formant synthsiser filtering operation is implemented using a six phase multiplexers clocking scheme. In this strategy, acoustic parameters are queued at the input nodes to the mixer-gain MGC and coefficient generator CG operators. The 6867S/EM 6 j ~I computational latencies of these operators are matched so that the composite excitation function values and resonance filter coefficients arrive synchronously at the input to the second order filter operator F. The output nodes from this operator connect to the fixed filter FF1 and output combiner OC operators. Multiplexers within the output combiner OC operator select the appropriate input node corresponding to the phase of the multiplexing clock and accumulate the formant channel sample values to generate the output speech waveform.

oo Sao a The mixer-gain MGC operator controls the degree of 0 0Q o voiced and fricative components, U' and Uf(t) I f o s o0 respectively, in the composite excitation function applied 0 0 ao°° to the resonant filters F, where U' represents the radiation corrected version of the glottal volume-velocity function. The algorithmic description of the mixer-gain 0o":o operator is given by ej(t) Gv. U' Gf. Uf(t) Soj j j where Gvj and Gfj are the voiced and fricative gains for the jth formant channel respectively.

The implementation using this algorithm constitutes a development of previous approaches. The algorithm can be used in a manner which emulates the previous approaches but it provides an added flexibility which enables more complex voice source configurations to be applied to the filter, this permitting the production of a much wider variety of speech qualities. Moreover, it is this algorithmic approach to the mixer-gain operation which permits a two-multipler 6867S/EM 7 fi 4 -i implementation in the formant filter structure.

The resonance filter coefficients al. and a2j (for the jth formant channel) are calculated from the formant frequency f. and bandwidth using two product terms 2 alj/2 and -a2. (B'j) 2 where the mappings and are provided by two external look-up tables provided by cos(2pf.t) and J J exp(-pb t), 0 o ff I where t is the sampling interval, selected to be 0 0; 100 microseconds to provide an output speech rate of o The resonator difference equation is modified to 0 0 compensate for the coefficients generated by this method and 00 0 is given by the expression oj (t)=ej(t)+2oj(t-t)(alj/2)-oj(t-2t)(-a2j) o* o The output of the jth formant channel, o is 0° 0 constructed from the input excitation function ej(t) .o o combined with the two product terms computed from previous 0 0 resonants filter output values and the coefficients. These values are sequentially stored for each formant channel in 00 0o two recursively connected shift registers.

The fixed filter FF1 operator consists of a real pole and two real zeros and is implemented using shift and add/subtract techniques. The output combiner comprises a recursively connected adder-primitive and shift registerprimitive which operates as an integrator and is controlled by three multiplexers which sequentially select either the 6867S/EM 8 resonator output (FN or DIFF) or the fixed filter Fl output.

The above described functional operator structure was originally developed as a 6-multiplier-primitive structure with two multipliers dedicated to each operator function.

That is, a pipeline arrangement of three, 2-multiplier structures implemented each of the mixer-gain MGC, coefficient generator CG and resonance filter F operators following the data flows shown in FIG. 2. However, because of the large number of multiplier-primitives used, the structure proved to be extremely large and was impractical to fabricate as a single ASIC device.

6# o ao: However, it has been determined that such a device is 6i realisable as a single VLSI device if the 2-multiplier- 0 primitive architecture which is shown in FIG. 3 is employed. This structure, which embodies the present invention, contains two double precision, 16 bit, bit-serial ~multiplier-primitives DP which sequentially perform the naaa o6 o coefficient generation, excitation mixing and resonance oon~ filter operation for each formant filter channel.

0 e 6 Scheduling of the multiplier operations is controlled by a bank of multiplexers MX which control the inputs to the multipliers DP.

The multiplier output nodes are connected to a double precision, adder/subtractor network Al, A2, A3 and Sl which completes the coefficient, mixer and resonance filter calculations. Double precision arithmetic is desirable in the multipliers and adder/subtractor network in order to minimise the acoustical effects of "limit-cycles" on the 6867S/EM 9

I

output speech quality.

Synthesis of each formant channel is controlled by a three phase multiplexer clock, i.e. a clock having 3 sub-cycles within each main clock cycle. Thus an eighteen phase clocking arrangement is required when using the previously mentioned six phase clocking scheme.

Consequently for a 16 bit fully synchronous bit-serial embodiment, the synthesis of a single speech sample value requires a total of (3*6*16 =)288 clock cycles.

The two stages of delay lines required for the 0, implementation of the second order resonance filter are j 0 Q provided by respective shift registers SRl and SR2. The doo length of SRI is given by the number of clock cycles 0 do 0oa required to perform a single synthesis calculation, minus the computational latency (L resonator) of the filter calculation, including the latency of the I/O. For a single ooo synthesis channel, the length is given as 231 samples. The length of SR2 is given by the number of clocks required to o. C6 complete the one sample value synthesis calculation, i.e.

288. The input to this delay line is derived directly from the output from SR1.

For multiple (say N) channel synthesis, the lengths of the shift registers are increased in increments of 288 bits, i.e.

SRIN (N*288) Lresonator SR2 (N*288)

N

Eight multiplexer primitives are used to connect the appropriate data lines to the input of the multipliers i 6867S/EM 10 during the three-phase formant filter calculation. Inputs to the multiplexers are the two excitation functions representing the voice and the fricative noise sources, the mapped formant frequency and bandwidth values, the resonance filter coefficient (from the on-chip coefficient calculation) and the recursive delay line values from SR1 and SR2. The excitation functions and the frequency and bandwidth values are supplied from external sources. A three-phase clock is also supplied to control selection of oo0 the multiplexed lines. A delay of 33 bits is inserted 0 Q between the two sets of multiplexer primitives to synchronise timing and compensate for the 1-bit multiplexer So, latency.

ooo o Four signal inputs are supplied to the multipliers directly from the multiplexer operator. Four data outputs 000 are also provided, two from multiplier 1 (MSB and LSB, o o representing Most-Significant-16 serial-Bits and the oo .a Least-Significant-16 serial-Bits, respectively) and, O 0 0 0 similarly two from multiplier 2 (MSB and LSB, likewise).

The four data signals from the output of the 0 0 0 "O multiplier operator are provided to the adder/subtractor network. In total seven outputs are supplied by this network: FN which represents the nasal channel output; DIFF which is the direct differential output supplied as part of the FF1 fixed filter calculation; pairs of filter coefficients, DELAY which is the delayed output supplied to the FF1 fixed filter, and DIFF1 which is a delayed version of the DIFF which is supplied to the output accumulator.

6867S/EM 11 .j S l Two input control signals provide the control signals for the FF1 fixed filter and output accumulator. Also a control input and output are provided to the delay lines.

A doubling procedure is performed at the output of the first multiplier DP by adder Al which has the LSB and MSB inputs connected together. The output of this primitive is connected to a subtractor primitive arrangement Sl. A predelay is specified for the inputs from the second multiplier DP to compensate for the latency of the previous 0o .o adder Al primitive. Concurrently with the doubling primitive Al, a second adder A3 primitive is used to combine 00000e the weighted excitation function at the output of both o°0 multipliers to produce the composite excitation function.

000 0 The output of this primitive is fed to 17-bit delay lines D1 to synchronise the excitation function data with the o°°00 resonance filter calculation data produced on the following 0 o multiplexer phase. A further adder A2 primitive is used to S0000 combine the composite excitation function and difference from the resonance filter calculation to complete the second order difference calculation. The output of this final 000 adder A2 is supplied to a format converter FC primitive to produce a single precision data value. This is outputted from the chip and forms the input to the external progammable delay line arrangement SRI and SR2. This line is also connected to a 7-bit delay to produce the FN output (which is supplied to the output accummulator operator) and to a single precision subtract primitive to compute the differential outputs for formant channels F2 to F5. A delay S 6867S/EM 12 o o e re onan e f lte cal ula ion dat pro uce on the oll win -JI 'Y I0

I

i line arrangement, consisting of three delay lines is used to compensate for the computational latency and to provide the delayed input to the FF1 fixed filter (signal line designated DELAY). A 4-bit delay line is used at the output of the subtractor primitive to synchronise the differential output with the FF1 data signal at the output of the FF1 fixed filter primitive at the input to the accummulator primitive.

On the first sub-phase, the multiplexers MX select 0a 0 0 10 the mapped formant frequency and bandwidth values, and 9 t S to compute the resonator coefficients, al./2 and These are fed back via two format converters FC (double precision to single precision) and synchronising delay lines D2 (having 28 bit delay) to the input of the multiplexer MX and are used later during the resonator o o0 calculation. The intervening incorrect coefficient values 000 00o o generated on the following phases are ignored by the oo multiplexers.

0 9 During the second sub-phase, the multiplier DP inputs are connected to the excitation function generators and the 0 Q gain controls. The output of the multipliers DP is combined in adder primitive, A3, to generate the composite excitation sample value. Delay lines D1 (having 17 bit delay) are used to synchronise the arrival of this value at adder A2 with the resonance filter calculation.

The calculation at adder A2 is performed on the third clock sub-phase during which the multiplexers select the previously generated coefficients and the outputs of 6867S/EM 13 fl

LI

iR recursive delay lines SRI and SR2. The resonator difference equation product terms from the multiplexers DP are applied to the double precision adder/subtractor network to complete the modified resonance filter difference computation. A third format converter primitive FC3 converts the double precision resonance filter output oj(t), to single precision before being applied to the resonance filter recursive shift registers SRI and SR2.

The synthesis operation is completed by repeating the o 0 10 three-phase clocking scheme six times; once for each of the o eQ parallel formant filters (channels) in the synthesizer .901~r design, F1 to F5 and FN. Real-time speech synthesis in a Sfully synchronous bit-serial system operating at 10 kHz 0oo requires a clocking rate of 2.88 MHz. This is well within the processing bandwidth available from modern CMOS 0 fabrication technology which, generally, can be clocked in 0000 o 0 0 excess of 35 MHz. (Although this figure can vary considerably and is highly dependent upon the electrical characteristics of the circuit layout.) 20 Multi-channel synthesis is achieved by increasing the 00 clocking rate in increments of 2.88 MHz and expanding the recursive shift registers SRI and SR2 in increments of 288 bits to enable multi-channel synthesis operations to be interlaced. The maximum number of synthesiser channels available from a single ASIC device is achieved by increasing the clocking rate and shift register lengths until the maximum operating speed of the device is reached.

6867S/EM 14 68675/EM 14-

K:

Claims (4)

1. A formant filter structure which is arranged to effect mixer-(ain, co-efficient generation and resonance filter operations in a single very large scale integrated circuit, comprising: two concurrently operable bit-serial multiplier- primitives, a bank of multiplexers for controlling bit serial inputs to the multipliers and which are arranged to effchedule the multiplier operations, and an arrangement of bit serial adder/subtractor-primitives connected to the multipliers to perform algebraic summing and differencing of twosuccessive outputs from the multipliers. 2 2 A formant filter structure as claimed in claim 1, wherein the bank of multiplexers employ a three phase multiplexer clock. ip bit3. A formant filter structure as claimed in claim 2, i. multiplexers select the mapped formant frequency and bandwidth successive outputs fromand the multipliers use these to calculate S2. A formant filter structure as claimed in claim 3, wherein during the second phase of the clock cycle the multiplexers select the excitation functions and gains, these are multiplied by the multipliers, and the outputs of the multipliers are combined to form a composite excitation a sample value. A formant filter structure as claimed in claim 4, 68673/EM ij 6867S/EM 3 *II Ii I IiIm. mI 4 wherein during the third phase of the clock cycle, the composite excitation sample value, which has been delayed, is added to the result of a multiplication, in the multipliers, of the coefficients calculated during the first phase, which have been delayed and returned to the input of the multiplexers, and a recursive input obtained via shift registers from the filter output.

6. A formant filter structure according to any previous claim, wherein multi-channel synthesis is achieved 1 0 by an increase of the clocking rate and an expansion of the 0 shift registers.

7. A formant filter structure according to any claim, Q, wherein the structure is realized using fully synchronous bit-serial architecture.

8. A formant filter structure substantially as a described with reference to FIG. 3 of the accompanying o drawings. 0q 0, 0 00 C 0 o o 000q 100 00 000 0 0 00 Ob 0 0 00 2 C 00 o 0 0 0,%0 DATED this 7th day of March 1990 OVERSEAS TELECOMMUNICATIONS COMMISSION AUSTRALIA by their Patent Attorneys GRIFFITH HACK CO. 6867S/EM 16