Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L13/00—Speech synthesis; Text to speech systems
  • G10L13/08—Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
  • G10L13/10—Prosody rules derived from text; Stress or intonation
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L13/00—Speech synthesis; Text to speech systems
  • G10L13/02—Methods for producing synthetic speech; Speech synthesisers
  • G10L13/04—Details of speech synthesis systems, e.g. synthesiser structure or memory management
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
  • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
  • G10L21/0208—Noise filtering
  • G10L21/0264—Noise filtering characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques

Definitions

• the present invention is concerned with the automated generation of speech (for example from a coded text input). More particularly it concerns analysis-synthesis methods where the "synthetic" speech is generated from stored speech waveforms derived originally from a human speaker (as opposed to “synthesis by rule” systems). In order to produce natural-sounding speech it is necessary to produce, in the synthetic speech, the same kind of context-dependent (prosodic) variation of intonation that occurs in human speech. This invention presupposes the generation of prosodic information defining variations of pitch that are to be made, and addresses the problem of processing speech signals to achieve such pitch variation.
• a waveform portion to be used is divided into overlapping segments using a Hamming window having a length equal to three times the pitch period.
• a global spectral envelope is obtained for the waveform, and a short term spectral envelope obtained using a Discrete Fourier transform; a "source component" is obtained which is the short term spectrum divided by the spectral envelope.
• the source component then has its pitch modified by a linear interpolation process and it is then recombined with the envelope information. After preprocessing in this way the segments are concatenated by an overlap-add process to give a desired fundamental pitch.
• time-domain overlap- add process may be applied to an excitation component, for example by using PC analysis to produce a residual- signal (or a parametric representation of it) and applying the overlap-add process to the residual prior to passing it through an LPC synthesis filter (see “Pitch-synchronous Waveform Processing Techniques for Text-to Speech Synthesis using Diphones", F. Charpentier and E. Moulines, European Conference on Speech Communications and Technology, Paris, 1989, vol. II, pp. 13-19).
• FIG. 1 The basic principle of the overlap-add process is shown in Figure 1 where a speech signal S is shown with pitch marks P centred on the excitation peaks; it is separated into overlapping segments by multiplication by windowing waveforms W (only two of which are shown).
• the synthesised waveform is generated by adding the segments together with time shifting to raise or lower the pitch with a segment being respectively occasionally omitted or repeated.
• a speech synthesis apparatus including means controllable to vary the pitch of speech signals synthesised thereby, having:
• the windows consist of first windows, one per pitch period, employing the timing mark portions and a plurality of intermediate windows, and the intermediate windows each have a width less than that of the first windows.
• the invention provides a speech synthesis apparatus including means controllable to vary the pitch of speech signals synthesised thereby, having:
• (iii) means for recombining the spectral and excitation components wherein the multiplying means employs at least two windows per pitch period, each having a duration of less than one pitch period.
• the compression/expansion means is operable in response to timing mark information corresponding at least approximately to instants of vocal excitation to vary the degree of compression/expansion synchronously therewith such that the excitation signal is compressed/expanded less in the vicinity of the timing marks than it is in the centre of the pitch period between two consecutive such marks.
• Figure 2 is a block diagram of one form of synthesis apparatus according to the invention.
• Figures 3 and 5 are timing diagrams illustrating two methods of overlap-add pitch adjustment; and Figure 4 is a timing diagram showing windowing of a speech signal for the purposes of spectral analysis.
• portions of digital speech waveform S are stored in a store 100, each with corresponding pitchmark timing information P, as explained earlier.
• Waveform portions are read out under control of a text-to-speech driver 101 which produces the necessary store addresses; the operation of the driver 101 is conventional and it will not be described further except to note that it also produces pitch information PP.
• the excitation and vocal tract components of a waveform portion read out from the store 100 are separated by an LPC analysis unit 102 which periodically produces the coefficients of a synthesis filter having a frequency response resembling the frequency spectrum of the speech waveform portion.
• This drives an analysis filter 103 which is the inverse of the synthesis filter and produces at its output a residual signal R.
• the LPC analysis and inverse filtering operation is synchronous with the pitchmarks P, as will be described below.
• the next step in the process is that of modifying the pitch of the residual signal.
• This is (for voiced speech segments) performed by a multiple-window method in which the residual is separated into segments in a processing unit 104 by multiplying by a series of overlapping window functions, at least two per pitch period; five are shown in Figure 3, which shows one trapezoidal window centred on the pitch period and four intermediate triangular windows.
• the pitch period windows are somewhat wider than the intermediate ones to avoid duplication of the main excitation when lowering the pitch.
• the windowed segments are added together, but with a reduced temporal spacing, as shown in the lower part of Figure 3; if the pitch is lowered, the temporal spacing is increased.
• the relative window widths are chosen to give overlap of the sloping flanks (i. e. 50% overlap on the intermediate windows) during synthesis to ensure the correct signal amplitude.
• the temporal adjustment is controlled by the signals PP. Typical widths for the intermediate windows are 2 ms whilst the width of the windows located on the pitch marks will depend on the pitch period of the particular signal but is likely to be in the range 2 to 10ms. The use of multiple windows is thought to reduce phase distortion compared with the use of one window per pitch period.
• the residual is passed to an LPC filter 105 to re-form the desired speech signal.
• the store 100 also contains a voiced/unvoiced indicator for each waveform portion, and unvoiced portions are processed .by a pitch unit 104' identical to the unit 104, but bypassing the LPC analysis and synthesis. Switching between the two paths is controlled at 106. Alternatively, the unvoiced portions could follow the same route as the voiced ones; in either case, arbitrary positions are taken for the pitch marks.
• Linear interpolation is not ideal for resampling, but is simple to implement and should at least give an indication of how useful the technique could be.
• the signal When downsampling to reduce the pitch period, the signal must be low-pass filtered to avoid aliasing. Initially, a separate filter has been designed for each pitch period using the window design method. Eventually, these could be generated by table lookup to reduce computation.
• the resampling factor varies smoothly over the segment to be processed to avoid a sharp change in signal characteristics at the boundaries. Without this, the effective sampling rate of the signal would undergo step changes.
• a sinusoidal function is used, and the degree of smoothing is controllable.
• the variable resampling is implemented in the mapping process according to the following equation:
• M number of samples of original signal
• N number of samples of new signal
• ⁇ [0, 1J controls the degree of smoothing
• T(n) position of the n' th sample of the resampled signal.
• a major difference between this and single window overlap-add is that the change in pitch period is achieved without overlap-add of time-shifted segments, provided that the synthesis pitchmarks are mapped to consecutive analysis pitchmarks. If the pitchmarks are not consecutive, overlap-add is still required to give a smooth signal after resampling. This occurs when periods are duplicated or omitted to give the required duration.
• An alternative implementation involves resampling of the whole signal rather than a selected part of each pitch period. This presents no problems for pitch raising provided that appropriate filtering is applied to prevent aliasing, since the harmonic structure still occupies the whole frequency range. When lowering pitch, however, interpolation leaves a gap at the high end of the spectrum.
• this is synchronous with the pitch markings. More particularly, one set of LPC parameters is required for each pitchmark in the speech signal. As part of the speech modification process, a mapping is performed between original and modified pitchmarks. The appropriate LPC parameters can then be selected for each modified pitchmark to resynthesise speech from the residual.
• LPC parameters are interpolated at the speech sampling rate in both analysis and synthesis phases.
• each set of LPC parameters would be obtained for a section of the speech portion (analysis frame) of length equal to the pitch period (centred on the midpoint of the pitch period rather than on the pitch mark), or alternatively longer, overlapping sections might be used which has the advantage of permitting the use of an analysis frame of fixed length according to pitch.
• a windowed analysis frame is preferred, as shown in Figure 4.
• the frames in Figure 4 are shown with a triangular window for clarity, the choice of window function actually depends on the analysis method used.
• a Hanning window might be used.
• the frame centre is aligned with the centre of the pitch period, rather than the pitchmark. The purpose of this is to reduce the influence of glottal excitation on the LPC analysis without resorting to closed-phase analysis with short frames.
• each parameter set is referenced to the period centre rather than the pitchmark.
• the frame length is fixed, as this was found to give more consistent results than a pitch-dependent value.
• the stabilised covariance method would be preferable in terms of accuracy.
• the autocorrelation method is preferred as it is computationally efficient and guaranteed to give a stable synthesis filter.
• the next step is to inverse filter the speech on a pitch-synchronous basis.
• the parameters are interpolated to minimise transients due to large changes in parameter values at frame boundaries.
• the filter corresponds exactly to that obtained from the analysis.
• the filter is a weighted combination of the two filters obtained from the analysis.
• the interpolation is applied directly to the filter coefficients. This has been shown to produce less spectral distortion than other parameters (LAR' s, LSP' s etc), but is not guaranteed to give a stable interpolated filter. No instability problems have been encountered practice.
• ⁇ n is the value of a weighting function at sample n. a
• a r represent the parameter sets referenced to the nearest left and right period centres.
• the filter coefficients for the re-synthesis filter 105 are calculated in the same way as for inverse filtering. Modifications to pitch and durations mean that the sequence of filters and the period values will be different from those used in the analysis, but the interpolation still ensures a smooth variation in filter coefficients from sample-to-sample. For the first pitchmark in a voiced segment, filtering starts at the pitchmark and no interpolation is applied until the period centre is reached. For the last pitchmark in a voiced segment, the period is assumed to be the maximum allowed value for the purposes of positioning the analysis frame, and filtering stops at the pitchmark. These filtering conditions apply to both analysis and re- synthesis. When re-synthesising from the first pitchmark, the filter memory is initialised from preceding signal samples.
• a single-window overlap-add process may be used, with however a window width of less than two pitch period duration (preferably less than 1.7 e. g. in the range 1.25 - 1.6).
• the window function necessarily has a flat top, moreover it is preferably asymmetrically located relative to the pitch marks (preferably embracing a complete period between two pitchmarks).
• a typical window function is shown in Figure 5, with a flat top having a length equal to the synthesis pitch period and flanks of raised half-cosine or linear shape.
• This form of window is beneficial because a smaller temporal portion of the signal is constructed by the overlap-add process than with a longer window, and the asymmetric form places the overlap-add distortion towards the end of the pitch period where the speech energy is lower than immediately after the glottal excitation.
• Use of the resampling and multi-window pitch control is envisaged (as shown in Figure 2) as operating on the residual signal (to avoid distortion of the formants), however, the short asymmetric window method may also be employed without separation of the spectrum end excitation, but directly on the speech signal, in which case the analysis unit 102 and filters 103, 105 of Figure 2 would be omitted, the speech signals from the store 100 being fed directly to the pitch units 104, 104' .

Landscapes

• Engineering & Computer Science (AREA)
• Computational Linguistics (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Signal Processing Not Specific To The Method Of Recording And Reproducing (AREA)
• Electrophonic Musical Instruments (AREA)
• Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)

Priority Applications (2)

Applications Claiming Priority (5)

Publications (2)

Family

ID=26136991

Family Applications (1)

Country Status (10)

Families Citing this family (5)

Family Cites Families (1)

• 1994
  • 1994-03-18 SG SG1996003308A patent/SG43076A1/en unknown
• 1995
  • 1995-03-17 ES ES95911420T patent/ES2152390T3/es not_active Expired - Lifetime
  • 1995-03-17 AU AU18995/95A patent/AU692238B2/en not_active Ceased
  • 1995-03-17 WO PCT/GB1995/000588 patent/WO1995026024A1/en active IP Right Grant
  • 1995-03-17 NZ NZ282012A patent/NZ282012A/en not_active IP Right Cessation
  • 1995-03-17 EP EP95911420A patent/EP0750778B1/de not_active Expired - Lifetime
  • 1995-03-17 DE DE69519086T patent/DE69519086T2/de not_active Expired - Lifetime
  • 1995-03-17 CN CN95192141A patent/CN1144008A/zh active Pending
  • 1995-03-17 CA CA002185134A patent/CA2185134C/en not_active Expired - Fee Related
  • 1995-03-17 JP JP7524461A patent/JPH09510554A/ja not_active Ceased

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events