WO1994017517A1 - Waveform blending technique for text-to-speech system - Google Patents
Waveform blending technique for text-to-speech system Download PDFInfo
- Publication number
- WO1994017517A1 WO1994017517A1 PCT/US1994/000770 US9400770W WO9417517A1 WO 1994017517 A1 WO1994017517 A1 WO 1994017517A1 US 9400770 W US9400770 W US 9400770W WO 9417517 A1 WO9417517 A1 WO 9417517A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- frame
- digital
- subset
- samples
- sequence
- Prior art date
Links
- 238000002156 mixing Methods 0.000 title claims description 37
- 238000000034 method Methods 0.000 title description 29
- 239000000203 mixture Substances 0.000 claims abstract description 44
- 230000004044 response Effects 0.000 claims abstract description 20
- 239000000872 buffer Substances 0.000 claims description 30
- 230000015572 biosynthetic process Effects 0.000 claims description 7
- 238000003786 synthesis reaction Methods 0.000 claims description 7
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 230000000737 periodic effect Effects 0.000 claims description 2
- 230000002463 transducing effect Effects 0.000 claims 3
- MQJKPEGWNLWLTK-UHFFFAOYSA-N Dapsone Chemical group C1=CC(N)=CC=C1S(=O)(=O)C1=CC=C(N)C=C1 MQJKPEGWNLWLTK-UHFFFAOYSA-N 0.000 abstract description 45
- OENHQHLEOONYIE-JLTXGRSLSA-N β-Carotene Chemical compound CC=1CCCC(C)(C)C=1\C=C\C(\C)=C\C=C\C(\C)=C\C=C\C=C(/C)\C=C\C=C(/C)\C=C\C1=C(C)CCCC1(C)C OENHQHLEOONYIE-JLTXGRSLSA-N 0.000 abstract 1
- 239000013598 vector Substances 0.000 description 71
- 101100235787 Schizosaccharomyces pombe (strain 972 / ATCC 24843) pim1 gene Proteins 0.000 description 30
- 101150114015 ptr-2 gene Proteins 0.000 description 30
- 238000013139 quantization Methods 0.000 description 24
- 101150059273 PTR1 gene Proteins 0.000 description 20
- 230000006870 function Effects 0.000 description 19
- 238000007906 compression Methods 0.000 description 12
- 230000006835 compression Effects 0.000 description 12
- 238000007493 shaping process Methods 0.000 description 10
- 238000001914 filtration Methods 0.000 description 8
- 101100534231 Xenopus laevis src-b gene Proteins 0.000 description 7
- 230000003247 decreasing effect Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000003068 static effect Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 239000011800 void material Substances 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 101100534223 Caenorhabditis elegans src-1 gene Proteins 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000013016 damping Methods 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 101100215778 Neurospora crassa (strain ATCC 24698 / 74-OR23-1A / CBS 708.71 / DSM 1257 / FGSC 987) ptr-1 gene Proteins 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000001915 proofreading effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L13/00—Speech synthesis; Text to speech systems
- G10L13/06—Elementary speech units used in speech synthesisers; Concatenation rules
- G10L13/07—Concatenation rules
Definitions
- the present invention relates to systems for smoothly concatenating quasi-periodic waveforms, such as encoded diphone records used in translating text in a computer system to synthesized speech.
- text-to-speech systems stored text in a computer is translated to synthesized speech.
- this kind of system would have wide spread application if it were of reasonable cost.
- a text-to-speech system could be used for reviewing electronic mail remotely across a telephone line, by causing the computer storing the electronic mail to synthesize speech representing the electronic mail.
- such systems could be used for reading to people who are visually impaired.
- text-to-speech systems might be used to assist in proofreading a large document.
- text-to-speech systems an algorithm reviews an input text string, and translates the words in the text string into a sequence of diphones which must be translated into synthesized speech. Also, text- to-speech systems analyze the text based on word type and context to generate intonation control used for adjusting the duration of the sounds and the pitch of the sounds involved in the speech. Diphones consist of a unit of speech composed of the transition between one sound, or phoneme, and an adjacent sound, or phoneme. Diphones typically start at the center of one phoneme and end at the center of a neighboring phoneme. This preserves the transition between the sounds relatively well. American English based text-to-speech systems, depending on the particular implementation, use about fifty different sounds referred to as phones.
- the present invention provides an apparatus for concatenating a first digital frame with a second digital frame of quasi-periodic waveforms, such as the ending and beginning of adjacent diphone strings being concatenated to form speech.
- the system is based on determining an optimum blend point for the first and second digital frames in response to the magnitudes of samples in the first and second digital frames.
- the frames are then blended to generate a digital sequence representing a concatenation of the first and second frames, with reference to the optimum blend point. This has the effect of providing much better continuity in the blending or concatenation of diphones in text-to-speech systems than has been available in the prior art.
- the technique is applicable to concatenating any two quasi-periodic waveforms, commonly encountered in sound synthesis or speech, music, sound effects, or the like.
- the system operates by first computing an extended frame in response to the first digital frame, and then finding a subset of the extended frame which matches the second digital frame relatively well.
- the optimum blend point is then defined as a sample in the subset of the extended frame.
- the subset of the extended frame which matches the second digital frame relatively well is determined using a minimum average magnitude difference function over the samples in the subset.
- the blend point in this aspect comprises the first sample of the subset.
- the subset of the extended frame is combined with the second digital frame and concatenated with the beginning segment of the extended frame to produce the concatenate waveform.
- the present invention provides an apparatus for synthesizing speech in response to text.
- the system includes a translator, by which text is translated to a sequence of sound segment codes which identify diphones.
- a decoder is applied to the sequence of sound segment codes to produce strings of digital frames which represent diphones for respective sound segment codes in the sequence.
- a concatenator is provided by which a first digital frame at the ending of an identified string of digital frames for a particular sound segment code in the sequence is concatenated with a second digital frame at the beginning of an identified string of digital frames of an adjacent sound segment code in the sequence to produce a speech data sequence.
- the concatenating system includes a buffer to store samples of the first and second digital frames.
- Software, or other processing resources determine a blend point for the first and second digital frames and blend the first and second frames in response to the blend point to produce a concatenation of the first and second sound segments.
- An audio transducer is coupled to the concatenating system to generate synthesized speech in response to the speech data sequence.
- the resources that determine the optimum blend point include computing resources that compute an extended frame comprising a discontinuity smoothed concatenation of the first digital frame with a replica of the first digital frame. Further resources find a subset of the extended frame with a minimum average magnitude difference between the samples in the subset and in the second digital frame and define the optimum blend point as the first sample in the subset.
- the blending resources include software or other computing resources that supply a first set of samples derived from the first digital frame and the blend point as a first segment of the digital sequence.
- the second digital frame is combined with the subset of the extended frame, with emphasis on the subset of the extended frame in a starting sample and emphasis on the second digital frame in an ending sample to produce a second segment of the digital sequence.
- the first segment and second segment are combined produce the speech data sequence.
- the text-to-speech apparatus includes a processing module for adjusting the pitch and duration of the identified strings of digital frames in the speech data sequence in response to the input text.
- the decoder is based on a vector quantization technique which provides excellent quality compression with very small decoding resources required.
- FIG. 1 is a block diagram of a generic hardware platform incorporating the text-to-speech system of the present invention.
- Fig. 2 is a flow chart illustrating the basic text-to-speech routine according to the present invention.
- Fig. 3 illustrates the format of diphone records according to one embodiment of the present invention.
- Fig. 4 is a flow chart illustrating the encoder for speech data according to the present invention.
- Fig. 5 is a graph discussed in reference to the estimation of pitch filter parameters in the encoder of Fig. 4.
- Fig. 6 is a flow chart illustrating the full search used in the encoder of Fig. 4.
- Fig. 7 is a flow chart illustrating a decoder for speech data according to the present invention.
- Fig. 8 is a flow chart illustrating a technique for blending the beginning and ending of adjacent diphone records.
- Fig. 9 consists of a set of graphs referred to in explanation of the blending technique of Fig. 8.
- Fig. 10 is a graph illustrating a typical pitch versus time diagram for a sequence of frames of speech data.
- Fig. 1 1 is a flow chart illustrating a technique for increasing the pitch period of a particular frame.
- Fig. 12 is a set of graphs referred to in explanation of the technique of Fig. 1 1.
- Fig. 13 is a flow chart illustrating a technique for decreasing the pitch period of a particular frame.
- Fig. 14 is a set of graphs referred to in explanation of the technique of Fig. 1 3.
- Fig. 1 5 is a flow chart illustrating a technique for inserting a pitch period between two frames in a sequence.
- Fig. 1 6 is a set of graphs referred to in explanation of the technique of Fig. 1 5.
- Fig. 1 7 is a flow chart illustrating a technique for deleting a pitch period in a sequence of frames.
- Fig. 18 is a set of graphs referred to in explanation of the technique of Fig. 1 7.
- Figs. 1 and 2 provide a overview of a system incorporating the present invention.
- Fig. 3 illustrates the basic manner in which diphone records are stored according to the present invention.
- Figs. 4-6 illustrate the encoding methods based on vector quantization of the present invention.
- Fig. 7 illustrates the decoding algorithm according to the present invention.
- Figs. 8 and 9 illustrate a preferred technique for blending the beginning and ending of adjacent diphone records.
- Figs. 10-18 illustrate the techniques for controlling the pitch and duration of sounds in the text-to-speech system.
- Fig. 1 illustrates a basic microcomputer platform incorporating a text-to-speech system based on vector quantization according to the present invention.
- the platform includes a central processing unit 10 coupled to a host system bus 1 1 .
- a keyboard 1 2 or other text input device is provided in the system.
- a display system 13 is coupled to the host system bus.
- the host system also includes a non-volatile storage system such as a disk drive 14.
- the system includes host memory 15.
- the host memory includes text-to-speech (TTS) code, including encoded voice tables, buffers, and other host memory.
- the text-to-speech code is used to generate speech data for supply to an audio output module 16 which includes a speaker 17.
- the code also includes an optimum blend point, diphone concatenation routine as described in detail with reference to Figs. 8 and 9.
- the encoded voice tables include a TTS dictionary which is used to translate text to a string of diphones. Also included is a diphone table which translates the diphones to identified strings of quantization vectors.
- a quantization vector table is used for decoding the sound segment codes of the diphone table into the speech data for audio output.
- the system may include a vector quantization table for encoding which is loaded into the host memory 15 when necessary.
- the platform illustrated in Fig. 1 represents any generic microcomputer system, including a Macintosh based system, an DOS based system, a UNIX based system or other types of microcomputers.
- the text-to-speech code and encoded voice tables according to the present invention for decoding occupy a relatively small amount of host memory 15.
- a text-to-speech decoding system according to the present invention may be implemented which occupies less than 640 kilobytes of main memory, and yet produces high quality, natural sounding synthesized speech.
- the basic algorithm executed by the text-to-speech code is illustrated in Fig. 2.
- the system first receives the input text (block 20).
- the input text is translated to diphone strings using the TTS dictionary (block 21 ).
- the input text is analyzed to generate intonation control data, to control the pitch and duration of the diphones making up the speech (block 22).
- the diphone strings are decompressed to generate vector quantized data frames (block 23).
- the vector quantized (VQ) data frames are produced, the beginnings and endings of adjacent diphones are blended to smooth any discontinuities (block 24).
- the duration and pitch of the diphone VQ data frames are adjusted in response to the intonation control data (block 25 and 26).
- the speech data is supplied to the audio output system for real time speech production (block 27).
- an adaptive post filter may be applied to further improve the speech quality.
- the TTS dictionary can be implemented using any one of a variety of techniques known in the art.
- diphone records are implemented as shown in Fig. 3 in a highly compressed format.
- records for a left diphone 30 and a record for a right diphone 31 are shown.
- the record for the left diphone 30 includes a count 32 of the number NL of pitch periods in the diphone.
- a pointer 33 is included which points to a table of length NL storing the number LP. for each pitch period, i goes from 0 to NL-1 of pitch values for corresponding compressed frame records.
- pointer 34 is included to a table 36 of ML vector quantized compressed speech records, each having a fixed set length of encoded frame size related to nominal pitch of the encoded speech for the left diphone.
- the nominal pitch is based upon the average number of samples for a given pitch period for the speech data base.
- the encoder routine is illustrated in Fig. 4.
- the encoder accepts as input a frame s of speech data.
- the speech samples are represented as 12 or 16 bit two's complement numbers, sampled at 22,252 Hz.
- This data is divided into non-overlapping frames s having a length of N, where N is referred to as the frame size.
- the value of N depends on the nominal pitch of the speech data. If the nominal pitch of the recorded speech is less than 165 samples (or 135 Hz), the value of N is chosen to be 96. Otherwise a frame size of 160 is used.
- the encoder transforms the N-point data sequence s into a byte stream of shorter length, which depends on the desired compression rate.
- the output byte stream can be as short as 12 eight bit bytes.
- a block diagram of the encoder is shown in Fig. 4.
- the routine begins by accepting a frame s (block 50).
- signal s is passed through a high pass filter.
- a difference equation used in a preferred system to accomplish this is set out in Equation 1 for 0 n ⁇ N.
- x s - s ⁇ + 0.999 *x - n n n-1 n-1
- Equation 1 The value x is the "offset free" signal.
- the variables s 1 and x _, are initialized to zero for each diphone and are subsequently updated using the relation of Equation 2.
- Equation 2 This step can be referred to as offset compensation or DC removal (block 51 ).
- Equation 3 The linear prediction filtering of Equation 3 produces a frame y
- the filter parameter which is equal to 0.875 in Equation 3, will have to be modified if a different speech sampling rate is used.
- the value of x 1 is initialized to zero for each diphone, but will be updated in the step of inverse linear prediction filtering (block 60) as described below.
- filter types including, for instance, an adaptive filter in which the filter parameters are dependent on the diphones to be encoded, or higher order filters.
- Equation 3 The sequence y produced by Equation 3 is then utilized to determine an optimum pitch value, P__ t # and an associated gain factor,
- PBUF is a pitch buffer of size P , which is initialized to zero, ⁇ max and updated in the pitch buffer update block 59 as described below.
- P ⁇ is the value of P for which Coh(P) is maximum and s (P) is opt xy positive.
- the range of P considered depends on the nominal pitch of the speech being coded. The range is (96 to 350) if the frame size is equal to 96 and is (1 60 to 414) if the frame size is equal to 160. P is
- the parameter P can be represented using 8 bits.
- the buffer PBUF is represented by the sequence 100 and the frame y is represented by the sequence 101 .
- the frame y is represented by the sequence 101 .
- the vector y 101 matches as closely as possible a corresponding segment of similar length in PBUF 100.
- Equation 8 ⁇ is quantized to four bits, so that the quantized value of ⁇ can range from 1 /1 6 to 1 , in steps of 1 /1 6.
- a pitch filter is applied (block 54).
- the long term correlations in the pre-emphasized speech data y are removed using the relation of Equation 9.
- r y 0 .
- a scaling parameter G is generated using a block gain estimation routine (block 55).
- the residual signal r is rescaled.
- the scaling parameter, G is obtained by first determining the largest magnitude of the signal r and quantizing it using a 7-level quantizer.
- the parameter G can take one of the following 7 values: 256, 51 2, 1024, 2048, 4096, 8192, and 1 6384. The consequence of choosing these quantization levels is that the rescaling operation can be implemented using only shift operations.
- the routine proceeds to residual coding using a full search vector quantization code (block 56).
- the n point sequence r is divided into non-overlapping blocks of length M, where M is referred to as the "vector size".
- M sample blocks b.. are created, where i is an index from zero to M-1 on the block number, and j is an index from zero to N/M-1 on the sample within the block.
- Each block may be defined as set out in Equation 10.
- b.. r. .. , . , (0 ⁇ i ⁇ N/M and j ⁇ 0 ⁇ M) ij Mi +j J
- Each of these M sample blocks b.. will be coded into an 8 bit number using vector quantization.
- M depends on the desired compression ratio. For example, with M equal to 1 6, very high compression is achieved (i.e., 1 6 residual samples are coded using only
- the length of the compressed speech records will be longer.
- the value M can take values 2,
- a sequence of quantization vectors is identified (block 120).
- the components of block b.. are passed through a noise shaping filter and scaled as set out in Equation 1 1 (block 121 ).
- w. 0.875 * w. , - 0.5 * w. prepare + 0.4375 * w. + b.., J J-1 J-2 J-3 ij'
- v.. is the jth component of the vector v.
- w is the w
- wlaut andrang are the states of the noise shaping filter and are initialized to zero for each diphone.
- the filter coefficients are chosen to shape the quantization noise spectra in order to improve the subjective quality of the decompressed speech. After each vector is coded and decoded, these states are updated as described below with reference to blocks 1 24-126.
- the routine finds a pointer to the best match in a vector quantization table (block 122).
- the vector quantization table 123 consists of a sequence of vectors C Q through C 255 (block 123).
- the vector v. is compared against 256 M-point vectors, which are precomputed and stored in the code table 123.
- the vector v. is compared against 256 M-point vectors, which are precomputed and stored in the code table 123.
- the closest vector C . can also be determined efficiently using the technique of Equation 1 3.
- Equation 13 the value vT represen _t.s _ t._h.e _ t,ranspose o .f_. _ t_._h__e vector v, and "•" represents the inner product operation in the inequality.
- decoding vector table 125 which consists of a sequence of vectors QV .
- the decoded samples corresponding to the vector b. which is produced at step 55 of Fig. 4, is the M-point vector (1 /G) * QV ..
- the vector C is related to qi p the vector QV by the noise shaping filter operation of Equation 1 1.
- Fig. 6 thus includes noise compensated quantization vectors.
- the decoding vector of the pointer to the vector b. is accessed (block 124). That decoding vector is used for filter and PBUF updates (block 126).
- the error vector (b.-QV .) is passed i i qi through the noise shaping filter as shown in Equation 14.
- W. 0.875 * W. 1 - 0.5 * W. 0 + 0.4375 * W. , + [b.. -
- Equation 14 the value QV .(j) represents the j component of the decoding vector QV ..
- the noise shaping filter states for the next block are updated as shown in Equation 15.
- Equation 1 5 This coding and decoding is performed for all of the N/M sub- blocks to obtain N/M indices to the decoding vector table 1 25.
- This string of indices Q , for n going from zero to N/M-1 represent identifiers for a string of decoding vectors for the residual signal r .
- Fig. 4 four parameters identifying the speech data are stored (block 57). In a preferred system, they are stored in a structure as described with respect to Fig. 3 where the structure of the frame can be characterized as follows:
- the encoder continues decoding the data being encoded in order to update the filter and PBUF values.
- the first step involved in this is an inverse pitch filter (block 58).
- the inverse filter is implemented as set out in Equation 1 6.
- y' r' + ⁇ * PBUF D D + , , 0 ⁇ n ⁇ N.
- the pitch buffer is updated (block 59) with the output of the inverse pitch filter.
- the pitch buffer PBUF is updated as set out in
- PBUF n n PBUF (,n + N.) 0 ⁇ n ⁇ (P m ma a x v - N)
- linear prediction filter parameters are updated using an inverse linear prediction filter step (block 60).
- the output of the inverse pitch filter is passed through a first order inverse linear prediction filter to obtain the decoded speech.
- Equation 1 8 In Equation 1 8, x' is the decompressed speech. From this, the value of x ' for the next frame is set to the value x * . for use in the step of block 52.
- Fig. 7 illustrates the decoder routine.
- the decoder module accepts as input (N/M) + 2 bytes of data, generated by the encoder module, and applies as output N samples of speech.
- the value of N depends on the nominal pitch of the speech data and the value of M depends on the desired compression ratio.
- FIG. 7 A block diagram of the encoder is shown in Fig. 7.
- the routine starts by accepting diphone records at block 200.
- the first step involves parsing the parameters G, ⁇ , P , and the vector quantization string Q (block 201 ).
- the residual signal r' is decoded (block 202). This involves accessing and concatenating the decoding vectors for the vector quantization string as shown schematically at block 203 with access to the decoding vector table
- Equation 19 SPBUF is a synthesizer pitch buffer of length P initialized as zero for each diphone, as described above with respect to the encoder pitch buffer PBUF.
- the synthesis pitch buffer is updated (block 205).
- the sequence y' is applied to an inverse linear prediction filtering step (block 206).
- the output of the inverse pitch filter y' is passed through a first order inverse linear prediction filter to obtain the decoded speech.
- Equation 21 the vector x' corresponds to the decompressed speech.
- This filtering operation can be implemented using simple shift operations without requiring any multiplication. Therefore, it executes very quickly and utilizes a very small amount of the host computer resources.
- Encoding and decoding speech according to the algorithms described above provide several advantages over prior art systems.
- this technique offers higher speech compression rates with decoders simple enough to be used in the implementation of software only text-to-speech systems on computer systems with low processing power.
- the technique offers a very flexible trade-off between the compression ratio and synthesizer speech quality. A high-end computer system can opt for higher quality synthesized speech at the expense of a bigger RAM memory requirement.
- the right boundary of the first diphone and the left boundary of the second diphone correspond to the same phoneme in most situations, they are expected to be similar looking at the point of concatenation. However, because the two diphone codings are extracted from different context, they will not look identical.
- This blending technique is applied to eliminate discontinuities at the point of concatenation.
- the last frame, referring here to one pitch period, of the left diphone is designated L (0 ⁇ n ⁇ PL) at the top of the page.
- the first frame (pitch period) of the right diphone is designated R (0 ⁇ n ⁇ PR).
- the blending of L and R according to the present invention will alter these two pitch periods only and is performed as discussed with reference to Fig. 8.
- the waveforms in Fig. 9 are chosen to illustrate the algorithm, and may not be representative of real speech data.
- the algorithm as shown in Fig. 8 begins with receiving the left and right diphone in a sequence (block 300). Next, the last frame of the left diphone is stored in the buffer L (block 301 ). Also, the first frame of the right diphone is stored in buffer R (block 302).
- Equation 22 The extended sequence El (0 ⁇ n ⁇ PL) is obtained in the first step as shown in Equation 22:
- the extended sequence El is substantially equal to L on the left hand side, has a smoothed region beginning at the point PL and converges on the original shape of L toward the point 2PL. If L was perfectly periodic, then
- This function is computed for values of p in the range of 0 to PL-
- W is the window size for the AMDF computation.
- the waveforms are blended (block 306).
- WR is shown in Fig. 9 at the R trace which is lined up with P .
- the value of El is emphasized.
- the value of R is emphasized.
- the length PL of L is altered as needed to ensure that when the modified L and R are concatenated, the waveforms are n n as continuous as possible.
- the length P'L is set to P _ if P _ is p a opt opt greater than PL/2. Otherwise, the length P'L is equal to W + P and the sequence L is equal to El for 0 ⁇ n ⁇ (P'L-1 ).
- the blending ramp beginning at P is set out in Equation 25:
- R R W ⁇ n ⁇ PR n n
- Equation 25 the sequences L and R are windowed and added to get the blended R .
- the beginning of L and the ending of R are preserved to prevent any discontinuities with adjacent frames.
- This blending technique is believed to minimize blending noise in synthesized speech produced by any concatenated speech synthesis.
- a text analysis program analyzes the text and determines the duration and pitch contour of each phone that needs to be synthesized and generates intonation control signals.
- a typical control for a phone will indicate that a given phoneme, such as AE, should have a duration of 200 milliseconds and a pitch should rise linearly from 220Hz to 300Hz. This requirement is graphically shown in Fig. 1 0.
- T equals the desired duration (e.g. 200 milliseconds) of the phoneme.
- the frequency f. is the desired beginning pitch in Hz.
- the frequency f is the desired ending pitch in Hz.
- P,-...,P fi indicate the number of samples of each frame to achieve the desired pitch frequencies f. , f supplement...,f fi .
- the pitch period for a lower frequency period of the phoneme is longer than the pitch period for a higher frequency period of the phoneme. If the nominal frequency were P Palm, then the algorithm would be required to lengthen the pitch period for frames P_. and P ? and decrease the pitch periods for frames P . , P p . and P fi . Also, the given duration T of the phoneme will indicate how many pitch periods should be inserted or deleted from the encoded phoneme to achieve the desired duration period.
- Figs. 1 1 through 18 illustrate a preferred implementation of such algorithms.
- Fig. 1 1 illustrates an algorithm for increasing the pitch period, with reference to the graphs of Fig. 12.
- the algorithm begins by receiving a control to increase the pitch period to N + ⁇ , where N is the pitch period of the encoded frame. (Block 350).
- the pitch period data is stored in a buffer x (block 351 ).
- x is shown in n n Fig. 1 2 at the top of the page.
- the weighting function WL is constant from the first sample to sample ⁇ , and decreases from ⁇ to N.
- Equation 27 a weighting function WR is applied to x (block 353) as can be seen in the Fig. 1 2. This weighting function is executed as shown in Equation 27:
- R x * (n + 1 )/(M + 1 ) for O ⁇ n ⁇ N- ⁇ n n + ⁇
- R x , ⁇ for N- ⁇ ⁇ n ⁇ N. n n + ⁇
- Equation 27 As can be seen in Fig. 1 2, the weighting function WR increases from 0 to N- ⁇ and remains constant from N- ⁇ to N.
- the resulting waveforms L and R are shown conceptually in Fig. 1 2. As can be seen, L maintains the beginning of the sequence x , while R maintains the ending of the data x . a n
- Fig. 12 This is graphically shown in Fig. 12 by placing R shifted by ⁇ below L .
- the combination of L and R shifted by ⁇ is shown to be y at the n n n 7 n bottom of Fig. 1 2.
- the pitch period for y is N + ⁇ .
- the beginning of y is the same as the beginning of x
- the ending of y is substantially the same as the ending of x . This maintains continuity with adjacent frames in the sequence, and accomplishes a smooth transition while extending the pitch period of the data.
- Equation 28 is executed with the assumption that L is 0, for n ⁇ N, and R is 0 for n ⁇ 0. This is illustrated pictorially in Fig. 1 2.
- Equation 29 This results in a new pitch period having a pitch period of N + ⁇ .
- the algorithm begins with a control signal indicating that the pitch period must be decreased to N- ⁇ .
- the first step is to store two consecutive pitch periods in the buffer x (block 401 ).
- the buffer x as can be seen in Fig. 14 consists of two consecutive pitch periods, with the period N. being the length of the first pitch period, and N being the length of the second pitch period.
- two sequences L and R are conceptually created using weighting functions WL and WR (blocks 402 and 403).
- the weighting function WL emphasizes the beginning of the first pitch period
- the weighting function WR emphasizes the ending of the second pitch period.
- R n x n * (n-N. I + W- ⁇ + 1 )/ ⁇ W+ 1 ) for N. I-W + ⁇ n ⁇ N,I -l- ⁇
- R x for N, + ⁇ n ⁇ N, + N
- Equation 31 In these equations, ⁇ is equal to the difference between N. and the desired pitch period N ..
- the value W is equal to 2* ⁇ , unless 2* ⁇ is greater than N ., in which case W is equal to N ..
- pitch modified sequence y (block 404).
- Equation 32 When a pitch period is decreased, two consecutive pitch periods of data are affected, even though only the length of one pitch period is changed. This is done because pitch periods are divided at places where short-term energy is the lowest within a pitch period. Thus, this strategy affects only the low energy portion of the pitch periods. This minimizes the degradation in speech quality due to the pitch modification. It should be appreciated that the drawings in Fig. 14 are simplified and do not represent actual pitch period data. An efficient implementation of this scheme, which requires at most one multiply per sample, is set out in Equations 33 and 34.
- y n ⁇ n + [x n ⁇ -x n ]*(n-N
- Equation 33 The second pitch period of length N is generated as shown in
- the sequence L is essentially equal to the first pitch period until the point N.-W. At that point, a decreasing ramp WL is applied to the signal to dampen the effect of the first pitch period.
- the weighting function WR begins at the point N.-W + ⁇ and applies an increasing ramp to the sequence x until the point N. + ⁇ . From that point, a constant value is applied. This has the effect of damping the effect of the right sequence and emphasizing the left during the beginning of the weighting functions, and generating a ending segment which is substantially equal to the ending segment of x emphasizing the right sequence and damping the left.
- the resulting waveform y is substantially equal to the beginning of x at the beginning of the sequence, at the point
- the algorithm begins by receiving a control signal to insert a pitch period between frames L and R (block 450).
- both L and R n n n n are stored in the buffer (block 451 ), where L and R are two adjacent n n pitch periods of a voice diphone. (Without loss of generality, it is assumed for the description that the two sequences are of equal lengths
- the algorithm proceeds by generating a left vector WL(L ), essentially applying to the increasing ramp WL to the signal L . (Block 452).
- a right vector WR (R ) is generated using the weighting vector
- the algorithm operates to modify the pitch period L which precedes R (to be deleted) so that it resembles R , as n approaches N.
- L' L + (R - L ) * [(n + 1 )/(N + 1 )] 0 ⁇ n ⁇ N-1 n n n n n
- Equation 36 the resulting sequence L' is shown at the bottom of Fig. 18.
- Equation 36 applies a weighting function WL to the sequence L (block 502). This emphasizes the beginning of the sequence L as shown.
- a right vector WR (R ) is generated by applying a weighting vector WR to the sequence R that emphasizes the ending of R (block 503).
- WL (L ) and WR (R ) are blended to create the resulting vector n n
- the present invention presents a software only text- to-speech system which is efficient, uses a very small amount of memory, and is portable to a wide variety of standard microcomputer platforms. It takes advantage of knowledge about speech data, and to create a speech compression, blending, and duration control routine which produces very high quality speech with very little computational resources.
- a source code listing of the software for executing the compression and decompression, the blending, and the duration and pitch control routines is provided in the Appendix as an example of a preferred embodiment of the present invention.
- #include ⁇ math.h> include ⁇ StdLib.h>
- PBUF SIZE 440 static float oc_state[2], nsf_state[NSF_ORDER+ 1]; static short pstatelPORDER + 1], dstate[PORDER+ 1]; static short AnaPbuf[PBUF_SIZE]; static short vsize, cbook_size, bs_size;
- GetPitchFilterPars (x, len, pbuf, min pitch, max_pitch, pitch, beta) float *beta; short *x, *pbuf; short min_pitch, max_pitch; short len; unsigned int * pitch;
- ⁇ syy + (*ptr) * (*ptr); ptr + + ;
- ⁇ syy syy - pbuf [PBUF_SIZE - j + len - 1 ] * pbuf [PBUF_SIZE - j + len - 1 ]
- PitchFilter(data, len, pbuf, pitch, ibeta) float *data; short ibeta; short *pbuf; short len; unsigned int pitch;
- VQCoder float *x, float *nsf_state, short len, struct frame *bs
- Decoded data is 14-bits, convert to 16 bits */ if (lshift_count)
- PitchFilter preemp_xn, frame_size, AnaPbuf, pitch, ibeta
- VQCoder preemp_xn, nsf_state, frame_size, bs
- ⁇ bs_size frame_size / vsize + 2;
- Encoder(input + i, frame_size, min_pitch, max_pitch, output -t-j); j + bs_size;
- PFILT_ORDER 8 struct frame ⁇ unsigned gcode : 4; unsigned bcode : 4; unsigned pitch : 8; unsigned char vqcodetl;
- *ptr++ 0.074539 * 32768 + 0.5;
- *sptr14- 4- *bptr+ - ; ⁇ ⁇
- *sptr14- 4- 4- ((*sptr2- 4-) > > 4); break; case 1 : while (-count)
- *sptr14-4- 4- ((3 * (*sptr2+ +)) > > 4); break; case 3: while (-count)
- *sptr14-4- 4- ((5 * (*sptr2+ +)) > > 4); break; case 5: while (-count)
- *sptr14-4- 4- ((3 * (*sptr2-t- •+-)) >> 2); break; case 12: while (-count)
- *sptr14- 4- 4- (((tmp ⁇ ⁇ 4) - 3 * tmp) > > 4);
- a module for blending two diphones */ typedef struct ⁇ short Iptr, pitch; short weight, weightjnc; ⁇ bstate; void SnBlend(pitchp lp, pitehp rp, short cur_tot, short tot, short type, bstate *bs)
- *ptr1 *ptr2 4- (i * (*ptr1 - *ptr2)) > > 4; ptrl 4- 4-; ptr24- 4- ;
- This module is used to change pitch information in the concatenated speech */ // This routine depends on the desired length (deslen) being at least half // and no more than twice the actual length (len). void SnChangePitchfshort * buf, short *next, short len, short deslen, short Ivoc, short rvoc, short dosmooth)
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Electrophonic Musical Instruments (AREA)
Abstract
Description
Claims
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE69420547T DE69420547T2 (en) | 1993-01-21 | 1994-01-18 | WAVEFORM MIXING METHOD FOR TEXT-TO-LANGUAGE SYSTEM |
EP94907854A EP0680652B1 (en) | 1993-01-21 | 1994-01-18 | Waveform blending technique for text-to-speech system |
AU61261/94A AU6126194A (en) | 1993-01-21 | 1994-01-18 | Waveform blending technique for text-to-speech system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US08/007,621 US5490234A (en) | 1993-01-21 | 1993-01-21 | Waveform blending technique for text-to-speech system |
US08/007,621 | 1993-01-21 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO1994017517A1 true WO1994017517A1 (en) | 1994-08-04 |
Family
ID=21727228
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US1994/000770 WO1994017517A1 (en) | 1993-01-21 | 1994-01-18 | Waveform blending technique for text-to-speech system |
Country Status (6)
Country | Link |
---|---|
US (1) | US5490234A (en) |
EP (1) | EP0680652B1 (en) |
AU (1) | AU6126194A (en) |
DE (1) | DE69420547T2 (en) |
ES (1) | ES2136191T3 (en) |
WO (1) | WO1994017517A1 (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1996032711A1 (en) * | 1995-04-12 | 1996-10-17 | British Telecommunications Public Limited Company | Waveform speech synthesis |
EP0805433A2 (en) * | 1996-04-30 | 1997-11-05 | Microsoft Corporation | Method and system of runtime acoustic unit selection for speech synthesis |
WO1999007132A1 (en) * | 1997-07-31 | 1999-02-11 | British Telecommunications Public Limited Company | Generation of voice messages |
US7805307B2 (en) | 2003-09-30 | 2010-09-28 | Sharp Laboratories Of America, Inc. | Text to speech conversion system |
KR20180082521A (en) * | 2016-01-14 | 2018-07-18 | 텐센트 테크놀로지(센젠) 컴퍼니 리미티드 | Method and apparatus for processing audio data |
Families Citing this family (148)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5890104A (en) * | 1992-06-24 | 1999-03-30 | British Telecommunications Public Limited Company | Method and apparatus for testing telecommunications equipment using a reduced redundancy test signal |
WO1995001011A1 (en) * | 1993-06-21 | 1995-01-05 | British Telecommunications Public Limited Company | Method and apparatus for testing telecommunications equipment using a reduced redundancy test signal |
JP2782147B2 (en) * | 1993-03-10 | 1998-07-30 | 日本電信電話株式会社 | Waveform editing type speech synthesizer |
US6502074B1 (en) * | 1993-08-04 | 2002-12-31 | British Telecommunications Public Limited Company | Synthesising speech by converting phonemes to digital waveforms |
US5987412A (en) * | 1993-08-04 | 1999-11-16 | British Telecommunications Public Limited Company | Synthesising speech by converting phonemes to digital waveforms |
US5832442A (en) * | 1995-06-23 | 1998-11-03 | Electronics Research & Service Organization | High-effeciency algorithms using minimum mean absolute error splicing for pitch and rate modification of audio signals |
US5751907A (en) * | 1995-08-16 | 1998-05-12 | Lucent Technologies Inc. | Speech synthesizer having an acoustic element database |
DE69925932T2 (en) * | 1998-11-13 | 2006-05-11 | Lernout & Hauspie Speech Products N.V. | LANGUAGE SYNTHESIS BY CHAINING LANGUAGE SHAPES |
US6202049B1 (en) * | 1999-03-09 | 2001-03-13 | Matsushita Electric Industrial Co., Ltd. | Identification of unit overlap regions for concatenative speech synthesis system |
US6385581B1 (en) | 1999-05-05 | 2002-05-07 | Stanley W. Stephenson | System and method of providing emotive background sound to text |
AU7991900A (en) * | 1999-10-04 | 2001-05-10 | Joseph E. Pechter | Method for producing a viable speech rendition of text |
US8645137B2 (en) | 2000-03-16 | 2014-02-04 | Apple Inc. | Fast, language-independent method for user authentication by voice |
DE10033104C2 (en) * | 2000-07-07 | 2003-02-27 | Siemens Ag | Methods for generating statistics of phone durations and methods for determining the duration of individual phones for speech synthesis |
DE60127274T2 (en) * | 2000-09-15 | 2007-12-20 | Lernout & Hauspie Speech Products N.V. | FAST WAVE FORMS SYNCHRONIZATION FOR CHAINING AND TIME CALENDAR MODIFICATION OF LANGUAGE SIGNALS |
US7280969B2 (en) * | 2000-12-07 | 2007-10-09 | International Business Machines Corporation | Method and apparatus for producing natural sounding pitch contours in a speech synthesizer |
US20040064308A1 (en) * | 2002-09-30 | 2004-04-01 | Intel Corporation | Method and apparatus for speech packet loss recovery |
US20040102964A1 (en) * | 2002-11-21 | 2004-05-27 | Rapoport Ezra J. | Speech compression using principal component analysis |
KR100486734B1 (en) | 2003-02-25 | 2005-05-03 | 삼성전자주식회사 | Method and apparatus for text to speech synthesis |
US20050075865A1 (en) * | 2003-10-06 | 2005-04-07 | Rapoport Ezra J. | Speech recognition |
US7409347B1 (en) * | 2003-10-23 | 2008-08-05 | Apple Inc. | Data-driven global boundary optimization |
US7643990B1 (en) * | 2003-10-23 | 2010-01-05 | Apple Inc. | Global boundary-centric feature extraction and associated discontinuity metrics |
US20050102144A1 (en) * | 2003-11-06 | 2005-05-12 | Rapoport Ezra J. | Speech synthesis |
WO2005071663A2 (en) * | 2004-01-16 | 2005-08-04 | Scansoft, Inc. | Corpus-based speech synthesis based on segment recombination |
US8677377B2 (en) | 2005-09-08 | 2014-03-18 | Apple Inc. | Method and apparatus for building an intelligent automated assistant |
US20070106513A1 (en) * | 2005-11-10 | 2007-05-10 | Boillot Marc A | Method for facilitating text to speech synthesis using a differential vocoder |
GB2433150B (en) * | 2005-12-08 | 2009-10-07 | Toshiba Res Europ Ltd | Method and apparatus for labelling speech |
US8027377B2 (en) * | 2006-08-14 | 2011-09-27 | Intersil Americas Inc. | Differential driver with common-mode voltage tracking and method |
US9318108B2 (en) | 2010-01-18 | 2016-04-19 | Apple Inc. | Intelligent automated assistant |
GB0704772D0 (en) * | 2007-03-12 | 2007-04-18 | Mongoose Ventures Ltd | Aural similarity measuring system for text |
US20090299731A1 (en) * | 2007-03-12 | 2009-12-03 | Mongoose Ventures Limited | Aural similarity measuring system for text |
US8977255B2 (en) | 2007-04-03 | 2015-03-10 | Apple Inc. | Method and system for operating a multi-function portable electronic device using voice-activation |
US8321222B2 (en) * | 2007-08-14 | 2012-11-27 | Nuance Communications, Inc. | Synthesis by generation and concatenation of multi-form segments |
JP2009109805A (en) * | 2007-10-31 | 2009-05-21 | Toshiba Corp | Speech processing apparatus and method of speech processing |
US9330720B2 (en) | 2008-01-03 | 2016-05-03 | Apple Inc. | Methods and apparatus for altering audio output signals |
US8996376B2 (en) | 2008-04-05 | 2015-03-31 | Apple Inc. | Intelligent text-to-speech conversion |
US10496753B2 (en) | 2010-01-18 | 2019-12-03 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US20100030549A1 (en) | 2008-07-31 | 2010-02-04 | Lee Michael M | Mobile device having human language translation capability with positional feedback |
US8712776B2 (en) * | 2008-09-29 | 2014-04-29 | Apple Inc. | Systems and methods for selective text to speech synthesis |
US8396714B2 (en) * | 2008-09-29 | 2013-03-12 | Apple Inc. | Systems and methods for concatenation of words in text to speech synthesis |
US8352272B2 (en) * | 2008-09-29 | 2013-01-08 | Apple Inc. | Systems and methods for text to speech synthesis |
US8352268B2 (en) * | 2008-09-29 | 2013-01-08 | Apple Inc. | Systems and methods for selective rate of speech and speech preferences for text to speech synthesis |
WO2010067118A1 (en) | 2008-12-11 | 2010-06-17 | Novauris Technologies Limited | Speech recognition involving a mobile device |
US8380507B2 (en) * | 2009-03-09 | 2013-02-19 | Apple Inc. | Systems and methods for determining the language to use for speech generated by a text to speech engine |
US9858925B2 (en) | 2009-06-05 | 2018-01-02 | Apple Inc. | Using context information to facilitate processing of commands in a virtual assistant |
US20120309363A1 (en) | 2011-06-03 | 2012-12-06 | Apple Inc. | Triggering notifications associated with tasks items that represent tasks to perform |
US10241752B2 (en) | 2011-09-30 | 2019-03-26 | Apple Inc. | Interface for a virtual digital assistant |
US10241644B2 (en) | 2011-06-03 | 2019-03-26 | Apple Inc. | Actionable reminder entries |
US9431006B2 (en) | 2009-07-02 | 2016-08-30 | Apple Inc. | Methods and apparatuses for automatic speech recognition |
US10705794B2 (en) | 2010-01-18 | 2020-07-07 | Apple Inc. | Automatically adapting user interfaces for hands-free interaction |
US10276170B2 (en) | 2010-01-18 | 2019-04-30 | Apple Inc. | Intelligent automated assistant |
US10679605B2 (en) | 2010-01-18 | 2020-06-09 | Apple Inc. | Hands-free list-reading by intelligent automated assistant |
US10553209B2 (en) | 2010-01-18 | 2020-02-04 | Apple Inc. | Systems and methods for hands-free notification summaries |
DE202011111062U1 (en) | 2010-01-25 | 2019-02-19 | Newvaluexchange Ltd. | Device and system for a digital conversation management platform |
US8682667B2 (en) | 2010-02-25 | 2014-03-25 | Apple Inc. | User profiling for selecting user specific voice input processing information |
US10762293B2 (en) | 2010-12-22 | 2020-09-01 | Apple Inc. | Using parts-of-speech tagging and named entity recognition for spelling correction |
JP5743625B2 (en) * | 2011-03-17 | 2015-07-01 | 株式会社東芝 | Speech synthesis editing apparatus and speech synthesis editing method |
US9262612B2 (en) | 2011-03-21 | 2016-02-16 | Apple Inc. | Device access using voice authentication |
US10057736B2 (en) | 2011-06-03 | 2018-08-21 | Apple Inc. | Active transport based notifications |
US8994660B2 (en) | 2011-08-29 | 2015-03-31 | Apple Inc. | Text correction processing |
US10134385B2 (en) | 2012-03-02 | 2018-11-20 | Apple Inc. | Systems and methods for name pronunciation |
US9483461B2 (en) | 2012-03-06 | 2016-11-01 | Apple Inc. | Handling speech synthesis of content for multiple languages |
US9280610B2 (en) | 2012-05-14 | 2016-03-08 | Apple Inc. | Crowd sourcing information to fulfill user requests |
US9721563B2 (en) | 2012-06-08 | 2017-08-01 | Apple Inc. | Name recognition system |
US9495129B2 (en) | 2012-06-29 | 2016-11-15 | Apple Inc. | Device, method, and user interface for voice-activated navigation and browsing of a document |
US9576574B2 (en) | 2012-09-10 | 2017-02-21 | Apple Inc. | Context-sensitive handling of interruptions by intelligent digital assistant |
US9547647B2 (en) | 2012-09-19 | 2017-01-17 | Apple Inc. | Voice-based media searching |
DE212014000045U1 (en) | 2013-02-07 | 2015-09-24 | Apple Inc. | Voice trigger for a digital assistant |
US9368114B2 (en) | 2013-03-14 | 2016-06-14 | Apple Inc. | Context-sensitive handling of interruptions |
WO2014144579A1 (en) | 2013-03-15 | 2014-09-18 | Apple Inc. | System and method for updating an adaptive speech recognition model |
CN105027197B (en) | 2013-03-15 | 2018-12-14 | 苹果公司 | Training at least partly voice command system |
WO2014197336A1 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for detecting errors in interactions with a voice-based digital assistant |
US9582608B2 (en) | 2013-06-07 | 2017-02-28 | Apple Inc. | Unified ranking with entropy-weighted information for phrase-based semantic auto-completion |
WO2014197334A2 (en) | 2013-06-07 | 2014-12-11 | Apple Inc. | System and method for user-specified pronunciation of words for speech synthesis and recognition |
WO2014197335A1 (en) | 2013-06-08 | 2014-12-11 | Apple Inc. | Interpreting and acting upon commands that involve sharing information with remote devices |
US10176167B2 (en) | 2013-06-09 | 2019-01-08 | Apple Inc. | System and method for inferring user intent from speech inputs |
AU2014278592B2 (en) | 2013-06-09 | 2017-09-07 | Apple Inc. | Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant |
EP3008964B1 (en) | 2013-06-13 | 2019-09-25 | Apple Inc. | System and method for emergency calls initiated by voice command |
WO2015020942A1 (en) | 2013-08-06 | 2015-02-12 | Apple Inc. | Auto-activating smart responses based on activities from remote devices |
US9620105B2 (en) | 2014-05-15 | 2017-04-11 | Apple Inc. | Analyzing audio input for efficient speech and music recognition |
US10592095B2 (en) | 2014-05-23 | 2020-03-17 | Apple Inc. | Instantaneous speaking of content on touch devices |
US9502031B2 (en) | 2014-05-27 | 2016-11-22 | Apple Inc. | Method for supporting dynamic grammars in WFST-based ASR |
US9842101B2 (en) | 2014-05-30 | 2017-12-12 | Apple Inc. | Predictive conversion of language input |
US10289433B2 (en) | 2014-05-30 | 2019-05-14 | Apple Inc. | Domain specific language for encoding assistant dialog |
AU2015266863B2 (en) | 2014-05-30 | 2018-03-15 | Apple Inc. | Multi-command single utterance input method |
US9715875B2 (en) | 2014-05-30 | 2017-07-25 | Apple Inc. | Reducing the need for manual start/end-pointing and trigger phrases |
US10170123B2 (en) | 2014-05-30 | 2019-01-01 | Apple Inc. | Intelligent assistant for home automation |
US9633004B2 (en) | 2014-05-30 | 2017-04-25 | Apple Inc. | Better resolution when referencing to concepts |
US9760559B2 (en) | 2014-05-30 | 2017-09-12 | Apple Inc. | Predictive text input |
US10078631B2 (en) | 2014-05-30 | 2018-09-18 | Apple Inc. | Entropy-guided text prediction using combined word and character n-gram language models |
US9430463B2 (en) | 2014-05-30 | 2016-08-30 | Apple Inc. | Exemplar-based natural language processing |
US9734193B2 (en) | 2014-05-30 | 2017-08-15 | Apple Inc. | Determining domain salience ranking from ambiguous words in natural speech |
US9785630B2 (en) | 2014-05-30 | 2017-10-10 | Apple Inc. | Text prediction using combined word N-gram and unigram language models |
US10659851B2 (en) | 2014-06-30 | 2020-05-19 | Apple Inc. | Real-time digital assistant knowledge updates |
US9338493B2 (en) | 2014-06-30 | 2016-05-10 | Apple Inc. | Intelligent automated assistant for TV user interactions |
US10446141B2 (en) | 2014-08-28 | 2019-10-15 | Apple Inc. | Automatic speech recognition based on user feedback |
US9818400B2 (en) | 2014-09-11 | 2017-11-14 | Apple Inc. | Method and apparatus for discovering trending terms in speech requests |
US10789041B2 (en) | 2014-09-12 | 2020-09-29 | Apple Inc. | Dynamic thresholds for always listening speech trigger |
US9606986B2 (en) | 2014-09-29 | 2017-03-28 | Apple Inc. | Integrated word N-gram and class M-gram language models |
US9646609B2 (en) | 2014-09-30 | 2017-05-09 | Apple Inc. | Caching apparatus for serving phonetic pronunciations |
US10074360B2 (en) | 2014-09-30 | 2018-09-11 | Apple Inc. | Providing an indication of the suitability of speech recognition |
US10127911B2 (en) | 2014-09-30 | 2018-11-13 | Apple Inc. | Speaker identification and unsupervised speaker adaptation techniques |
US9886432B2 (en) | 2014-09-30 | 2018-02-06 | Apple Inc. | Parsimonious handling of word inflection via categorical stem + suffix N-gram language models |
US9668121B2 (en) | 2014-09-30 | 2017-05-30 | Apple Inc. | Social reminders |
US10552013B2 (en) | 2014-12-02 | 2020-02-04 | Apple Inc. | Data detection |
US9711141B2 (en) | 2014-12-09 | 2017-07-18 | Apple Inc. | Disambiguating heteronyms in speech synthesis |
US9865280B2 (en) | 2015-03-06 | 2018-01-09 | Apple Inc. | Structured dictation using intelligent automated assistants |
US10567477B2 (en) | 2015-03-08 | 2020-02-18 | Apple Inc. | Virtual assistant continuity |
US9721566B2 (en) | 2015-03-08 | 2017-08-01 | Apple Inc. | Competing devices responding to voice triggers |
US9886953B2 (en) | 2015-03-08 | 2018-02-06 | Apple Inc. | Virtual assistant activation |
US9899019B2 (en) | 2015-03-18 | 2018-02-20 | Apple Inc. | Systems and methods for structured stem and suffix language models |
US9842105B2 (en) | 2015-04-16 | 2017-12-12 | Apple Inc. | Parsimonious continuous-space phrase representations for natural language processing |
US10083688B2 (en) | 2015-05-27 | 2018-09-25 | Apple Inc. | Device voice control for selecting a displayed affordance |
US10127220B2 (en) | 2015-06-04 | 2018-11-13 | Apple Inc. | Language identification from short strings |
US10101822B2 (en) | 2015-06-05 | 2018-10-16 | Apple Inc. | Language input correction |
US10255907B2 (en) | 2015-06-07 | 2019-04-09 | Apple Inc. | Automatic accent detection using acoustic models |
US11025565B2 (en) | 2015-06-07 | 2021-06-01 | Apple Inc. | Personalized prediction of responses for instant messaging |
US10186254B2 (en) | 2015-06-07 | 2019-01-22 | Apple Inc. | Context-based endpoint detection |
US10671428B2 (en) | 2015-09-08 | 2020-06-02 | Apple Inc. | Distributed personal assistant |
US10747498B2 (en) | 2015-09-08 | 2020-08-18 | Apple Inc. | Zero latency digital assistant |
US9697820B2 (en) | 2015-09-24 | 2017-07-04 | Apple Inc. | Unit-selection text-to-speech synthesis using concatenation-sensitive neural networks |
US11010550B2 (en) | 2015-09-29 | 2021-05-18 | Apple Inc. | Unified language modeling framework for word prediction, auto-completion and auto-correction |
US10366158B2 (en) | 2015-09-29 | 2019-07-30 | Apple Inc. | Efficient word encoding for recurrent neural network language models |
US11587559B2 (en) | 2015-09-30 | 2023-02-21 | Apple Inc. | Intelligent device identification |
US10691473B2 (en) | 2015-11-06 | 2020-06-23 | Apple Inc. | Intelligent automated assistant in a messaging environment |
US10049668B2 (en) | 2015-12-02 | 2018-08-14 | Apple Inc. | Applying neural network language models to weighted finite state transducers for automatic speech recognition |
US10223066B2 (en) | 2015-12-23 | 2019-03-05 | Apple Inc. | Proactive assistance based on dialog communication between devices |
US10446143B2 (en) | 2016-03-14 | 2019-10-15 | Apple Inc. | Identification of voice inputs providing credentials |
US9934775B2 (en) | 2016-05-26 | 2018-04-03 | Apple Inc. | Unit-selection text-to-speech synthesis based on predicted concatenation parameters |
US9972304B2 (en) | 2016-06-03 | 2018-05-15 | Apple Inc. | Privacy preserving distributed evaluation framework for embedded personalized systems |
US10249300B2 (en) | 2016-06-06 | 2019-04-02 | Apple Inc. | Intelligent list reading |
US10049663B2 (en) | 2016-06-08 | 2018-08-14 | Apple, Inc. | Intelligent automated assistant for media exploration |
DK179309B1 (en) | 2016-06-09 | 2018-04-23 | Apple Inc | Intelligent automated assistant in a home environment |
US10586535B2 (en) | 2016-06-10 | 2020-03-10 | Apple Inc. | Intelligent digital assistant in a multi-tasking environment |
US10509862B2 (en) | 2016-06-10 | 2019-12-17 | Apple Inc. | Dynamic phrase expansion of language input |
US10490187B2 (en) | 2016-06-10 | 2019-11-26 | Apple Inc. | Digital assistant providing automated status report |
US10192552B2 (en) | 2016-06-10 | 2019-01-29 | Apple Inc. | Digital assistant providing whispered speech |
US10067938B2 (en) | 2016-06-10 | 2018-09-04 | Apple Inc. | Multilingual word prediction |
DK201670540A1 (en) | 2016-06-11 | 2018-01-08 | Apple Inc | Application integration with a digital assistant |
DK179343B1 (en) | 2016-06-11 | 2018-05-14 | Apple Inc | Intelligent task discovery |
DK179049B1 (en) | 2016-06-11 | 2017-09-18 | Apple Inc | Data driven natural language event detection and classification |
DK179415B1 (en) | 2016-06-11 | 2018-06-14 | Apple Inc | Intelligent device arbitration and control |
US10593346B2 (en) | 2016-12-22 | 2020-03-17 | Apple Inc. | Rank-reduced token representation for automatic speech recognition |
US10262646B2 (en) | 2017-01-09 | 2019-04-16 | Media Overkill, LLC | Multi-source switched sequence oscillator waveform compositing system |
DK179745B1 (en) | 2017-05-12 | 2019-05-01 | Apple Inc. | SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT |
DK201770431A1 (en) | 2017-05-15 | 2018-12-20 | Apple Inc. | Optimizing dialogue policy decisions for digital assistants using implicit feedback |
US10347238B2 (en) * | 2017-10-27 | 2019-07-09 | Adobe Inc. | Text-based insertion and replacement in audio narration |
US10770063B2 (en) | 2018-04-13 | 2020-09-08 | Adobe Inc. | Real-time speaker-dependent neural vocoder |
US11830481B2 (en) * | 2021-11-30 | 2023-11-28 | Adobe Inc. | Context-aware prosody correction of edited speech |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0140777A1 (en) * | 1983-10-14 | 1985-05-08 | TEXAS INSTRUMENTS FRANCE Société dite: | Process for encoding speech and an apparatus for carrying out the process |
EP0232456A1 (en) * | 1985-12-26 | 1987-08-19 | AT&T Corp. | Digital speech processor using arbitrary excitation coding |
EP0515709A1 (en) * | 1991-05-27 | 1992-12-02 | International Business Machines Corporation | Method and apparatus for segmental unit representation in text-to-speech synthesis |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4384169A (en) * | 1977-01-21 | 1983-05-17 | Forrest S. Mozer | Method and apparatus for speech synthesizing |
US4692941A (en) * | 1984-04-10 | 1987-09-08 | First Byte | Real-time text-to-speech conversion system |
US4852168A (en) * | 1986-11-18 | 1989-07-25 | Sprague Richard P | Compression of stored waveforms for artificial speech |
DE3850885D1 (en) * | 1987-10-09 | 1994-09-01 | Sound Entertainment Inc | VOICE GENERATION FROM DIGITALLY STORED COARTICULATED LANGUAGE SEGMENTS. |
FR2636163B1 (en) * | 1988-09-02 | 1991-07-05 | Hamon Christian | METHOD AND DEVICE FOR SYNTHESIZING SPEECH BY ADDING-COVERING WAVEFORMS |
US5220629A (en) * | 1989-11-06 | 1993-06-15 | Canon Kabushiki Kaisha | Speech synthesis apparatus and method |
-
1993
- 1993-01-21 US US08/007,621 patent/US5490234A/en not_active Expired - Lifetime
-
1994
- 1994-01-18 ES ES94907854T patent/ES2136191T3/en not_active Expired - Lifetime
- 1994-01-18 WO PCT/US1994/000770 patent/WO1994017517A1/en active IP Right Grant
- 1994-01-18 EP EP94907854A patent/EP0680652B1/en not_active Expired - Lifetime
- 1994-01-18 AU AU61261/94A patent/AU6126194A/en not_active Abandoned
- 1994-01-18 DE DE69420547T patent/DE69420547T2/en not_active Expired - Lifetime
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0140777A1 (en) * | 1983-10-14 | 1985-05-08 | TEXAS INSTRUMENTS FRANCE Société dite: | Process for encoding speech and an apparatus for carrying out the process |
EP0232456A1 (en) * | 1985-12-26 | 1987-08-19 | AT&T Corp. | Digital speech processor using arbitrary excitation coding |
EP0515709A1 (en) * | 1991-05-27 | 1992-12-02 | International Business Machines Corporation | Method and apparatus for segmental unit representation in text-to-speech synthesis |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1996032711A1 (en) * | 1995-04-12 | 1996-10-17 | British Telecommunications Public Limited Company | Waveform speech synthesis |
AU707489B2 (en) * | 1995-04-12 | 1999-07-08 | British Telecommunications Public Limited Company | Waveform speech synthesis |
US6067519A (en) * | 1995-04-12 | 2000-05-23 | British Telecommunications Public Limited Company | Waveform speech synthesis |
EP0805433A2 (en) * | 1996-04-30 | 1997-11-05 | Microsoft Corporation | Method and system of runtime acoustic unit selection for speech synthesis |
EP0805433A3 (en) * | 1996-04-30 | 1998-09-30 | Microsoft Corporation | Method and system of runtime acoustic unit selection for speech synthesis |
US5913193A (en) * | 1996-04-30 | 1999-06-15 | Microsoft Corporation | Method and system of runtime acoustic unit selection for speech synthesis |
WO1999007132A1 (en) * | 1997-07-31 | 1999-02-11 | British Telecommunications Public Limited Company | Generation of voice messages |
US6175821B1 (en) | 1997-07-31 | 2001-01-16 | British Telecommunications Public Limited Company | Generation of voice messages |
US7805307B2 (en) | 2003-09-30 | 2010-09-28 | Sharp Laboratories Of America, Inc. | Text to speech conversion system |
KR20180082521A (en) * | 2016-01-14 | 2018-07-18 | 텐센트 테크놀로지(센젠) 컴퍼니 리미티드 | Method and apparatus for processing audio data |
EP3404652A4 (en) * | 2016-01-14 | 2018-11-21 | Tencent Technology (Shenzhen) Company Limited | Audio data processing method and terminal |
KR102099029B1 (en) | 2016-01-14 | 2020-04-08 | 텐센트 테크놀로지(센젠) 컴퍼니 리미티드 | Audio data processing method and terminal |
Also Published As
Publication number | Publication date |
---|---|
US5490234A (en) | 1996-02-06 |
ES2136191T3 (en) | 1999-11-16 |
DE69420547D1 (en) | 1999-10-14 |
DE69420547T2 (en) | 2000-07-13 |
EP0680652A1 (en) | 1995-11-08 |
EP0680652B1 (en) | 1999-09-08 |
AU6126194A (en) | 1994-08-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP0680652A1 (en) | Waveform blending technique for text-to-speech system | |
EP0689706B1 (en) | Intonation adjustment in text-to-speech systems | |
US5717827A (en) | Text-to-speech system using vector quantization based speech enconding/decoding | |
US4852168A (en) | Compression of stored waveforms for artificial speech | |
US4833718A (en) | Compression of stored waveforms for artificial speech | |
US20070106513A1 (en) | Method for facilitating text to speech synthesis using a differential vocoder | |
US4625286A (en) | Time encoding of LPC roots | |
EP0380572B1 (en) | Generating speech from digitally stored coarticulated speech segments | |
US6240384B1 (en) | Speech synthesis method | |
WO1985004747A1 (en) | Real-time text-to-speech conversion system | |
JPS6156400A (en) | Voice processor | |
US5633984A (en) | Method and apparatus for speech processing | |
EP0515709A1 (en) | Method and apparatus for segmental unit representation in text-to-speech synthesis | |
US7092878B1 (en) | Speech synthesis using multi-mode coding with a speech segment dictionary | |
JP3268750B2 (en) | Speech synthesis method and system | |
JPS58161000A (en) | Voice synthesizer | |
EP0984432B1 (en) | Pulse position control for an algebraic speech coder | |
JPS6187199A (en) | Voice analyzer/synthesizer | |
KR100477224B1 (en) | Method for storing and searching phase information and coding a speech unit using phase information | |
KR0133467B1 (en) | Vector quantization method for korean voice synthesizing | |
JPS593496A (en) | Fundamental frequency control system for rule synthesization system | |
KR100624545B1 (en) | Method for the speech compression and synthesis in TTS system | |
Ansari et al. | Compression of prosody for speech modification in synthesis | |
JPS63118800A (en) | Waveform synthesization system | |
Morton | Naturalness in synthetic speech |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AT AU BB BG BR CA CH CZ DE DK ES FI GB HU JP KP KR LK LU MG MN MW NL NO PL RO RU SD SE SK US |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
EX32 | Extension under rule 32 effected after completion of technical preparation for international publication |
Free format text: GE |
|
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 1994907854 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 1994907854 Country of ref document: EP |
|
REG | Reference to national code |
Ref country code: DE Ref legal event code: 8642 |
|
NENP | Non-entry into the national phase |
Ref country code: CA |
|
WWG | Wipo information: grant in national office |
Ref document number: 1994907854 Country of ref document: EP |