JPH0125080B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention relates to a speech synthesis device, and an object of the present invention is to improve the quality of a synthesized speech signal. In general, the quality of speech signals (words, phrases, speech) synthesized by combining and editing phoneme fragments (words, syllables, or even shorter speech segments) depends on the processing of connections between phoneme fragments, which are the constituent units of speech. It can be said that it depends on the situation. For example, a sudden change in the waveform that occurs at the connection, that is, a discontinuity in the waveform, causes harmonic noise, lowers the S/N ratio of the synthesized sound, and reduces the clarity. It is also known that fluctuations in the pitch frequency, which is the fundamental frequency of vocal cord vibration, degrade the naturalness of synthesized speech. Human hearing is extremely sensitive to changes in pitch frequency (the detection limit is
0.1%), and if the pitch frequencies of the combined phoneme segments are discontinuous, the synthesized speech will be difficult to hear and unnatural. The present invention makes it possible to obtain high-quality synthesized speech by recognizing the pattern of phoneme segment waveforms and combining each speech segment in a natural manner. As the phoneme segment waveform, there are methods to use one cut out from natural speech, for example, for each pitch interval, or to use a phoneme segment waveform synthesized by another speech synthesizer, but the present invention uses a comparison method. The purpose of this study is to clarify a method for combining phoneme segments of a short period of time, specifically several tens of milliseconds, without discontinuities in waveforms at connections and without fluctuations in pitch frequency. In other words, such short-time phoneme segments should have similar waveforms at least at the joints of adjacent phoneme segments. Therefore, by slightly modifying the time axis of each phoneme, the waveforms of adjacent phoneme segments should be similar. can be smoothly combined. The present invention grasps the degree of waveform similarity in the form of a signal level for the connecting portions of phoneme segments to be combined, and makes appropriate temporal corrections to the time axes of the phoneme segments based on this. The detailed content of the present invention will be explained below using an audio time axis conversion device as a specific example. FIG. 1 is a block diagram illustrating a conventional time axis expansion device. In the figure, terminal 1 is an audio input terminal, 2 is an output terminal, and 3 and 4 are N-bit analog shift registers such as BBD.
5 is a low pass filter (LPF). 6,7,
8 and 9 are analog switches, and input terminal 1
From analog shift register 3 or 4, LPF
The switch controls the audio signal that reaches the output terminal 2 via the terminal 5. In addition, these analog switches control the write clock circuit 10 of the analog shift registers 3 and 4.
Opening/closing is controlled as shown by the Q and output of a frequency dividing circuit 11 which divides the frequency by 2 mN (m will be described later). Analog shift registers 3 and 4 are connected to OR gates 14 and 15 by the clock circuit 10 and Q of the frequency divider circuit 11, and the output AND gates 12 and 13.
The write clock is alternately controlled via the read clock circuit 16 and the frequency divider circuit 11.
Similarly, OR is performed by output AND gates 17 and 18.
It is alternately read clock controlled via gates 14 and 15. That is, for example, an audio signal in which the time axis applied to the input terminal is compressed by m times (m>1) (such a compressed signal can be obtained by, for example, increasing the playback speed of a tape recorder to m times the recording speed). When the Q output of the frequency divider circuit 11 is 1, the signal is sent to the analog shift register 4 via the analog switch 8.
written to. The number of bits of the shift register is N
Therefore, when the input audio signal completes sequential input as mN sampling strings, the rear end N of the mN sampling strings are stored in the shift register, and the Q output of the frequency dividing circuit 11 is inverted. becomes 0, and the switch 8 is closed. At the same time, the Q output of the frequency dividing circuit becomes 1, the switch 6 is opened, and data is written into the analog shift register 3 in the same way. As is clear from the structure shown, the analog shift register 4 is then clocked by a readout clock circuit 16 and read out via a switch 9 which is also controlled by the output. During the writing period to the analog shift register 3, another analog shift register 4 performs reading in this way, and then when the Q and output of the frequency divider circuit 11 is inverted, the analog shift register 4 writes again, and the 3
performs the read. Here write clock circuit 1
0 clock frequency to ₁ , read clock circuit 1
If the clock frequency of 6 is set to ₂ , then if each _clock frequency is determined so that _1/2 = m (1), the time axis will be expanded by m times, and the compressed audio input to audio input terminal 1 will be The restored time axis appears on output terminal 2. The read clock frequency ₂ is, of course,
It is determined to satisfy Nyquist's sampling theorem for the required output audio frequency band. In the conventional device as described above, the connection timing of the phoneme pieces that are alternately output from the analog shift registers 3 and 4 is automatically determined every mN/ _second by the output of the frequency dividing circuit 11 that divides the write clock 10 by 2 mN. Therefore, as shown in FIG. 6, discontinuous waveform changes and pitch frequency fluctuations occur at the connecting portions of phoneme pieces. As mentioned above, such discontinuities in waveform and pitch at the junctions of phoneme segments significantly degrade sound quality and clarity. Next, the contents of the present invention which can improve the drawbacks of the conventional device will be explained with reference to the block diagram of FIG. In the figure, 103 and 104 are analog shift registers 110 and 11 whose opening and closing are controlled by analog switches 107 and 109.
6 are clock circuits with frequencies ₁ and ₂ , respectively;
111 is a 2 mN frequency dividing circuit, and the configuration thereof is the same as that of the conventional device shown in FIG. The present invention adds temporal correction to the connected parts of connected phoneme pieces as described above.
This is performed by a computer (processing unit) (CPU) 121 programmed by a ROM 120. The counting circuit (counter) 122 is the frequency dividing circuit 11
A circuit that is cleared every time the output of clock circuit 110 is inverted, counts the output of clock circuit 110, and instructs timing to CPU 121 via I/O port 123;
The A/D converter 124 is a circuit that digitally converts input signals, and the memory circuit (RAM) 125 cascaded to the CPU 121 stores only the most significant digit of these A/D converted signals.
It also has a function of temporarily storing the results of arithmetic processing by the CPU 121. The reason why only the most significant digit of the A/D converter output is used here is because, as mentioned above, for short phoneme segments of several tens of milliseconds, the waveforms are similar at least at the joints of adjacent phoneme pieces. In order to suppress fluctuations in the basic pitch of the audio, the purpose is achieved by combining the zero-crossing points of the basic pitch waveform of the audio with the least error, so it is necessary to roughly pattern the input waveform. This is because almost the same result can be obtained when compared with the case where all digits of the A/D converter output are used through the calculation processing described later. Patterning the input waveform using only the most significant digit of the A/D converter output means that if the A/D converter output is a natural binary code, the input waveform is
[1] depending on whether it exceeds 1/2 of the converter's dynamic range.

It will output an output of [0]. Another way to get the same functionality is to
For example, this can be done by a comparator 126 as shown in the dashed line in FIG. The same function can also be obtained by saturating the amplitude of the signal using an amplifier and determining the polarity. Next, the main part of an embodiment using such an amplifier for amplitude saturation is shown in FIG. In Figure 3, 1
28 is an amplifier with a sufficiently large gain, and 129 is a clamp circuit. Now, the fourth input terminal (In) 101 is connected.
When an input signal as shown in figure a is applied, the amplifier 12
8, the input signal is amplified and saturated, and the output of the amplifier 128 becomes as shown in FIG. 4b. Furthermore, the lower end (or upper end) of the signal is clamped by the clamp circuit 129.
is clamped, the output of the clamp circuit 129 becomes as shown in FIG. 4c. and clamp circuit 1
The output of 29 is applied to AND gate 127,
It is converted to a TTL level digital signal. If the configuration uses only the most significant digit of the A/D converter, it is possible to use an A/D converter with a small number of output bits.
A converter can be used, and a comparator or saturating amplifier can also perform the same function. In other words, an A/D converter, a comparator, or the like can be used as a binary signal converter that converts an input signal into a binary signal corresponding to the polarity. These A/D converters, comparators, etc. can be constructed at low cost. In the case of such a configuration, the amount of information processed by the computer is further reduced, the capacity of ROM and RAM can be reduced, and the configuration of the computer section can be made inexpensive. First, based on the output of the counting circuit 122, the CPU 121 digitizes M samples from the last part of the input clock using the A/D converter output 124, and only the most significant digit is sent to the I/O port 122.
, and store it in the memory circuit 125. Next, when the output of the frequency dividing circuit 111 is inverted, the CPU 12
1 similarly reads (M+r) samples from the front end of the input clock based on the output of the counting circuit 122. Subsequently, the CPU 121 calculates the similarity between the rear end of the input preceding phoneme and the following phoneme based on the program in the ROM 120, and it is preferable to calculate the squared error of each sampling sequence. The sampling sequence at the end of the preceding phoneme is Xp
(P = 1, 2, 3...M), the front end sampling sequence of the subsequent phoneme segment is Yp (p = 1, 2, 3,...M+r)
Then, the square error between the two waveforms is: ek ² = 1/M _M 〓 ^p=1 (Xp-X/σx-Yp+k-y/σy) ² ...(2) However, = 1/M _M 〓 ^{p =1} xp, =1/M _M 〓 ^p=1 yp, It is expressed as k=0, 1, 2,..., r,. This represents the degree of similarity when yp is shifted by k points and superimposed on the sampling waveform xp. However, the arithmetic processing based on equation (2) actually requires a huge number of calculation steps, and requires a high-performance computer to perform calculations in a short period of time (at least several tens of milliseconds). Originally (2)
The formula examines the correlation between two waveforms with different amplitudes and levels, and therefore the standard deviation (σx),
The error is calculated by normalizing the waveform by (σy) and then calculating the sum of squares of the difference from the average level. By the way, in the case of the speech synthesis apparatus of the present invention, the phoneme pieces handled have waveforms that are close in time, and therefore, it can be considered that the amplitude and level are originally similar. In this case the difference between the two waveforms is (2)
Instead of the formula, ek ² =1/M _M 〓 ^p=1 (xp−yp+k) ² (3) may be calculated. Furthermore, in the case of the present invention, it is only necessary to know the timing at which the similarity between the two waveforms is maximum, and therefore equation (3) can be further replaced with the following equation (4). ek= _M 〓 ^p=1 |xp−yp+k| …(4) Here, (xp) and (yp+k) are only the data of the most significant digit of the A/D converter, so they are both [1] or

It is [0]. That is, this is the integral of the absolute value of the difference between the corresponding sampling values, and the connection timing is determined by knowing k at which this is the minimum. In the present invention, in order to minimize calculation processing time,
Instead of formula (4), calculate gk= _M 〓 ^p=1 (XpYp+k)...(5). In equation (5), (xp) and (Yp+
k) is the most significant digit data of the A/D converter,
[1] or

It is [0]. The symbol is the symbol for exclusive OR, therefore, (XpYp+k)
is the exclusive OR of (Xp) and (Yp+k), that is, (Xp) and (Yp+k) are both [1], or

[0]
When

[0] is given, and at other times [1] is given. Therefore, the similarity between the binary signal sampling data (Xp) at the rear end of the preceding phoneme and the binary signal sampling data (Yp) at the tip of the following phoneme is given by (gk), and this (gk) The connection timing is determined by knowing k that minimizes . That is, the arithmetic processing unit 121 converts gk to k=
Calculate each of 0, 1, . . . r, and determine k for which this is the smallest. That is, as shown in FIG. 5, the least error is achieved when the M sample strings at the end of the preceding phoneme are superimposed from a portion shifted by k from the beginning of the succeeding phoneme.
Furthermore, Fig. 5a shows the rear end of the preceding phoneme, Fig. 5b shows the front end of the succeeding phoneme, and Fig. 5c
is a drawing showing a timing chart. Therefore, the arithmetic processing unit 121 takes in (k+M+N) samples from the beginning of the subsequent phoneme, and
AND gate 112 through I/O port 123
Alternatively, it controls 113 and stops the write clock. Analog shift register 103 or 104
Since the capacity of is N, therefore, N bits are stored in the analog memory from the (k+M+1)th as shown in the figure, and are sequentially read out at the next read timing. Since the last M samples of a phoneme and the (k+1)th M samples of the subsequent phoneme overlap with the least error, the phoneme is successively output in a completely natural manner. As mentioned above, (k+M+N) samples of the subsequent phoneme are taken into the analog memory, and among these, M samples from the end are similarly sent to the CPU 121 via the A/D converter output 124 and the I/O port 123.
The data is stored in the storage device 125 of. This is necessary in order to check the similarity with (M+r) samples from the beginning of the subsequent phoneme and connect them. A time chart of the above processing is shown in FIG. 5c. Note that the analog shift register 103 or 104 has N bits, so even if a sample value of more than this bit (M+k+N) is read, only the rear N bits are stored. Although the processing method described above can perform processing in a relatively short time, it does not provide sufficient processing speed. In order to clarify this point using specific numerical values and solve the problem,
Introducing a new processing method. It is known that among speech sounds, vowels have a relatively long duration of one hundred milliseconds to several hundred milliseconds, and consonants have a shorter duration than vowels. Therefore,
When the Q inversion time becomes longer in the embodiment of the present invention,
This increases the probability of causing a so-called omission of consonants, which is a loss of speech information. On the other hand, this Q inversion time must be a necessary amount in order to take in binary signal sampling data and perform calculation processing. FIG. 7 shows the Q inversion time portion of the time chart shown in FIG. 5c. In this Figure 7, a
indicates the case where k=0, and c indicates the case where k=r. still,
In FIG. 7, the time axis is plotted in units of the number of write clocks. Therefore, the number of write clocks in the Q inversion period is m×N. The time (To) that can be used to calculate the similarity between the preceding phoneme and the following phoneme is the time from when the sample data of the preceding phoneme is taken until when the sample data of the following phoneme is taken. Figure 7a
For b, Tc = [N-(M+k)]τ. For c, Tc = [N-M]τ. Here, τ is the period of the write clock. Here, if the clock frequency ₁ of the write clock circuit 10 is 40 KHz, the clock frequency ₂ of the read clock circuit 16 is 20 KHz, and N = 512, then τ = _1/1 = 1/m・_1/2 = 0.025 [millisecond]. Therefore, the Q inversion time is mNτ = 25.6 [milliseconds]. Furthermore, it is desirable that the period (Mτ) of the trailing end sampling data of the preceding phoneme corresponds to a period equal to or longer than the basic pitch of the voice. In addition, the sampling data section at the front end of the subsequent phoneme segment "(M+r)τ" minus "Mτ" requires a similar time width, and therefore, in order to correspond to a male voice with a basic pitch of 100 Hz. Then, Mτ=1/m×10 [milliseconds], (M+r)τ=1/m×20 [milliseconds], and therefore M=r=200. Substituting this into equation (6) yields Tc = 112τ = 2.8 [milliseconds]. In addition, the similarity calculation is performed by calculating the exclusive OR of 200 binary signal sampling data of the trailing end of the preceding phoneme and the sampling data of the binary signal at the tip of the succeeding phoneme. It has to be done twice, and it cannot be processed in 2.8 milliseconds. Therefore, the basic idea of the present invention is to make a compromise between increasing the sampling interval of the binary signal and decreasing Mτ and rτ. However, increasing the sampling interval of the binary signal lowers the resolution of the speech unit connection part, and decreasing Mτ and rτ has the disadvantage that low-pitched voices are not connected. Therefore, the present invention has high resolution in the audio connection section,
Furthermore, a processing method with short data calculation time is adopted, which will be explained in detail below. FIG. 8 shows audio signals for the trailing end a of the preceding phoneme and the front end b of the succeeding phoneme, and 2
and a value signal. The sampling interval for sampling the binary signal is a sampling interval (hereinafter referred to as the second clock) that is an integral multiple (for example, four times) of the write clock ₁ of the analog shift register (hereinafter referred to as the first clock). . As a result, the sampling data of the binary signal is reduced to, for example, 1/4. When capturing the binary signal sampling data at the end of one end of the preceding phoneme in FIG. The clock (referred to as the third clock) count number CM from the start of capture to the rise of the binary signal is stored. The third clock is
The same thing as the clock can also be used. Also in the acquisition of the binary signal sampling data at the tip of the subsequent phoneme, the second clock
The value signal is sampled and sampling data of the binary signal is taken in. In this case, the count value Cx (C ₁ , C ₂ , C _{3 . .} . ) of the third clock is stored every time the binary signal rises from the start of acquisition. As described above, in the same way as the binary signal sampling data is taken in, the calculated value of the third clock corresponding to the rising edge of the binary signal is stored. On the other hand, the similarity calculation calculates the CM by C ₁ , C ₂ , C ₃ , etc.
The binary signal sampling data at the rear end of the preceding phoneme segment is shifted to correspond to the binary signal sampling data at the rear end of the preceding phoneme segment and the binary signal sampling data at the leading end of each corresponding subsequent phoneme segment for each shift. The count value of the third clock when the exclusive OR is counted for the data and the count value is the minimum
The shift amount k is determined from Cx. (However, k=
Cx−C _M ). In this way, the binary signal is sampled by the second clock, and therefore the number of sampled data is small, but by using the third clock, the sampled clock for C _M and _C You can make a meal. Furthermore, the period of the rise of the binary signal can be configured to be sufficiently longer than the period of the second clock, and therefore the number of times the above-mentioned exclusive OR counting is performed can be reduced.
It is possible to shorten calculation time. FIG. 9 shows a specific example of configuring the rising cycle of the binary signal to be sufficiently longer than the cycle of the second clock. The binary signal conversion circuit is the A/D converter 124 in FIG. 2 or the comparator 126 and gate 127 in the dashed-dotted line in FIG.
28, clamp circuit 129 and gate 127. However, the clock circuit 110 that generates the write clock ₁ is a voltage controlled oscillator circuit,
The control voltage is applied to the sliding contact 202 of the variable resistor VR.
shall be obtained from This variable resistor VR is connected between the DC power supply terminal 201 and the ground, and the potential of the sliding contact 202 is determined by the adjusted value of the variable resistor VR. As already mentioned in equation (1), the time axis conversion ratio m is the ratio of the read clock ₂ and the write clock ₁ , and since the read clock ₂ is configured to be constant, it is difficult to adjust the variable resistor VR. Therefore, the time axis conversion ratio m is varied. The reproduced audio signal obtained at the input terminal 101 has a high frequency depending on the reproduction speed of the reproduction device. In order to make the rising cycle of the binary signal converted by the binary signal conversion circuit 150 as long as possible, the voltage-controlled filter 130 is controlled by the potential of the sliding contact 202, so that the fundamental frequency information of the audio signal is A configuration is adopted that limits the bandwidth to the extent that it does not disappear. As described above, the band-limited audio signal is amplified by the saturation amplification circuit 128, clamped by the clamp circuit 129, converted to a TTL level signal by the gate 127, and sent to the I/O port 12 as a binary signal.
3 is output. As is clear from the above description, the device of the present invention performs an exclusive OR of the sampling value at the rear end of the preceding phoneme and the sampling value at the front end of the succeeding phoneme at the connection point between the preceding and succeeding phonemes. The time axes are corrected so that the integral value of the signals is superimposed to a minimum. Therefore, it is possible to obtain a synthesized sound without discontinuities in the waveform at the connection part or fluctuations in the pitch frequency as in the conventional device.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of an existing speech synthesis device, FIG. 2 is a block diagram showing the configuration of the device of the present invention, FIG. 3 is a block diagram of the main part of FIG. Figure 5 is a waveform diagram to explain the operation of the device of the present invention, Figure 6 is a waveform diagram showing the characteristics depending on the device of Figure 1, and Figure 7 is a waveform diagram of the device of the present invention. 8 is a waveform diagram showing the data processing situation in the device of the present invention, and FIG. 9 is a block diagram showing the configuration of the main parts of the device of the present invention. 101 is a signal input terminal, 102 is a signal output terminal, 103,
104 is an analog storage means, 110 is a writing clock, 116 is a reading clock, 120
121 represents a ROM, 121 represents a CPU, 124 represents an A/D converter, and 125 represents a RAM.

Claims (1)

[Claims] 1. A speech synthesis device that performs editing and synthesis using phoneme pieces extracted from an analog speech waveform, comprising: (a) storage means for sampling and storing an analog input signal in accordance with a first clock; (b) binary signal converting means for converting the polarity of an analog input signal into a binary signal; and (c) the rear end of the preceding phoneme and the subsequent phoneme converted by the binary signal converting means. 2 at the front end
(d) binary signal sampling data storage means for storing sampling data of both phonemes obtained by sampling the value signal at a second clock related to the first clock; (e) exclusive OR means for calculating an exclusive OR of binary signal sampling data at the rear end of the preceding phoneme and binary signal sampling data at the front end of the subsequent phoneme; (f) a polarity inversion point of the binary signals at the rear end and front end of the phoneme segment at a third clock related to the first clock; and a polarity reversal time storage means for counting the counted polarity reversal time and storing the counted polarity reversal time. The sampling data at the rear end or the front end of the preceding phoneme segment is relatively shifted to correspond to the sampling data at the rear end of the preceding phoneme and the binary signal at the front end of the succeeding phoneme. The sampling data at the rear end of the preceding phoneme and the front end of the succeeding phoneme are relatively shifted so that the exclusive OR with the sampling data is minimized, and the sampling data at the shifted polarity inversion point is A speech synthesis device characterized in that the counted value of the counting means is read out from the polarity reversal time point storage means, and analog phoneme pieces are connected based on this.