JPH0376480B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention relates to a speech synthesis device that synthesizes speech based on a phoneme segment synthesis method. Among human voices, for example, voiced sounds that involve vibration of the vocal cords, such as "a", differ depending on the speaking speed, but if you look at short intervals of about 10 msec, the waveforms have a similar shape. It is made up of repetitions of Therefore, in order to synthesize voiced sounds, prepare several repeating unit waveforms (the length L is called a bit, see Figure 1) (for example, the waveform for "A", the waveform for "I"). Such),
The unit waveforms may be repeatedly reproduced for the required number of repetitions in each predetermined order. In addition, to synthesize an unvoiced sound that does not involve vibration of the vocal cords, such as the first sound of "shi", you can either prepare the unvoiced sound as is, or prepare a waveform of the same length as the voiced sound. You can repeat this several times to play. Such a speech synthesis method is called a phoneme piece synthesis method, and the waveform that is a unit of repetition is called a phoneme piece. Therefore, in this phoneme segment synthesis method, the quality of synthesized speech depends on how efficiently phonemes with good acoustic properties (physical properties) are extracted from the original speech. An example of a conventionally known method for creating phoneme piece data is shown below. First, when extracting phoneme fragments from the original speech, the original speech signal is A-D converted and stored in memory, and when necessary, the signal is read from the memory and supplied to the display device to form the original speech signal waveform. There is a method of extracting unit waveforms that become phoneme pieces while projecting the phoneme. For example, in Figure 2, section K is a repetition of a similar waveform, so...
This method extracts one of the waveforms as a phoneme piece. However, with this method, as a phoneme piece,
, any of the waveforms can be used, so there is no standard for extracting phoneme pieces. Also~
Strictly speaking, the waveforms of are slightly different, so if you extract phoneme segments by fixing them to one of ~, you may end up extracting phoneme segments that have no relation to the physical properties of speech. It could happen. Furthermore, in this method, the section K is represented by the frequency characteristic of one unit waveform, which leads to deterioration in sound quality. That is, when section K is subjected to discrete Fourier transform, frequency-amplitude characteristics, ie, spectral intensity characteristics, as shown in FIG. 3 are obtained. The severe unevenness shown in this spectral intensity characteristic is caused by the repetition of similar waveforms in section K, and the phonological information in section K is carried by the spectral envelope shown by the broken line. If the frequency characteristics of section K are represented by the frequency characteristics of one unit waveform as described above, a spectral envelope as shown in FIG. 3 will not be obtained, resulting in deterioration of sound quality. Therefore, the method of creating phoneme segment data using such an extraction method cannot realize speech synthesis that is faithful to the original speech. Another method is to use LPC (Linear Predictive Coding) for each section where the audio signal can be considered stationary.
There is also a creation method in which a phoneme segment is created by LPC synthesis so that the spectral envelope obtained by LPC analysis of that interval is the spectral characteristic of the phoneme segment, but this method assumes an all-pole model. Therefore, as shown in Figure 4, the section K
of the spectrum envelope, the convex portions are better approximated than the concave portions. Therefore, for phonemes such as nasal sounds, where a sharp concavity in the spectral envelope plays an essential role in hearing,
It cannot be formed using this method. In addition, by taking advantage of the fact that the human ear is relatively insensitive to the phase of the spectrum over a short period (several tens of milliseconds), when storing phoneme data, we use discrete Fourier transform to convert the phoneme into frequency domain data. After that, set the phase of this amplitude data to 0.
By distributing the data into phase or π phase, and then performing inverse discrete Fourier transform to form a signal with a symmetrical waveform (even function) as shown in Figure 5, only one data that is axially symmetric is memorized. There are some methods that reduce the amount of memory used. However, when phoneme piece data is formed by such a method, both ends e (fifth
The value shown in the figure) differs depending on the voiced sound, so it is not always a constant value. Therefore, when performing speech synthesis using this phoneme piece data, the connection between different phoneme pieces V ₁ and V ₂ may become discontinuous, as shown in Figure 6, and this may cause the synthesized speech to become discontinuous. quality deteriorates. In addition, as a method of adding pitch to phoneme pieces,
There is a way to pad the data with "0". For example, as shown in Fig. 7, if you want to make a phoneme piece shorter than pitch P ₀ to the length of pitch P ₀ , you can fill in the missing part with data "0". This is to create a phoneme segment with pitch P ₀ . However, since this phoneme piece data supplements the data "0", it is different from the original phoneme piece data, and therefore,
The frequency characteristics of the phonemes change, and the quality of the synthesized speech naturally deteriorates. In order to extract phoneme segments from the original speech and create phoneme segment data without impairing the acoustic characteristics of the original speech, a configuration as shown in FIG. 8 may be used, for example. First, the original audio is converted into an analog audio signal (electrical signal) using a microphone, etc., and this is supplied to gate 1 for an appropriate length of time l (about 20 msec).
After cutting out the signal (FIG. 9A), the A/D converter 2 converts it into a digital signal Sa of a predetermined number of bits. The sampling rate of A-D converter 2 is 6k
It is set to about Hz. This digital signal Sa is discriminated in the subsequent circuit 3 as to whether it is the original voiced sound or unvoiced sound, and a discrimination output V/VL is outputted.If it is a voiced sound, the pitch period _P0 is Extraction is performed and the data is output. For convenience of explanation, the output is also referred to as P ₀ . Next, a spectral envelope containing phonetic information of the phoneme is extracted from this digital signal Sa. A cepstral analyzer 10 is used for extracting this spectral envelope. That is, the digital signal Sa is converted into a power spectrum in the frequency domain that has more physical meaning by the sound by discrete fast Fourier transform (step (○a).The outline of this power spectrum Sb is This is shown in Figure 9B.In Step ○Pro, after taking the logarithm of the absolute value of the power spectrum Sb, this power spectrum Sb is converted into a signal waveform in order to extract information on the spectral envelope Sc of the original voice shown in Figure 3. The waveform obtained by the Fourier inverse transform is the cepstrum in Figure 9C.
It is Se. Next, in step ◯◯◯, data of the high quenching frequency part with pitch period P ₀ or more is set to zero based on the pitch period P ₀ (Fig. 9D), and the obtained low quenching rate part Sf is set to high speed again in step ◯◯. By performing Fourier transform, the low frequency component Sg of the fast Fourier transform output in step ○I is obtained,
This low frequency component is further anti-logarithmized in step ◯ to obtain the spectral envelope Sh (first digital signal) shown in FIG. 9E. In this way, the spectral envelope Sh extracted by cepstral analysis has an envelope characteristic that uniformly approximates both the convex and concave parts of the spectral envelope Sc of the original data, so it can sufficiently capture the phonological information of the original speech. Therefore, a constant sound quality is always guaranteed for any original sound. Now, as mentioned above, a plurality of encoded digital data obtained by the A-D converter 2 are supplied to the cepstrum analyzer 10 to form one spectral envelope Sh for the original voice. The envelope Sh does not include any information about the pitch period P ₀ . Therefore, in step○,
Spectral envelope Sh based on information of pitch period P ₀
will be reorganized. In other words, the data forming the spectral envelope Sh included within the pitch period P ₀ determined by the sampling frequency fs of the A/D converter 2 is divided into fs P ₀ data (second Reorganized (resampled) into digital signals)
be done. In the next step, a phoneme determining element and a function conversion element to an odd function are added to the data constituting the spectral envelope Sh that has been reorganized based on the information of the pitch period _P0 . In other words, when the data constituting the spectral envelope Sh is phase-converted to data on the imaginary axis (+π/2, -π/2 axes) and viewed as an output waveform, the function conversion process is performed so that all the data become odd functions. At the same time, the data phase is distributed to the +π/2 axis and the −π/2 axis to concentrate the spectrum at a predetermined quefrency. The quefrency at which the spectrum concentrates differs depending on how the data phase is distributed, and this difference results in a difference in phoneme. Therefore, by allocating this data, a phonological element is added to the spectral envelope Sh. In addition, if the original audio to be handled is unvoiced,
The data can be sorted using random numbers or the like. Therefore, the distribution of this data refers to the voiced/unvoiced sound discrimination output V/VL. After adding the phoneme determining element and the function transformation element in this way, in step ○, this spectral envelope Sh is further inversely discrete Fourier transformed to convert frequency domain data to time domain data (digital signal Si for phoneme segment). ) to form. This digital signal Si is data corresponding to the pitch period P ₀ in the extracted audio signal section, that is, the information-compressed time domain corresponding to the original audio, and the output waveform is an antisymmetric waveform (see Fig. 9). F). This data further includes the number of repetitions n(n
=l/P ₀ ) is added, and in step ○, the data is stored in the memory device as final phoneme piece data. The phoneme piece data processing similar to the phoneme piece data processing described above is performed in the next voice section, and the above operations are repeated until there are no more voice signals. In order to synthesize speech, the required speech can be synthesized by repeatedly using segment data stored in a memory or the like n times and performing this processing in a predetermined order. If phoneme segment data is created by the means described above, first, if a method is adopted in which the spectral envelope is extracted from the speech signal using the cepstrum analyzer 10, the convex and concave parts of the spectral envelope Sc can be combined. Since it is possible to extract a spectral envelope Sh that is similar to the above, almost all of the phonetic information of the speech to be obtained can be stored as data, so that a constant sound quality can always be ensured. In addition, we added information about the pitch period P ₀ to the spectral envelope Sh and reorganized the data that forms this spectral envelope Sh. Compared to the case where data "0" which is completely unrelated to the phoneme piece data is filled, the frequency characteristics of the phoneme piece do not deteriorate, and therefore the quality of the synthesized speech does not deteriorate. In other words, a sound quality closer to the original sound can be obtained. Since the data was converted so that the waveform of the phoneme piece became an inversely symmetrized waveform, both ends e of the waveform are always zero. Therefore, there is no discontinuity between the waveforms of different phoneme pieces, so the deterioration of sound quality as in the conventional case does not occur. The present invention proposes a speech synthesis device that can efficiently store phoneme segment data formed using, for example, the above-described means in a memory device. Therefore, first of all, the phoneme piece data used in this invention is data whose analog waveform becomes a point-symmetric waveform with an odd function as described above when it is converted into analog data,
Stored as data. Moreover, since the analog waveform of this phoneme piece data is a point-symmetric waveform as shown in FIG. 10, for example, the phoneme piece data of the first half (half cycle) of the waveform S _IF of the pair of point-symmetric waveforms is stored. , the remaining phoneme piece data of the waveform S _IB in the latter half is restored using the phoneme piece data of the waveform S _IF in the first half. When restoring data, when there is an even number of sampling points as shown in Figure 11A, the phoneme data at the last sampling point q of the waveform S _IF in the first half is used to restore the waveform S _IB in the second half. Although it is not used, when there is an odd number of sampling points as shown in B of the same figure, the phoneme piece data of the last sampling point q is the waveform of the latter half.
It is necessary to restore it using the phoneme piece data of the first sampling point q of S _IB . Therefore, as shown in the data storage format of Fig. 12, it is determined whether the phoneme piece data D _A , D _B . piece of data
A code X indicating whether the numbers of D _A1 , D _{A2 .} . . D _{B1 .} . . is odd or even is applied. In addition, this phoneme piece data is divided into differences as described above.
PCM is used, but as shown in Figure 10, the difference value (+7, -2, etc.) of the difference PCM varies greatly depending on the waveform of the phoneme, so in this example, the difference value (+7, -2, etc.) The number of PCM data is changed. In the example, when the absolute value of the difference value is 7 or more, 8
The number of bits of data is guessed, and if it is less than that, the number of 4 bits of data is guessed. Therefore,
As shown in Fig. 12, phoneme piece data D _A , D _B . . .
may each be configured as bits of unequal length. When configured as unequal length bits, it is necessary to restore 4-bit data to 8-bit data. Therefore, the identification code is assigned to the code X for identifying the number of data mentioned above. An example of the relationship between the number of bits, the number of data, and code X is shown in Table 1 below.

[Table] Code X consists of 4 bits and is distributed according to the number of bits and the number of data. Note that in addition to code X, code Y is shown in <Table 1>, which indicates how many times the phoneme piece data D _A , D _B . . . are repeated during speech synthesis. It is a repeating code, in this example 4
Since it is composed of bits, the number of repetitions n is from 0 to 15.
You can give instructions up to once. Therefore, codes X and Y become 8-bit data (see FIG. 12). In addition, as shown in FIG. 12, the delimiter code Z,
The identification codes X, Y and the phoneme piece data D _A constitute a unit block of data, and a large number of these blocks are stored in the memory device 20 . In addition, when multiple unit blocks are used to collect multiple phoneme data, a phoneme such as one word or one coherent sentence is formed, so the beginning, end, and end of the phoneme are each A delimiter code Z is assigned to distinguish between the two. This delimiter code Z is expressed in hexadecimal, and in this example, the delimiter code for the phoneme segment is "80" and the delimiter code for the speech piece is "7F". Therefore, the delimiter code Z is either a phoneme piece code or a phoneme piece code. An example of the storage format of these identification codes X, Y, delimiter code Z, and phoneme piece data D ( _DA , _DB ...) is shown in FIG. In this figure, one block is 8-bit data. The data is stored in the memory device in this format. It is assumed that the phoneme piece data D includes sounds such as semi-voiced sounds and persistent sounds, in addition to sounds such as A, I.... Now, in order to synthesize desired speech using the output of this memory device, a speech synthesis device as shown in FIG. 13, for example, may be used. In FIG. 13, 20 is a memory device in which phoneme piece data D and the like are stored, 30 is an input device, and 40 is a sequence controller. Further, 50 is a speech unit address table, and 60 is a speech unit code table. Desired data read from the memory device 20 is supplied to a read register 21 and then to a data restorer 22, where 4-bit data is restored to 8-bit data. Therefore, that register 2
The data fetched in 1 is supplied to a data identification circuit 23, where the data content is determined and codes X, Y, and Z, which will be described later, are identified, and then added to the sequence controller 40. The entire speech synthesis apparatus is controlled by various control signals output from the controller 40. The identification circuit 23 first determines the data content of the read register 21, and if it is 8-bit data, the data itself is supplied to the subsequent accumulator (including a demodulation register) 24 by the control signal Pa, and if it is 4-bit data, it is supplied to the subsequent accumulator (including the demodulation register) 24. This data is restored to 8-bit data by signal Pa. In other words, 4-bit data is, for example, “0011”.
When the data is a positive value, such as ``1100'', the upper 4 bits ``0000'' are added to the MSB side to restore the data to 8-bit data, which is then supplied to the accumulator 24; If so, the upper 4 bits "1111" are added to the MBS side and the data is restored to 8 bits before being supplied to the accumulator 24. The waveform is reproduced in the accumulator 24. That is, since the phoneme piece data D is data corresponding to the waveform _SIF in the first half of the waveform S _I , the data corresponding to the waveform _SIB in the latter half is sent to the accumulator 24 so that the phoneme piece data D is reciprocated. It is formed by being added to a demodulation register provided. At this time, the number of additions of the data D is controlled by the control signal Pa based on the output of the detected data number identification code X (see FIG. 11). Furthermore, the number of repetitions of one cycle of phoneme piece data reproduced by the control signal Pc based on the output of the detected repetition code Y is determined. This one cycle of phoneme piece data is demodulated into an analog audio signal by a demodulation register (DA converter) 25, and then an audio signal in a desired audio frequency band is extracted by a low-pass filter 26. In that case, by performing one addition in the demodulation register 25 M times at M times the speed, the apparent sampling rate can be increased by M times. As a result, the cutoff frequency of the low-pass filter 26 is also multiplied by M, so that filtering processing can be performed without using a filter with a steep frequency characteristic at the cutoff point. In this manner, the desired phoneme piece data D is read out and synthesized based on the data from the input device 30, so that the audio signal of the desired phoneme or sentence can be obtained from the output terminal 27. As explained above, according to the speech synthesis device according to the present invention, since the phoneme piece data is stored in the format of unequal length bits, the memory capacity can be reduced without deteriorating the sound quality. . On the other hand, since a large amount of phoneme data can be stored in the memory, the sound quality can be further improved.

[Brief explanation of drawings]

Fig. 1 is a diagram showing an example of the waveform of a voiced sound, Fig. 2 is an explanatory diagram for phoneme segment extraction, Fig. 3 is a power spectrum diagram of a phoneme segment, and Fig. 4 is a power spectrum diagram based on LPC synthesis. Fig. 5 is an explanatory diagram of a waveform obtained by symmetricalizing phoneme pieces, Fig. 6 is an explanatory diagram of the synthesis of this symmetrical waveform, Fig. 7 is an explanatory diagram of bit addition, and Fig. 8
The figure is a signal processing system diagram showing an example of a method for creating phoneme piece data, FIG. 9 is a waveform diagram for explaining its operation, FIGS. 10 and 11 are diagrams for explaining the present invention, and FIG. 12 is a diagram for explaining the operation. FIG. 13, which is a diagram illustrating an example of a data storage format, is a system diagram of essential parts showing an example of a speech synthesis device according to the present invention. 10 is the cepstrum analyzer, P ₀ is the pitch period,
n is the number of repetitions of P ₀ , Sh is the spectral envelope,
20 is a memory device, 21 is a read register, 2
2 is a data restorer, 24 is an accumulator, 30
is an input device, and 40 is a sequence controller.

Claims (1)

[Claims] 1 A speech synthesis device that performs speech synthesis based on phoneme piece data formed by differential PCM, in which the differential PCM data representing each phoneme piece is stored in the form of unequal length bits, and each phoneme is A delimiter code for dividing piece data and a repetition code for indicating the number of repetitions of phoneme piece data are stored, and the equal length bits, delimiter code, and repetition code corresponding to each phoneme piece are read out from the memory and used as desired. A speech synthesis device is characterized in that the speech of is synthesized into phoneme segments.