JPS63236100A - Generation of spectral rough form for voice rule synthesization - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Application Field] Compared to natural speech, speech created by the rule synthesis method is
Because the sound quality is poor and lacks naturalness and clarity (understandability), a full-scale practical system has not yet appeared. However, the voices obtained through rule synthesis can not only output arbitrary voices, but also have diversity, such as voices with varying speaking speeds, voices of young and old men and women, voices expressing emotions and feelings, etc. is possible. Spectral trajectory control method of the present invention (
Spectral Locus Control Me
If thod (hereinafter abbreviated as SLCM) improves the sound quality of regular synthesis, it will simply replace the waveform encoding method that has been put into practical use in telephone time signals, station guidance announcements, vending machines, etc. Currently, it is still in the prototype stage, but it can be used in dialogue systems with machines that have high utility as man-machine interfaces, text-to-speech conversion systems for reading devices for the deaf, automatic translation systems for translation phones, etc. This is a technology that fully fulfills the role of output.

[Prior Art] The general procedure for rule synthesis is to determine the intonation, accent, strength, rhythm, and other prosodic features required for sound source generation using rules, and then combine them with the generated drive sound source signal based on the rules. , synthesized speech can be obtained from parameters (acoustic parameters) that express the acoustic characteristics of the articulatory process, including modifications due to the combination of fragments and the expansion and contraction of phoneme length. Three techniques have been introduced into this synthesis process: ■ Speech analysis method using a speech generation model ■ Regularization of prosodic features ■ Modification of spectral characteristics using dynamic features of articulation.

Among these, ■ is based on the spectral characteristics of synthesized speech created by editing memorized synthesis units (phoneme fragments), the articulatory combination of phonemes in continuous speech, and the undersho in the constant part of vowels.
This is a technique to express acoustic characteristics as close as possible to the acoustic characteristics of natural speech by adding acoustic characteristics such as ot or by creating smooth transitions between segments. Currently, synthesis units (phoneme fragments) are made into multiple phoneme chains, and the acoustic characteristics of the phoneme fragments include some deformation of the spectral characteristics that occurs in continuous speech, and the transition of formants between phoneme fragments is enhanced. We are working to improve the sound quality by making it smoother in the next model. Until now, phoneme fragments have focused on syllabic units that have good correspondence with kana characters, and there have been many monosyllables such as V (vowel) and CV (consonant-vowel), phonological chain CvC, and compound syllable VCV. has been used. Among them, CvC, which contains many effects of articulatory combination within the fragment.
It has been reported that phoneme pieces and VCV phoneme pieces have relatively good sound quality.

[Problems to be Solved by the Invention] The present invention focuses on the spectral characteristics of continuous speech, which are among the techniques that affect the sound quality of the above-mentioned regular synthesis of speech. The phonological environment of the analysis word used to create the segment is
Since it can be reflected in the elemental fragment, multiple phoneme chains whose basic unit is longer than a single syllable can absorb spectral deformation in a fixed manner. However, CvC phoneme pieces and VCV
A phoneme cannot express the effects of differences in the phonetic environment before and after the phoneme. Furthermore, it is impossible to use longer phoneme chains due to the large increase in the total number of segments (the amount of stored information). In methods that use advanced spectral transformation rules, the transformation of spectral characteristics is complicatedly intertwined with several elements that determine the phonological environment, so it is necessary to clarify the correspondence between them. It is difficult to properly adjust the spectral characteristics. At present, the spectral characteristics are only locally modified for smooth transitions between fragments. In any case, the spectral characteristics of synthesized continuous speech are far from natural speech, causing unnaturalness between segments and deterioration of overall sound quality.

The present invention aims to solve the cause of this deterioration of sound quality and to generate synthetic speech that is as close to natural speech as possible. In order to make the spectral characteristics of synthesized speech equal to those of natural speech, it is necessary to create flexible speech units (phonetic segments) whose characteristics can be modified to approximate natural speech depending on the environment in which they are placed. . If technology (2) is improved and the sound quality is improved, it will be possible to more accurately and objectively evaluate the quality of the regularization of prosodic control in technology (2), which will greatly contribute to the development of technology (2).

[Means for Solving the Problems] The present invention divides the spectral characteristics of continuous speech into vC units (an interval from the constant part of the vowel to the constant part of the vowel across the consonant), and calculates the time of the acoustic parameter. Extracting the characteristics of the series using principal component analysis, and approximating the amount of correction from the representative spectral trajectory obtained by broadly classifying the series to an arbitrary spectral trajectory using the least squares method based on this, and It is characterized by estimating spectral characteristics using phoneme segments whose basic unit is VCV, which have accumulated analysis data of component analysis method and least squares method, and vowels for connecting phonetic segments in consideration of the phonetic environment.

[Operation] The principle of SLCM of the present invention will be explained. This method
SLCM shown in the outline of speech synthesis using SLCM in the figure
It consists of an analysis section 1 and an SLCM synthesis section 2.

<SLCM analysis> The audio material is a VCv section that focuses on a specific consonant C and cuts out the waveform from the steady part of the preceding vowel V to the steady part of the following vowel V from the continuous speech that includes that consonant. Let the audio be the same sample group. In order to ensure that the VCV containing the consonant in question contains all the patterns of acoustic characteristics that exist in continuous speech, we also take into consideration the types of phonemes in the several places before and after the consonant, the position of the accent nucleus, and changes in speech rate. ing. Acoustic parameters are calculated for each frame using this VCV audio material, and principal component analysis is performed after appropriate variable transformation.

Next, creation of the reference trajectory and correction vector will be explained using an approximate model of the VCV trajectory on the first to third principal component spaces in FIG. 2. The same sample group is further classified into several types of word sets based on the similarity of principal component trajectories. Then, the average trajectory of the principal component in each word set is determined, and this is defined as the average trajectory PS1 (reference trajectory). Each VCV principal component trajectory P1 is obtained by approximating the deviation from the reference trajectory Psi (correction vector) for each frame using the least squares method, as shown in equations (1) and (2). However, the correction vectors DB-DE at both ends of the cut-out VCV are given as default values during approximation. That is, this analysis (
In the SLCM analysis), as VCV phoneme data, the transformation matrix and inverse transformation matrix from the principal component analysis (3) in Figure 1, and the reference trajectory p of the principal component for the number of frames in the VCV section (4) are used as VCV phoneme data.
The purpose is to derive sl and coefficient matrices A and B1 of the approximate polynomial.

<SLCM Synthesis> When synthesizing arbitrary speech, first extract the chain of C and VC2 from the phoneme sequence to be synthesized. Dashi, C7・C
2 represents a consonant (which may be silent at the beginning or end of a word), and ■ represents a vowel in the middle. Then, from the acoustic parameters of vowel 5 in Figure 1 prepared in advance, C1 and C2
The acoustic parameters of the stationary part of the intermediate vowel V are selected for the combination. In this way, after obtaining the spectra of the stationary parts of all the vowels to be synthesized, the phoneme series is divided into V, C2V
It is divided into two units, and a spectrum outline is generated using the v, C2V2 phoneme piece data accumulated in the SLCM analysis of 3 and 4 in FIG. The deformed V, C2V2 trajectory in continuous speech can be expressed as (IX2
), the principal component trajectory P1 can be controlled using the correction vector D8·D. The spectral characteristics can then be obtained by returning this to the acoustic parameter space.

This process is named SLCM synthesis.

The connection of the V, C2V2 phoneme is performed at the vowel parts ■, and ■2 at both ends, but the deformation adjustment for the connection is mainly done at the consonant part and its adjacent transition part. And, it has the characteristic that it can be connected smoothly by simply giving a value near the center of ■ in own vC2. FIG. 3 shows a generation model of the spectral trajectory of continuous speech using pixel fragments in order to compare the SLCMI-based fragments with conventional fragments.

[Example] Hereinafter, embodiments will be described with reference to the drawings.

<Creation of VC ■ Phoneme Pieces by SLCM Analysis> Before applying the statistical method of SLCM analysis, we have conducted sufficient preliminary experiments and decided on the following items (1) to (2). but,
The present invention is not limited to the following examples, and various modifications are possible.

■ Selection of audio materials In this example, the audio materials include C, V, and C2V22 moly words uttered by one adult male speaker, with the consonant C2 fixed and the preceding and following phonemes C4, V, and v2 changed. We have prepared 100 possible patterns. These words excluded consonants, consonants, and consonants, and care was taken to maintain a constant rate of speech and accent type. For the SLCM analysis, the waveform of the V and C2V2 sections sandwiched between vowel stationary parts is cut out and used.

When consonant C2 was further classified into principal component analysis, the number of analyzed words was reduced accordingly. sandwiched consonant C
2, in /ni/ and /s/, the waveform and duration of the consonant are different depending on whether the following vowel is /i/ or not, so the same consonant is further repeated as shown in equation (3). Classified into two types. In addition, as a method of adjusting the phonological length of the cut out parts V, C2V,
Normalized matching of the time axis was performed using [IP].

However, the spectral matching measure is the LPC cepstral distance.

■ Selection of acoustic parameters and their transformation Acoustic features resulting from the spectrum-forming action of articulation are substituted by acoustic parameters that have an equivalent relationship. As acoustic parameters, LPC from linear all-pole system function
Many examples can be found, such as system parameters, LSP, formants in the frequency dimension, cepstrum by unidirectional processing, vocal tract cross-sectional area ratio of a physical acoustic tube model, and articulatory parameters of an articulatory model. Depending on which features of the voice you place emphasis on,
Acoustic parameters are determined. In this example, we also tried auditory parameters such as LPC cepstrum, but PARC
A logarithmic vocal tract cross-sectional area ratio is used to ensure stable PARCOR coefficients when using an OR synthesis filter. however,
As reverse filtering, a first-order difference is applied to the audio waveform.

As shown in equations (4) and (5), acoustic parameters of several frame lengths are grouped into blocks and weighted. The next block is shifted by one frame. In this way, by including several frames worth of dynamic features of articulatory action in one block, temporal changes can be extracted stably. The specifications of LPC analysis in this example are as follows.
As shown in the figure. In addition, the weighting function ˜V of equation (6) is used for weighting the blocks, and the covariance matrix is used as the matrix representing the relationship between the acoustic parameters.

p: Order of acoustic parameter■ Evaluation scale cepstrum is an orthonormal transformation of a logarithmic spectrum as shown in equation (7), so its distance scale is equal to that of the spectrum.Therefore, in LPC analysis, it is used as an objective evaluation of the spectrum. , LP defined by equation (8)
Cepstrum distance
e: hereinafter abbreviated as CD) can be used directly. The present invention uses CD as a spectral evaluation measure to keep it within the perceptual error tolerance range (3 to 4 [dB]), and performs information compression in the principal component space with reference to the spectral envelope diagram.

2π-π = Iim E (C, -C',)' (7)
Ci + C': : Kevstrum e: Kevstrum order ■ The factors that determine the optimization approximation ability of SLCM analysis are reduction of the order of principal components, least squares approximation of principal component values, and redundancy in the trajectory of principal components. However, the spectral distortion is greatly affected by the least squares approximation and the rate of thinning.

Taking this into consideration, we optimized the SLCM. Here, as an example of the analysis results, the results of an experiment using +00 words in each sample group for the consonants C2 /p/ and /r/(D in the CIVIC2V2 audio material) will be described.

For the coefficient matrix A, B in equation (1), the top m matrix elements are calculated to approximate the values of the principal components up to the mth order.
In addition, only diagonal elements were used to approximate other principal components. The format of A1φB1 is shown in equation (9).

The approximation power of the least squares approximation, in which all matrix elements of m:mXm are determined, was adjusted by changing m in equation (9). Also, /p/・/ shown in Figure 5
The linear change section and constant section were thinned out from the VCV reference trajectory on the first and second principal components of each sample group of r/.

Figures 6(a) and (b) show the average C of 100 analyzed words when m of the coefficient matrix A1 and B is reduced from 36th order to 1st order for each sample group of /p/./r/. [It is 1.

As can be seen from FIG. 6, when m is decreased from the 36th order to the 6th order, the average CD of the thinned out frames tends to decrease. Since there are fluctuations in the audio spectrum when the same word is uttered every time, we believe that distortion of the same magnitude as the spectral distortion caused by thinning out the frames to be analyzed is within an acceptable range.

FIGS. 7(a) and 7(b) show the average CO obtained by reducing the order of the principal component when the coefficient matrices A and -B are thinned out by m=5. For both consonants sandwiched between VCVs /p/ and /r/, if the degree n decreases beyond the 18th degree, the average CD will decrease.
is rapidly increasing.

In FIG. 8 (aXb), n=+8. The average CD for each frame of the VCV section when SLCM analysis is thinned out with m=5 is shown. In addition, Figures 9(a) and 9(b) show the influence of the above information compression process on the spectral characteristics of the word /mepi/
It is indicated using /dari/. As can be seen from FIGS. 8 and 9, the CD is large in the consonant part and the transition part adjacent to it, and the spectral envelope is averaged compared to the original speech. In particular, the band width of formant frequencies in high frequency components tends to widen.

/p/・/``S
Since the LCM analysis showed a similar tendency to the above results, the order n of the principal component analysis was set to 18 for all other consonants.
Next, m of the coefficient matrix of the least squares approximation polynomial was determined to be 5, SLCM analysis was performed, and VCV phoneme pieces were created. The average CD for each sample group is summarized in Figure 10. Moreover, one reference trajectory is mainly prepared for each sample group, and the number of frames in the VCV section is thinned out by an average of 41.3%.

FIG. 11 shows the VCV reference trajectory on the first and second principal components on the following vowel side for the laryngeal crossing sound /h/.

The reference trajectory clearly differs depending on the following vowel, with a steady state of /h/ located close to the following vowel. Koyouni/
The reason why h/ exhibits characteristics similar to the following vowel is that the vocal tract remains in the same stance as the following vowel even while uttering /h/. Therefore, the reference trajectory was also classified into five types on the trailing vowel side.

In order to evaluate SLCM analysis for arbitrary words, we attempted SLCM synthesis by giving the logarithmic vocal tract cross-sectional area ratios of vowels at both ends of the VCV from natural speech. The evaluation of bimolar words other than analysis words is as follows:
It was confirmed from FIG. 10 that the average CD only deteriorated by about 0.5 [dB] further. In the evaluation of 91 meaningful trimorphic words, the average values of average CD and maximum CD were 3.8 [dB] and 6.9 [dB:l], respectively. This can be evaluated as a fairly accurate synthesis of the spectral characteristics, considering the influence of the difference in consonant duration from natural speech and the problem of how to give default values in the vowel stationary part.

<Method of connecting VCV phoneme segments> The logarithmic vocal tract cross-sectional area ratio of the vowel stationary part is necessary for estimating the spectral characteristics inside the VCV phoneme segment created by SLCM analysis and for connecting the phoneme segments.

Figure 12 shows the analysis words C, V, C2v2), of which C3 is /
b/(DV.

(AI, Il, Ul, El, 01) and C2 is /b#)
Range of distribution of logarithmic vocal tract cross-sectional area ratio of V2 (A2. +2.U2.C2゜02) and isolated voiced vowels (A, I, tJ
, E, 0) is shown on the space obtained by principal component analysis. The distribution of vl has a larger variance compared to ■2. The reason for this is that v2 is influenced only by the preceding phoneme V and C2, whereas ■, is influenced by C1 and the following phoneme C2.
This is because it is influenced by v2.

Figure 13 shows C of analysis single M C+ VI C2V2.
2 shows, for each frame, the average CD of the results of SLCM synthesis for 100 words each with /n/ and /r/, with vowel steady values set to the following three types.

■ When using the value within the analysis word.

■ When using the average value of vowel v2 for each C2v2. In other words, the given vowel steady value V, ・v2 is the value of each preceding consonant C
Only l-C2 is considered. A vowel stationary value is created by collecting analysis words for each C2v2 and finding the average of v2.

■ When giving isolated vocal vowels.

From Figure 13, /Total・/'/In both cases, the way in which the setting ■ is given is that the preceding vowel V is greatly different from the setting ■.

This is because the performance of SLCM synthesis is greatly influenced by the logarithmic vocal tract cross-sectional area ratio of the vowel used to connect VCV phoneme pieces. Naturally, a vowel stationary value close to the actual vowel V, v2 in the analysis word used in the SLCM analysis can reduce the average CD of v1C2v2. Therefore, in this embodiment, emphasis is placed on lowering the average CD within a segment rather than sharing the vowel steady values at both ends of the VCV between segments for segment connection. S.L.C.
For M synthesis, for each type of consonant C2 classified by SLCM analysis, the preceding vowel V is replaced with the adjacent consonant C, such as C, V, C2.
75 types of vowel steady values are prepared in consideration of 2.C2, and 5 types of vowel steady values are prepared in consideration of the preceding consonant C2, such as C2v2 for the subsequent vowel v2.

<SLCM synthesis of sentence speech> 653 VCs included in 40 sentences with an average length of 3.5 seconds
For V, SLCM synthesis was attempted using the method of giving the vowels described above. In this example, since the VCV segment created by SLCM analysis is used, the VCV of cvcv audio material is used for SLCM synthesis.
Spectral characteristics of the CV interval can be estimated. Therefore, although the VCV in a sentence and the phonological environment are slightly different, the spectrum estimation error caused by this is considered to be about the same as the error caused by fluctuations in each utterance. In Fig. 14 (a) and (b), (1) shows the vC ■ spectrum of natural speech, (2) shows the vc cis vector in cvcv shown in [] obtained by SLCM synthesis, and its error CD is ( 3). Same type of VCV/
In unge, the VCv spectrum of natural speech differs depending on the preceding consonants /c/ and /ni/ in the VCV, and this feature is also reflected in the estimated spectral characteristics. This indicates that the spectral characteristics estimated by SLCM synthesis match natural speech better than the fixed spectral characteristics of conventional vcv segments.

Spectral characteristics and SLCM obtained by analyzing natural speech
Using the spectral characteristics generated by synthesis, we attempted to evaluate the sound quality through comparative listening experiments of synthesized speech. In both cases, pitch pulses and residuals are used as sound sources, and prosody is controlled by a simple pitch bang model.

Even with vC■ speech, which has noticeable spectral errors, there was no noticeable deterioration in sound quality, and the overall sentence quality was confirmed to be on the same level as synthesized speech using spectral characteristics obtained from natural speech.

In this embodiment, the audio materials are limited to bimolar words. However, it is better to cut out words with 3 or more moles or 6 V, C2v2 intervals, so that the vowels v1 and v2 at both ends can have an equal environment where they are influenced by the adjacent consonants on both sides, so it is possible to maintain the constant vowel value between elements. Can be shared. Furthermore, it is possible to express the dynamic characteristics of articulation more effectively by creating blocks of four frames or more in length for deforming the acoustic parameters and by devising weighting. Word beginnings and endings, continuous vowels, and long vowels that cannot be synthesized using VCV segments can be synthesized by making slight changes to this method (selecting audio materials, creating a reference trajectory, giving stationary vowels, etc.).
It is possible to estimate spectral properties. Further, for VCVs containing obscene sounds, cursive sounds, and consonant sounds, VCV fragments can be created using the same analysis method.

Finally, a comparison of the amount of stored information between the conventional VCV segment and the SLCM ■Cv segment is summarized in FIG. Figure 15 shows 5 vowels before and after a specific consonant, limited to acoustic parameters.
This is a comparison of the total amount of information of a total of 25 types of VCV phoneme pieces prepared for each type. The segment created by SLCM analysis can obtain 15 times more kinds of spectral characteristics considering the consonant before the VCV with 1/2 the amount of information compared to the conventional segment.

[Effects of the Invention] As is clear from the above description, by using the SLCM analysis of the present invention, it is possible to effectively extract the features of the complexly changing spectral characteristics of continuous speech in units of VCV. Further, in a VCV phoneme whose extraction results are stored data, the spectral trajectory within the phoneme can be easily controlled by the stationary spectral characteristics of the vowels on both sides of the VCV phoneme. Furthermore, even if the VCV is the same, the spectral characteristics that vary slightly due to differences in the environment during continuous speech can be accurately estimated simply by making the vowel stationary value as close as possible to natural speech.

Furthermore, the rule-based synthesis method using the present invention can produce smooth speech that reflects the individuality of the speaker used to create the segment in the sound quality, and does not have the slightest sense of discontinuity between segments. Instead, it is possible to synthesize speech of such high quality that the difference in sound quality is almost inaudible, even when compared with synthesized speech using spectral characteristics obtained by analyzing natural speech.

[Brief explanation of drawings]

Figure 1 is a diagram showing an overview of a speech synthesis system using SLCM, Figure 2 is a diagram showing an approximation model of VCV trajectory in SLCM analysis, Figure 3 is a diagram showing a generation model of continuous speech spectral trajectory, Figure 4 shows the specifications of LPC analysis, Figure 5 shows an example of the VCV reference trajectory, Figure 6 shows an example of the approximation error due to m in the coefficient matrices A1 and B1, and Figure 7 shows the principal components. A diagram showing an example of approximation error depending on the order of analysis n. Figure 8 is a diagram showing an example of approximation error when the analysis specification is thinned out by n = 18° 1 = 59. Figure 9 is when the specification of SLCM analysis is changed. Figure 10 shows the approximation error when SLCM analysis is performed for each consonant with the optimal specifications. Figure 11 shows the Cv standard trajectory for each consonant followed by a vowel. Figure 12 shows the range of the distribution of vowel steady-state values and the distribution of isolated vocal vowels when the phonetic environment before and after the vowel differs, and Figure 13 shows the vCV when the steady-state values of the vowel are given differently. FIG. 14 is a diagram illustrating an example of an estimation error in a section, FIG. 14 is a diagram illustrating that a VCV spectrum is accurately generated in a text, and FIG. 15 is a diagram illustrating a comparison of the amount of stored information of VCV segments.

Claims (2)

[Claims] (1) Divide the spectral characteristics of continuous speech into VCV units (the section from the stationary part of the vowel to the stationary part of the vowel across the consonant) and extract the time-series characteristics of the acoustic parameters using principal component analysis. Based on this, a method for generating a spectral outline for regular speech synthesis uses the least squares method to approximate the amount of correction from the representative spectral trajectory obtained by broadly classifying it to an arbitrary spectral trajectory. (2) Ruled synthesis of speech that estimates spectral characteristics using phoneme segments whose basic unit is VCV, which has accumulated analysis data of principal component analysis and least squares methods, and vowels for connecting the segments, taking into account the phonological environment. A method for generating spectral outlines for.