JP2005189483A - Sound quality model generation method, sound quality conversion method, computer program for them, recording medium with program recorded thereon, and computer programmed with program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To control sound quality with a small number of parameters. <P>SOLUTION: A sound quality conversion method, in which the sound quality of an input speech waveform 50 is converted by using a glottal waveform model 36 consisting of a pair of a prototype sound chords wave made to correspond to prescribed sound quality and the prescribed number of main component expressions from the head obtained through a main component analysis of the prototype sound chords, includes a unit waveform extraction step 60 of extracting unit waveforms of the sound chords wave from parts meeting prescribed conditions respectively; and voice waveform generation steps 62 and 64 of generating an output speech waveform generated by converting the input voice waveform 60 to sound quality prescribed by a user on the basis of a glottal waveform model corresponding to sound quality prescribed as the sound quality of the input voice waveform and a glottal waveform model corresponding to sound quality prescribed by the user. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a method for controlling voice quality, and more particularly, to a method and apparatus for expressing voice quality by a parameter and changing the voice quality using the value of the parameter.

  In the last few decades, computer-based speech processing technology has made significant progress. For example, speech recognition technology can be recognized with considerably high accuracy, and speech synthesis can be performed with a certain degree of ease of hearing.

  However, there are still many differences between voice processing that humans usually perform and voice processing using a computer. A typical example is the handling of paralinguistic information.

  Paralinguistic information refers to elements of spoken language that cannot be expressed in letters. For example, gestures at the time of utterance, facial appearance, voice tone, etc. constitute paralinguistic information. Humans can feel the speaker's feelings through subtle changes in the tone of their voices. On the other hand, only the elements that can be expressed by characters can be obtained by speech recognition, and paralinguistic information cannot be captured. Similarly, even if the content of the utterance is the same, a person can convey various feelings at the time of utterance by the tone of the voice at the time of utterance. However, in speech synthesis, it is difficult to synthesize such speech.

  Voice quality is a typical paralinguistic information. The voice quality can be defined at various levels (for example, functional aspects of speech) in various areas (for example, articulatory, acoustic, and perceptual areas). In a broad sense, voice quality refers to the attributes of speech that is human speech generated by a speaker and perceived by a listener over multiple speech units (eg, phonemes).

  With the current technology, it is difficult to perform processing corresponding to a change in voice quality when a human utters in both speech recognition and speech synthesis. This field is considered to be a good field for joint research between phonology and speech processing technology.

JP2003-330478A Laver, J .; , "Voice description of voice quality", Cambridge University Press, 1980 (Laver, J. (1980), "The Physical Description of Voice Quality", Cambridge: Cambridge University Press).

  There is no doubt that speech processing technology will be used in more and more aspects as an interface between humans and computers. At that time, if paralinguistic information can also be used for communication, it is possible to realize something close to human communication.

  However, until now, it has been difficult to determine the voice quality of human voice by voice recognition, and it is very difficult to judge the emotion of the speaker from the voice quality. In speech synthesis, how to control voice quality in order to transmit certain paralinguistic information has not yet been clarified.

  In addition, it is desirable that the number of parameters for controlling voice quality is as small as possible, and more ideally, such parameters are meaningful from both physiological and perceptual viewpoints. It would be desirable to deepen the understanding of these phenomena in these two areas.

  Conventionally, however, it is unclear what parameters should be used as parameters for determining and controlling such voice quality, and naturally, how to change the parameters will improve the voice quality of speech synthesis. I didn't know if it could be what I wanted.

  Therefore, an object of the present invention is to provide a voice quality model generation method for generating a voice quality model for expressing voice quality.

  SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a voice quality model generation method for generating a voice quality model for expressing voice quality with a small number of parameters.

  Another object of the present invention is to provide a voice quality conversion apparatus and method capable of converting a voice quality to a desired one.

  Another object of the present invention is to provide a voice quality conversion apparatus and method capable of converting a voice quality to a desired one with a small number of parameters.

  In the voice quality model generation method according to the first aspect of the present invention, a portion of a plurality of reference speech waveforms prepared in advance corresponding to a predetermined voice quality satisfies a predetermined condition from a portion satisfying a predetermined condition. A vocal cord waveform estimation step for estimating a unit waveform of a vocal cord wave when uttered; a parameterization step for parameterizing each of the unit waveforms of the vocal cord wave according to a predetermined parameterization method; and a unit of the parameterized vocal cord wave A principal component analysis step of obtaining a principal component representation of each unit waveform of the vocal fold wave by performing principal component analysis on the waveform, each waveform of the unit waveform of the vocal fold wave, and a principal component representation corresponding to the waveform; Outputting as a voice quality model corresponding to the voice waveform from which the vocal fold wave is obtained.

  Preferably, the vocal cord waveform estimating step extracts a syllable nucleus of a plurality of speech waveforms prepared in advance corresponding to a predetermined sound quality, and influences of the vocal tract on each of the extracted syllable nuclei. Applying an inverse filter to detect the volume velocity waveform of the glottal airflow when speech is generated by removing, and extracting a unit waveform of the vocal cord wave from each of the syllable nuclei after the inverse filter is applied Unit waveform extraction step.

  More preferably, the unit waveform extraction step starts from a minimum portion of the volume velocity waveform existing in the center of the syllable nucleus, and goes back by one period determined by the fundamental frequency of a predetermined region including the syllable nucleus. A step of extracting up to a certain portion as a unit waveform.

  More preferably, prior to the unit waveform extraction step, the method further includes a step of normalizing the volume velocity waveform of the glottal airflow according to a predetermined normalization method.

  Preferably, the principal component analysis step obtains principal component representations of up to a predetermined number of principal components from the head of each unit waveform of the vocal fold wave by performing principal component analysis on the parameterized vocal fold unit waveform. Including the steps of:

  More preferably, the predetermined number of main components is from the first main component to the fourth main component.

  More preferably, the parameterizing step includes a resampling step of resampling the unit waveform of the vocal fold wave at a predetermined number of sampling points that divide the unit waveform of the vocal fold wave into a plurality of equal length portions.

  More preferably, it further includes a differentiation step of obtaining a differential data string of the unit waveform of the vocal fold wave by taking a difference of the unit waveform of the vocal fold wave resampled by the re-sampling step, and the principal component analysis step includes the differential data string The principal component analysis is performed on the unit waveform to obtain a principal component expression for each differential amount of the unit waveform of the vocal fold wave.

  More preferably, each of the differential data sequences obtained by the differentiation step includes a pair of a difference in resampling time and a difference in unit waveform of a vocal cord wave corresponding to the difference in resampling time, and a voice quality model generation method Further, prior to the principal component analysis step, for each of the differential data sequences obtained by the step of obtaining the differential data sequence, the effect due to the fluctuation in the time axis direction and the influence due to the fluctuation in the amplitude direction are equalized. The method further includes a step of performing a predetermined normalization process.

  The voice quality conversion method according to the second aspect of the present invention includes a main waveform predetermined for each of a plurality of prototype vocal cord wave unit waveforms respectively associated with a predetermined voice quality and a plurality of prototype vocal cord wave unit waveforms. A voice quality conversion method for converting the voice quality of an input speech waveform using a glottal waveform model consisting of pairs with a predetermined number of principal component expressions from the beginning obtained by component analysis. A unit waveform extraction step for extracting a unit waveform of a vocal cord wave from a portion satisfying the above, a glottal waveform model corresponding to a voice quality specified in advance as a voice quality of an input voice waveform, and a voice quality specified by a user Based on the glottal waveform model, the sound that generates the output speech waveform by converting the unit waveform of the vocal cords extracted from the input speech waveform into the voice quality specified by the user And a waveform generation step.

  Preferably, the voice waveform generation step includes a step of selecting a first prototype vocal cord wave from a glottal waveform model corresponding to the voice quality of the input voice waveform, and a second prototype from the glottal waveform model corresponding to the voice quality specified by the user. A step of selecting a vocal fold wave, and a conversion function for converting the input voice waveform into a voice waveform of voice quality designated by the user by performing a predetermined calculation between the first waveform and the second waveform. A conversion function calculating step for calculating, and a step of generating an output speech waveform by applying the conversion function to a unit waveform of a vocal cord wave of the input speech waveform.

  More preferably, the conversion function calculating step includes a step of calculating the conversion function by subtracting the first waveform from the second waveform.

  More preferably, the speech waveform generation step includes a step of generating an output speech waveform by adding a conversion function to a unit waveform of a vocal cord wave of the input speech waveform.

  Preferably, the unit waveform extraction step includes a step of extracting a syllable nucleus of the input speech waveform, and a volume of glottal airflow when a voice is generated by removing the influence of the vocal tract for each of the extracted syllable nuclei. Applying an inverse filter for detecting the velocity waveform, and extracting a unit waveform of a vocal cord wave from each of the syllable nuclei after the inverse filter is applied.

  More preferably, the step of extracting the unit waveform starts from a minimum portion of the volume velocity waveform existing in the center of the syllable nucleus, and from there is one period determined by the fundamental frequency of a predetermined region including the syllable nucleus. This includes a step of extracting as a unit waveform up to a portion that is traced back.

  More preferably, the method further includes the step of normalizing the volume velocity waveform of the glottal airflow according to a predetermined normalization method prior to the step of extracting the waveform.

  Preferably, the predetermined number of principal component representations from the top are based on the first principal component to the fourth principal component.

  When executed by a computer, the computer program according to the third aspect of the present invention is configured to operate the computer so as to realize all the steps of any one of the methods described above.

  A computer according to the fourth aspect of the present invention is programmed by the computer program described above.

  A computer-readable recording medium according to a fifth aspect of the present invention records the above-described computer program.

-Constitution-
FIG. 1 is a block diagram of a voice quality conversion system 30 according to an embodiment of the present invention. Referring to FIG. 1, this voice quality conversion system 30 uses a reference voice waveform 32 for determining a reference value of a parameter for controlling voice quality, selected as a voice having a specific voice quality, as a glottal waveform model. A model creation unit 34 for creating a PCA parameter model 36 representing parameters for controlling voice quality by principal component analysis (PCA), an input speech waveform 50, and voice quality specifying information 51 for specifying the voice quality of the input speech waveform 50. In response to the input voice waveform 50, the same analysis as that performed by the model creation unit 34 is performed to extract the waveform of the vocal fold wave, and based on the voice quality specifying information 51 and the target voice quality set by the user. And a voice quality conversion device 52 for regenerating the speech waveform 54 with the target voice quality using the PCA parameter model 36.

  In the present embodiment, as the reference speech waveform 32, 13 types of human speech waveforms selected in advance as speech of characteristic voice quality are used. These speech waveforms are pre-labeled to indicate such voice quality. In the present embodiment, voice data attached to Non-Patent Document 1 is used as this voice waveform. These voices and voice quality will be described later with reference to FIG. Note that audio data used in the present embodiment is prepared in advance as digital data sampled in units of frames at a predetermined sampling rate.

  FIG. 2 is a block diagram showing a detailed configuration of the model creation unit 34. Referring to FIG. 2, model creation unit 34 extracts a syllable for extracting a region (hereinafter, referred to as “syllable nucleus”) that is stably uttered by a speaker's utterance mechanism from a speech waveform. A nuclear extraction unit 80 is included. More specifically, the syllable nucleus extraction unit 80 calculates a waveform distribution on the time axis of the acoustic energy, and applies a convex hull algorithm to the contour of the distribution waveform to thereby obtain a valley portion in the contour of the acoustic energy. And the input speech is divided into pseudo syllables at the valleys. The syllable nucleus extraction unit 80 further sets a point at which the maximum value of acoustic energy in the pseudo syllable obtained as described above is obtained as a starting point of the syllable nucleus. The syllable nucleus extraction unit 80 further includes frames on both the left and right sides of the syllable nucleus, in which the acoustic energy is larger than a predetermined threshold value (0.8 × maximum value of acoustic energy) and determined to be voiced, and the same pseudo If there are frames in the syllable, those frames are added to the pseudo syllable one frame at a time to extract a continuous region as a syllable nucleus.

  The model creation unit 34 further includes a formant estimation unit for estimating the first four formant frequencies and bands by linear prediction using a linear prediction (LP) cepstrum for each of the syllable nuclei extracted by the syllable nucleus extraction unit 80. 81. The formant estimation unit 81 uses predetermined linear cepstrum-formant mapping, and has previously learned mapping for a vowel formant. The syllable nucleus extraction unit 80 and the formant estimation unit 81 are the same as those disclosed in the aforementioned Japanese Patent Application Laid-Open No. 2003-330478.

  The model creation unit 34 further generates an inverse filter for removing the influence of the vocal tract on the speech for each of the syllable nuclei extracted by the syllable nucleus extraction unit 80 and the formant estimation unit 81, and applies it to the speech waveform. An inverse filter processing unit 82 for applying, and a volume velocity waveform detecting unit 84 for detecting the volume velocity waveform of the glottis of the vocal cords of the speaker when the syllable nucleus is uttered from the output of the inverse filter processing unit 82; including.

  The model creation unit 34 further includes a normalization unit 86 for normalizing the glottal volume velocity waveform detected by the volume velocity waveform detection unit 84, and the glottal volume velocity waveform normalized by the normalization unit 86. A waveform extraction unit 87 for extracting waveform data of one cycle near the center of the syllable nucleus (voice vocal wave), and waveform data of one cycle of vocal fold wave extracted by the waveform extraction unit 87 will be described later. A PCA analysis unit 88 for performing such PCA analysis and calculating up to the fourth principal component.

  The value of the principal component of the PCA analysis output from the PCA analysis unit 88 is associated with the corresponding vocal cord waveform (referred to as a prototype vocal cord wave) to constitute the PCA parameter model 36. Prior to the PCA analysis by the PCA analysis unit 88, it is necessary to parameterize the speech waveform data, details of which will be described later. As will be described later, the PCA parameter model obtained in this way is considered to well represent the voice quality of each speech waveform constituting the reference speech waveform 32.

  FIG. 3 is a more detailed block diagram of the inverse filter processing unit 82 shown in FIG. With reference to FIG. 3, the inverse filter processing unit 82 further improves the accuracy of the formant estimated by the cepstrum-formant mapping by analysis and synthesis optimization for each syllable nucleus, and further changes with time. An inverse filter generation unit 120 for generating an inverse filter for removing the influence of the vocal tract, and a high-pass filter 122 for attenuating an unclear speech component having a low frequency in the speech waveform of the input syllable nucleus, Among the outputs of the high-pass filter 122, the low-pass filter 124 for attenuating the spectral components above the fourth formant, and the inverse filter generated by the inverse-filter generating unit 120 for the audio signal output from the low-pass filter 124 Apply to remove the effects of the first four resonance components of the vocal tract And a reverse filter applying unit 126 for.

  The volume velocity waveform detection unit 84 shown in FIG. 2 integrates the voice signal from which the influence of the vocal tract is removed, which is output from the inverse filter application unit 126, thereby removing the influence of the lip radiation, It has a function to output an estimated waveform of volume velocity wave.

  The normalization unit 86 shown in FIG. 2 is for normalizing the estimated waveform of the volume velocity wave of the glottal airflow output from the volume velocity waveform detection unit 84. Since it is not known in advance what the amplitude of this waveform is, it is necessary to normalize in this way. The normalizing unit 86 according to the present embodiment obtains an average value of the amplitudes of the volume velocity waves over the entire syllable nucleus, and normalizes the waveform by subtracting it from the original value.

  The waveform extraction unit 87 shown in FIG. 2 extracts a vocal cord wave near the syllable nucleus as follows. That is, the waveform extraction unit 87 searches for a minimum value portion of the waveform near the syllable nucleus, and extracts a portion that goes back from that point as a starting point as a one-cycle vocal cord wave. The period in this case is determined as the reciprocal of the fundamental frequency F0.

  FIG. 4 is a detailed block diagram of the PCA analyzer 88 shown in FIG. For PCA analysis, it is necessary to express the waveform with a certain number of parameters. The PCA analysis unit 88 parameterizes the vocal fold wave by a specific method as described below in order to enable PCA analysis related to the values of both the period and amplitude of one cycle of the vocal fold wave.

  Referring to FIG. 4, PCA analysis unit 88 includes a low-pass filter 140 having a cutoff frequency determined by the 15th harmonic of the vocal fold waveform to be analyzed, and a vocal cord whose low-frequency component is attenuated by low-pass filter 140. A re-sampling unit 142 for re-sampling the wave waveform so as to be composed of 30 partial waveforms at equal intervals. In the sampling by the re-sampling unit 142, sampling is performed such that the distances between the sampling points along the waveform itself are equal to each other. Such sampling can take into account the covariance between the amplitude axis of the waveform and the time axis, and can flexibly model changes that are simultaneously related to both two dimensions. Accordingly, each sampling point is a set of two values, a value in the time axis direction and a value in the amplitude axis direction.

  FIG. 5 conceptually shows an example vocal fold wave 160 and examples of sampling points for the vocal fold wave 160 (indicated by 0 to 30). As shown in FIG. 5, there are 31 sampling points, and as a result, the vocal cord wave 160 is divided into 30 partial waveforms of equal length.

  In the present embodiment, there are 31 sampling points. This number was chosen to minimize the number of parameters while preserving the detailed portions of the waveform. Of course, the number of sampling points is not limited to 31, and the number of sampling points can be selected according to the performance of the apparatus used, the required accuracy, and the like.

  According to the sampling theorem, by sampling at 31 sampling points that are equally spaced from each other, up to the 15th harmonic of the spectrum of each vocal cord waveform is preserved. Therefore, in order to avoid aliasing, the cut-off frequency of the low-pass filter 140 is set to the fifteenth harmonic of the vocal cord waveform.

  Referring back to FIG. 4, the PCA analysis unit 88 further includes a difference calculation unit 144 for calculating the primary difference of the waveform sampled by the re-sampling unit 142. This is because the offset of the amplitude of the vocal tract wave estimated by the inverse filter is unknown, and the influence is eliminated. It is also to avoid unnatural results in PCA analysis due to amplitude differences between the various vocal cord waveforms. As a result, a differential value of the vocal cord wave consisting of 60 parameters sampled at 30 coordinate points is obtained. PCA analysis can be performed on these 60 parameters.

  Referring to FIG. 4, PCA analysis unit 88 further includes a normalization processing unit 146 for performing normalization processing on the differential amount of the vocal cord wave calculated by difference calculation unit 144. The time and amplitude dimensions of the differentiation of the vocal cords are independent of each other, so the PCA analysis may be performed in a manner that improperly reflects the dimension with the larger amount of change. It is desirable to equalize. Therefore, prior to PCA analysis, for each dimension, the average value of the whole is subtracted from the value of each sampling point, and further, the value of each dimension of these sampling points is divided by their standard deviation. The normalization processing unit 146 performs that process.

  The PCA analysis unit 88 further performs a PCA analysis on a total of 60 values at the 30 sampling points normalized by the normalization processing unit 146, and calculates a PCA for calculating up to the fourth principal component. Part 148.

  FIG. 6 shows vocal cord waves obtained from the reference speech waveform 32 and sampling results for them. The waveform shown in FIG. 6 is obtained by averaging the waveforms calculated for a plurality of syllable nuclei determined to represent a specific voice quality for each voice quality. (In fact, since the differential amount of the waveform is obtained, what is shown in FIG. 6 is a waveform obtained by integrating it.) These are hereinafter referred to as prototype vocal cord waves.

  In FIG. 6, 1 to 3 alphabets described at the top of each waveform represent the voice quality of the prototype vocal cord wave. Table 1 below shows the alphabet sets and their meanings.

上記したように、モデル作成部３４によるＰＣＡ分析は声帯波の微分波形に対して行なう。しかし、以下では、この結果得られたプロトタイプ声帯波を分かりやすく比較するために、ＰＣＡ分析の結果を積分して振幅のスケールに戻し、これらの声帯波の分析結果を論じる。その際、声帯波形の先頭サンプルの振幅を０とした。図７に、その結果を示す。

As described above, the PCA analysis by the model creation unit 34 is performed on the differential waveform of the vocal fold wave. However, in the following, in order to compare the resulting prototype vocal cords in an easy-to-understand manner, the results of the PCA analysis are integrated and returned to the amplitude scale, and the analysis results of these vocal cords are discussed. At that time, the amplitude of the first sample of the vocal cord waveform was set to zero. FIG. 7 shows the result.

  The four graphs shown in FIG. 7 show the average value (shown by a solid line) of all the vocal cords that were tested for the first, second, third, and fourth principal components, and the average value ± standard. Deviations (indicated by lines including “+” and “□”, respectively) are shown. Here, the test object is composed of a group of 77 types of vocal cord waves.

  The cumulative total variance explained by these four leading principal components is 57.6%, 80.8%, 88.2%, and 92.1%, respectively. Therefore, as a result of PCA analysis on data represented by a 60-dimensional space, orthogonal basis functions are obtained, but it is understood that only four of them can explain 90% or more of the variance.

  The first graph in FIG. 7 shows the waveform obtained from the first principal component that explains 57.6% of the variance. From this graph, this main component mainly represents the duration of the waveform, that is, the fundamental frequency of the vocal fold wave. The first principal component also describes the accompanying deformation of the waveform. As the period is shortened, the waveform becomes more symmetric and the vertices become more rounded. The longer the period, the wider the waveform and the top becomes flat. Therefore, this first principal component does not reflect the change between the rising portion of the waveform (when the glottis is opened) and the falling portion (when the glottis are closed).

  The second principal component is shown in the second graph of FIG. The second principal component explains 23.2% of the original variance, and mainly explains the fluctuation of the waveform when the glottis is opened. In particular, the central part of the waveform is either high amplitude with a single apex biased slightly to the right of the center, or low amplitude that accounts for the indentation between the two pulses of the diphonic sound. Either. Unlike the first principal component, this second component has no significant relationship with the fundamental period of the waveform.

  The third principal component is shown in the third graph of FIG. The third principal component accounts for 7.4% of the original variance, but seems to primarily reflect the slope of the waveform at the time of opening and the peak shape of the vocal cords. For example, one pole has a sharp slope at the opening and is followed by a relatively flat top, while the other pole has a gentle slope at the opening, followed by a more gentle slope following the peak.

  The fourth graph in FIG. 7 shows a waveform based on the fourth principal component. The fourth principal component accounts for only 3.9% of the original variance, but reflects the pulse skew and closing speed. At one pole, the vocal cord waveform is relatively symmetric and exhibits a more gentle closing gradient, while at the other pole, the vocal cord waveform pulses are slightly biased to the right, indicating a steeper closing gradient.

  From the fifth principal component onwards, more detailed portions of the waveform will be described, but in any case, only portions that are less than 2% of the original variance will be described. Therefore, they are not considered in this embodiment.

  Of course, you may consider even after a 5th main component. It is only necessary to determine which principal component is to be considered in consideration of the available computer resources and the speed required by the application. However, as described above, since most of the change in the waveform can be explained up to the fourth principal component, it seems that there is little practical benefit considering the fifth and subsequent principal components.

  Referring again to FIG. 1, the voice quality conversion device 52 includes a prototype data storage unit 68 for storing the PCA parameter model 36 generated by the model creation unit 34 together with waveform data of each prototype vocal cord wave, and model creation. Stored in the vocal cord waveform extracting unit 60 for extracting a waveform of one vocal cord wave from the input speech waveform 50 in the same manner as that performed by the unit 34, the voice quality specifying information 51, and the prototype data storage unit 68. Voice waveform conversion having a function of converting a vocal cord waveform extracted from the input voice waveform 50 into a vocal cord waveform of a voice quality designated by the user based on the prototype vocal cord wave data and the target voice quality inputted by the user Using a conversion function generation unit 62 for generating a function and the conversion function obtained by the conversion function generation unit 62 By converting the vocal cord waveform output from the band waveform extracting section 60, and a waveform regenerating unit 64 for generating a speech waveform 54 of voice quality designated by the user.

  The vocal cord waveform extraction unit 60 has a function of extracting a vocal cord waveform by performing the same processing as the model creation unit 34 except that the processing target is the input speech waveform 50. Therefore, detailed description of the vocal cord waveform extraction unit 60 will not be given here.

  FIG. 8 is a more detailed block diagram of the conversion function generator 62. Referring to FIG. 8, the conversion function generation unit 62 includes an input / output device 184 that realizes dialogue with the user, such as a keyboard and a monitor, and an input speech waveform 50 that is determined based on the voice quality specifying information 51. A PCA parameter corresponding to the voice quality is presented to the user by using the input / output device 184, and further includes a target setting unit 182 for receiving the value designated by the user as the target of the PCA parameter via the input / output device 184. At this time, the target setting unit 182 refers to the PCA parameter model stored in the prototype data storage unit 68.

  The conversion function generation unit 62 further includes a prototype vocal cord wave corresponding to the voice quality of the input voice waveform 50 determined based on the voice quality specifying information 51 from the waveform of the prototype vocal cord wave corresponding to the target PCA parameter set by the target setting unit 182. And a waveform subtraction processing unit 188 for generating a waveform conversion function by subtracting the waveform.

  FIG. 9 shows one method of displaying the PCA parameters and setting the target PCA parameters by the target setting unit 182. Referring to FIG. 9, two PCA parameter setting areas 202 and 204 are displayed on the output screen 200 of the input / output device 184 shown in FIG. The PCA parameter setting area 202 is for setting values of the first principal component (PC1) and the second principal component (PC2). The PCA parameter setting area 204 is for setting values of the third principal component (PC3) and the fourth principal component (PC4).

  The PCA parameter setting areas 202 and 204 can display points represented by two-dimensional coordinates (PC1, PC2) and (PC3, PC4), respectively. By designating the voice quality of the input voice waveform 50, PC1 to PC4 are determined from the glottal waveform model stored in the prototype data storage unit 68. As points corresponding thereto, points 210 and 214 can be displayed in the PCA parameter setting areas 202 and 204, respectively. The first principal component to the fourth principal component of the input speech waveform 50 are specified by the display of these two points.

  On the display, for example, when the user newly designates a point 212 in the PCA parameter setting area 202, the values of the targets of PC1 and PC2 are determined as values on each axis corresponding to the point 212. Similarly, when the user newly designates a point 216 in the PCA parameter setting area 204, the target values of PC3 and PC4 are determined. The target setting unit 182 illustrated in FIG. 8 acquires target values from the first principal component to the fourth principal component set by the user in this way.

  Of course, the method shown in FIG. 9 is only one method for setting a target. Besides this, for example, a method for directly inputting values for each principal component, a prototype prepared in advance, and the like are displayed. Various methods such as a method for designating a target prototype from among them can be used.

  FIG. 10 is a detailed block diagram of the waveform regeneration unit 64 shown in FIG. Referring to FIG. 10, waveform regeneration unit 64 adds the conversion function generated by conversion function generation unit 62 to each vocal cord wave output from vocal cord waveform extraction unit 60, thereby obtaining input voice waveform 50. A waveform adding unit 240 for correcting the extracted vocal fold wave, and adjusting the voice pitch and the duration of the utterance to an appropriate one for the converted vocal cord waveform output from the waveform adding unit 240, A waveform adjusting unit 242 for performing processing that avoids unnatural foldback of the vocal cord waveform that is not realistic in nature or caused by extreme deformation.

  As described above, the differentiation of the normalized vocal cord waveform is represented by 30 time coordinate and amplitude coordinate pairs, respectively. Therefore, the conversion function generated by the conversion function generation unit 62 deforms the waveform not only in the amplitude axis direction but also in the time axis direction. As a result, the voice quality may change inappropriately. Therefore, the waveform adjustment unit 242 adjusts the waveform so that such a problem does not occur.

  The waveform regeneration unit 64 further generates an inverse filter (inverse filter shown in FIG. 3) generated in the vocal cord waveform extraction unit 60 shown in FIG. 1 with respect to the differentiation of the converted speech waveform output from the waveform adjustment unit 242. An inverse / inverse filter 244 for restoring the original formant and outputting the converted speech signal is applied by applying an inverse filter (equivalent to the processing unit 82).

-Operation-
The voice quality conversion system 30 having the above-described configuration operates as follows. There are two aspects to the operation of the voice quality conversion system 30. The first aspect relates to the process of creating the PCA parameter model 36, and the second aspect is an aspect in which the voice waveform 54 is generated by using the PCA parameter model 36 and changing the voice quality of the input voice waveform 50 according to the user input. . Hereinafter, the first aspect and then the second aspect will be described in order.

  First, the first aspect will be described. Referring to FIG. 1, it is assumed that a reference speech waveform 32 is prepared in advance. Each of the reference speech waveforms 32 is pre-labeled to specify the voice quality of the voice.

  Of the model creation unit 34, the syllable nucleus extraction unit 80 and the formant estimation unit 81 perform the above-described processing on each speech of the reference speech waveform 32 to extract syllable nuclei. That is, referring to FIG. 2, the syllable nucleus extraction unit 80 extracts syllable nuclei based on the power distribution waveform on the time axis of the speech waveform. The formant estimation unit 81 estimates the formant frequency and band in each syllable nucleus. The syllable nucleus extracted in this manner indicates a portion of the speech of the reference speech waveform 32 that is uttered as determined by the speech mechanism of the speaker.

  The inverse filter processing unit 82 shown in FIG. 2 removes the influence of the vocal tract by performing an inverse filter process on each of the syllable nuclei extracted by the syllable nucleus extraction unit 80 and the formant estimation unit 81. That is, with reference to FIG. 3, the inverse filter generation unit 120 generates a parameter for an inverse filter for removing the influence of the vocal tract for each syllable nucleus by optimization based on analysis and synthesis. This parameter varies with time. The audio signal from which the low-frequency component and the component above the fourth formant have been removed by the high-pass filter 122 and the low-pass filter 124 is supplied to the inverse filter application unit 126, and the inverse filter application unit 126 uses the first 4 of the vocal tract from the audio signal. The effect of the two resonance components is eliminated. The output of the inverse filter application unit 126 is given to the volume velocity waveform detection unit 84 shown in FIG.

  The volume velocity waveform detector 84 detects the volume velocity waveform of the glottal airflow in the syllable nucleus of each voice based on the output of the inverse filter processor 82. The detected volume velocity waveform is normalized by the normalizing unit 86 and supplied to the waveform extracting unit 87.

  The waveform extraction unit 87 extracts a waveform for one cycle existing in the vicinity of the center of the syllable nucleus from the normalized volume velocity waveform, and provides it to the PCA analysis unit 88.

  Referring to FIG. 4, low-pass filter 140 of PCA analysis unit 88 removes a frequency component above the cutoff frequency determined by the 15th harmonic of the target speech waveform, and sends the speech signal to resampling unit 142. give. The re-sampling unit 142 samples the speech waveform at 31 points selected to be divided into 30 partial waveforms that are equal to each other on the waveform of the input speech waveform, and sets a pair of time and amplitude. 31 are generated. The difference calculation unit 144 outputs the differential amount of the vocal fold wave sampled at 30 sampling points by taking the primary difference of these 31 pairs. The normalization processing unit 146 subtracts the average value obtained over one entire waveform to be processed from the time and amplitude values constituting this differential amount, and further calculates the resulting value by their standard deviation. Normalization is performed by division, and the obtained 60 values (30 pairs of time and amplitude differentials) are given to the PCA calculation unit 148. The PCA calculation unit 148 performs PCA analysis on the parameters given in this way, calculates the first principal component to the fourth principal component for speech representing each voice quality, and the corresponding vocal cord wave of the reference speech A PCA parameter model 36 is created together with the waveforms of The PCA parameter model 36 is stored in the prototype data storage unit 68 of the voice quality conversion device 52.

  This completes the PCA parameter model 36 creation process.

  Next, the operation of the voice quality conversion device 52 in the second aspect will be described. Referring to FIG. 1, vocal cord waveform extraction unit 60 extracts the vocal cord waveform of input speech waveform 50 by performing the same processing as model creation unit 34 on input speech waveform 50, and regenerates the waveform. 64.

  Referring to FIG. 8, target setting unit 182 receives voice quality specifying information 51 that specifies the voice quality corresponding to input speech waveform 50, and refers to the PCA model stored in prototype data storage unit 68 to refer to the voice quality. The PCA analysis corresponding to the first to fourth principal component values PC1 to PC4 is presented to the user in the format shown in FIG. The user uses the input / output device 184 to change these values to values corresponding to the desired voice quality by the operation as described above. The target setting unit 182 sets the value changed by the user as the PCA target value, and gives the value to the waveform subtraction processing unit 188. The waveform subtraction processing unit 188 subtracts the prototype glottal wave waveform designated as the voice quality of the input speech from the prototype glottal wave waveform corresponding to the target value of the PCA parameter set by the target setting unit 182 to obtain the waveform. 1 is generated and applied to the waveform regenerator 64 shown in FIG.

  Referring to FIG. 10, waveform adding section 240 of waveform regenerating section 64 adds the conversion function given from waveform subtraction processing section 188 to the vocal cord waveform obtained from input speech waveform 50, and the result Is given to the waveform adjustment unit 242. The waveform adjusting unit 242 adjusts the output of the waveform adding unit 240 so as not to be unnatural as described above, and gives the result to the inverse / inverse filter 244. The inverse / inverse filter 244 performs an inverse filter (inverse / inverse filter) process of the inverse filter generated in the vocal cord waveform extracting unit 60 shown in FIG. As a result, a change due to the vocal tract is again applied to the glottal waveform generated by the waveform adjustment unit 242 to obtain a voice waveform after the voice quality is changed. Thus, the speech waveform 54 having the same utterance content as the input speech waveform 50 and having the voice quality converted into the voice quality determined by the PCA parameter set by the user is output.

-Experimental result-
FIG. 11 shows an example of the processing result according to the present embodiment. FIG. 11 shows a contrast of a spectrogram 260 of a part of an utterance by Modal of Laver (Non-patent Document 1) and a spectrogram 262 after the utterance is converted into a more clear voice. In this example, the transformation function was generated based on the Modal prototype and the target was set to Creamy.

  In this embodiment, the voice quality of any input voice is sufficiently close to the voice quality of the prototype prepared in advance, and if the prototype is selected as the basis of the conversion function, the voice quality of the input voice is almost correctly converted to the target. Is assumed. In the example shown in FIG. 11, it is assumed that the voice quality of the input voice is sufficiently close to the voice quality of Modal.

  However, in reality, the glottal waveform fluctuates greatly even during utterances that are felt as having a specific voice quality as a whole. Therefore, the above assumption is not always true. Nevertheless, it can be seen from FIG. 11 that this conversion clearly preserves the acoustic information and the duration of the utterance. Furthermore, as can be seen from the vertical stripes, this conversion results in longer vocal cord waves. This is also predicted from the fact that F0 is shifting in the direction of a more crisp voice. By actually synthesizing speech based on this speech waveform, it can be seen that the speech after conversion has the same speech information as the speech before conversion, and the voice quality is clearly close to a clear voice.

  Both the model creation unit 34 and the voice quality conversion device 52 constituting the voice quality conversion system 30 described above can be realized by computer hardware and a computer program operating on the computer hardware. As this computer hardware, general-purpose hardware can be used as long as it has equipment for handling audio signals. Further, each functional block of the apparatus described above can be realized by a program by those skilled in the art based on the description in this specification. Such a program is also a piece of data and can be stored in a storage medium and distributed.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is a block diagram of a voice quality conversion system 30 according to an embodiment of the present invention. FIG. 2 is a detailed block diagram of a model creation unit 34 in the voice quality conversion system 30 shown in FIG. 1. FIG. 3 is a more detailed block diagram of an inverse filter processing unit 82 shown in FIG. 2. It is a more detailed block diagram of the PCA analysis part 88 shown in FIG. It is a figure for demonstrating the method of the resampling performed by the resampling part 142 shown in FIG. It is a figure which shows the waveform of a prototype vocal cord wave, and a sampling result. It is a figure for demonstrating the change of the waveform represented by the 4th main component from the 1st main component. FIG. 2 is a detailed block diagram of a conversion function generation unit 62 shown in FIG. It is a figure for demonstrating the setting method of the target by the target setting part 182 shown in FIG. FIG. 2 is a detailed block diagram of a waveform regeneration unit 64 shown in FIG. It is a spectrogram which shows the experimental result by one embodiment of this invention.

Explanation of symbols

30 voice quality conversion system, 32 reference speech waveform, 34 model creation unit, 36 PCA parameter model, 50 input speech waveform, 51 voice quality identification information, 52 voice quality conversion device, 54 speech waveform, 60 input waveform extraction unit, 62 conversion function generation unit , 64 waveform regeneration unit, 80 syllable nucleus extraction unit, 81 formant estimation unit, 82 inverse filter processing unit, 84 volume velocity waveform detection unit, 86 normalization unit, 87 vocal cord waveform extraction unit, 88 PCA analysis unit, 120 inverse filter Generator, 122 high-pass filter, 124,140 low-pass filter, 126 inverse filter application unit, 142 re-sampling unit, 144 difference calculation unit, 146 normalization processing unit, 148 PCA calculation unit, 182 target setting unit, 184 input / output device, 188 Waveform subtraction processing unit, 240 Waveform addition unit, 242 Waveform adjustment unit, 244 And inverse filter 246 integration processing unit

Claims (20)

A vocal fold waveform that estimates a unit waveform of a vocal fold wave when a portion of a plurality of reference sound waveforms prepared in advance corresponding to a predetermined voice quality satisfies a predetermined condition when the portion is uttered An estimation step;
A parameterization step of parameterizing each of the unit waveforms of the vocal fold according to a predetermined parameterization method;
A principal component analysis step of obtaining a principal component representation of each of the unit waveforms of the vocal fold wave by performing a principal component analysis on the unit waveform of the parameterized vocal fold wave;
Outputting a waveform of each unit waveform of the vocal fold wave and a principal component expression corresponding to the waveform as a voice quality model corresponding to the speech waveform from which the vocal fold wave is obtained. . The vocal cord waveform estimation step includes:
Extracting syllable nuclei of the plurality of speech waveforms, each prepared in advance corresponding to a predetermined sound quality;
Applying, to each of the extracted syllable nuclei, an inverse filter for detecting the volume velocity waveform of glottal airflow when sound is generated by removing the influence of the vocal tract;
The voice quality model generation method according to claim 1, further comprising: a unit waveform extraction step of extracting a unit waveform of the vocal cord wave from each of the syllable nuclei after the inverse filter is applied. The unit waveform extraction step starts from a minimum portion of the volume velocity waveform that exists in the central part of the syllable nucleus, and then goes back by one period determined by the basic frequency of a predetermined region including the syllable nucleus. The voice quality model generation method according to claim 2, comprising a step of extracting up to a portion as the unit waveform. The voice quality model generation method according to claim 2, further comprising a step of normalizing a volume velocity waveform of the glottal airflow according to a predetermined normalization method prior to the unit waveform extraction step. The principal component analysis step includes:
The method includes: obtaining principal component representations of principal components from the head to a predetermined number of each of the unit waveforms of the vocal fold wave by performing principal component analysis on the unit waveform of the parameterized vocal fold wave. The voice quality model generation method according to any one of claims 1 to 4. The voice quality model generation method according to claim 5, wherein the predetermined number of principal components is from a first principal component to a fourth principal component. The parameterizing step includes a resampling step of resampling the unit waveform of the vocal fold wave at a predetermined number of sampling points that divide the unit waveform of the vocal fold wave into a plurality of equal length portions. 7. The voice quality model generation method according to any one of 6. A differential step of obtaining a differential data string of the unit waveform of the vocal fold wave by taking a difference of the unit waveform of the vocal fold wave resampled by the re-sampling step;
The principal component analysis step includes the step of acquiring a principal component expression for each differential amount of the unit waveform of the vocal fold wave by performing the principal component analysis on the differential data string. Voice quality model generation method. Each of the differential data strings obtained by the differentiation step includes a pair of a difference in re-sampling time and a difference in unit waveform of the vocal fold wave corresponding to the difference in re-sampling time,
The voice quality model generation method further includes the influence of fluctuation in the time axis direction and the influence of fluctuation in the amplitude direction on each of the differential data strings obtained by the step of obtaining the differential data string prior to the principal component analysis step. The voice quality model generation method according to claim 8, further comprising a step of performing a predetermined normalization process for equalizing. A plurality of prototype vocal cord wave unit waveforms each associated with a predetermined voice quality, and a predetermined number of principal component representations from the head obtained by a predetermined principal component analysis for each of the plurality of prototype vocal cord wave unit waveforms A voice quality conversion method for converting the voice quality of an input voice waveform using a glottal waveform model consisting of
A unit waveform extraction step for extracting a unit waveform of each vocal fold wave from a portion satisfying a predetermined condition in the input speech waveform;
Based on the glottal waveform model corresponding to the voice quality specified in advance as the voice quality of the input voice waveform and the glottal waveform model corresponding to the voice quality specified by the user, the unit waveform of the vocal cord wave extracted from the input voice waveform A voice waveform conversion step including a voice waveform generation step of generating an output voice waveform by converting the voice quality specified by the user. The speech waveform generation step includes
Selecting a first prototype vocal cord wave from a glottal waveform model corresponding to the voice quality of the input speech waveform;
Selecting a second prototype vocal cord wave from a glottal waveform model corresponding to the voice quality specified by the user;
A conversion function for calculating a conversion function for converting the input speech waveform into a speech waveform of voice quality designated by the user by performing a predetermined calculation between the first waveform and the second waveform. A calculation step;
The voice quality conversion method according to claim 10, further comprising: generating the output voice waveform by applying the conversion function to a unit waveform of a vocal cord wave of the input voice waveform. The voice conversion method according to claim 11, wherein the conversion function calculating step includes a step of calculating the conversion function by subtracting the first waveform from the second waveform. The voice quality conversion method according to claim 12, wherein the voice waveform generation step includes the step of generating the output voice waveform by adding the conversion function to a unit waveform of a vocal cord wave of the input voice waveform. The unit waveform extraction step includes:
Extracting a syllable nucleus of the input speech waveform;
Applying, to each of the extracted syllable nuclei, an inverse filter for detecting the volume velocity waveform of glottal airflow when sound is generated by removing the influence of the vocal tract;
The voice quality conversion method according to claim 1, further comprising: extracting a unit waveform of the vocal cord wave from each of the syllable nuclei after the inverse filter is applied. The step of extracting the unit waveform starts from a minimum portion of the volume velocity waveform existing in the center of the syllable nucleus, and from there for one period determined by the fundamental frequency of a predetermined region including the syllable nucleus. The voice quality conversion generation method according to claim 14, further comprising a step of extracting a retroactive portion as the unit waveform. The voice quality conversion method according to claim 14 or 15, further comprising a step of normalizing a volume velocity waveform of the glottic airflow according to a predetermined normalization method prior to the step of extracting the waveform. The voice quality conversion method according to any one of claims 1 to 16, wherein the predetermined number of principal component representations from the top are based on a first principal component to a fourth principal component. A computer program configured to, when executed by a computer, operate the computer so as to realize all the steps according to any one of claims 1 to 17. A computer programmed by the computer program according to claim 18. A computer-readable recording medium on which the computer program according to claim 18 is recorded.

Images