JP4999757B2 - Speech analysis / synthesis apparatus, speech analysis / synthesis method, computer program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech analysis and synthesis device that outputs speech which is easy to hear, by converting a local speech speed for each part of input speech to a desired speed, and by reducing local variation of the speech speed of the input speech. <P>SOLUTION: In a speech analysis section 100, a speech spectrum and a parameter of a sound source or the like are extracted from a speech signal. In a speech conversion section 200, predetermined conversion is performed on the speech spectrum and the parameter of the sound source or the like, based on speed information of an articulation parameter, and the speech signal is generated by a vocoder type speech synthesis section 300. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a speech analysis / synthesis device, a speech analysis / synthesis method, a computer program, and a recording medium that can output a speech signal obtained by converting a speech rate of an input speech signal into a desired speed.

  So far, as a method of synthesizing speech by converting the speech style of a given speech signal, a method focusing on the speech spectrum has been proposed (for example, see Non-Patent Document 1). However, it is difficult to capture the features of the utterance style from the speech spectrum alone, and there is a problem that sufficient quality cannot be obtained at the present time.

  The method of capturing speech from the viewpoint of articulatory motion and converting the utterance style based on articulatory motion is expected to be more intuitive and more accurate than the case of using the speech spectrum described above. However, due to the non-linear mapping relationship between the articulatory motion and the speech spectrum, there is a problem that the features of the utterance style converted based on the articulatory motion are not well reflected in the speech spectrum after mapping (see, for example, Patent Document 1). ).

  The most basic utterance style control is conversion of speech speed, but there is a problem that if the time base expansion / contraction is applied to the audio signal itself, there is a problem that the fundamental frequency also changes. (For example, see Non-Patent Document 2). However, in the extraction of pitch marks necessary for performing the pitch synchronization analysis, the conventional method based on the threshold processing of the absolute value of the LPC (linear prediction coefficient) prediction residual signal is used to detect pitch marks particularly in a female voice having a high fundamental frequency. It is known that extraction cannot be performed well (see, for example, Non-Patent Document 3). Pitch synchronization analysis is capable of extracting a stable audio spectrum and sound source information that is not affected by the fundamental frequency compared to audio signal analysis using fixed window length and fixed frame shift length. Is known (see, for example, Non-Patent Document 3).

In speech synthesis, when a driving sound source that switches between a single pulse sequence and white noise is used, the quality of the synthesized speech signal is not good. Therefore, there is a method of using a multi-pulse sequence determined so as to minimize an error with a phase-equalized audio signal instead of a single pulse sequence, but a buzzy sound is produced at a portion where voiced and unvoiced are switched. There is a problem (see, for example, Patent Document 2). On the other hand, as a method for improving buzzy sound, a driving sound source that mixes a single pulse sequence and white noise based on voiced / unvoiced judgment for each frequency band has been proposed, but further quality improvement is required. (For example, refer nonpatent literature 4).
Tachibana, M., Yamagishi, J., Masuko, T., and Kobayashi, T., “Speech synthesis with various emotional expressions and speaking styles by style interpolation and morphing,” IEICE Trans. Information and Systems, E88-D, 11, pp. 2484-2491 (2005). Moulines, E., and Charpentier, F., “Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones,” Speech Communication, 9, pp. 453-467 (1990). Miyoshi, Y., Yamato, K., Mizoguchi.R., Yanagida, M., and Kakusho, O., “Analysis of speech signals of short pitch period by a sample-selective linear prediction,” IEEE Trans. Signal Processing, 35, 9, pp. 1233-1240 (1987). McCree, AV, and Barnwell, TP, “A mixed excitation LPC vocoder model for low bit rate speech coding,” IEEE Trans. Speech and Audio Processing, 3, 4, pp. 242-249 (1995). Japanese Patent No. 3421798 Japanese Patent Publication No. 7-82360

  In the conversion of speech style based on articulatory motion, there is a problem that a speech signal of sufficient quality cannot be synthesized when the mapping from the articulation parameter proposed in Patent Document 1 to the speech spectrum is used. Therefore, there is a need for a technique that directly utilizes the characteristics of the articulation parameters related to the speech style conversion without converting the articulation parameters to the speech spectrum.

  The present invention has been made in view of such circumstances, and an object of the present invention is to be able to output a voice obtained by converting a local speech speed of each part of an input voice into a desired speed. Another object of the present invention is to provide a speech analysis / synthesis device, a speech analysis / synthesis method, a computer program, and a recording medium capable of reducing local variations in the utterance speed of input speech and outputting easily audible speech.

The present invention has been made to solve the above-described problem, and measures a voice that is measured simultaneously as a voice signal and collects measurement data of articulatory movement, and a voice analyzer that analyzes the measurement data A speech analysis and synthesis device comprising: a speech conversion unit that performs predetermined conversion on the analysis result of the speech analysis unit; and a vocoder-type speech synthesis unit that synthesizes speech based on the conversion result of the speech conversion unit. The voice analysis unit detects a voice section from the voice signal and calculates a fundamental frequency in the voice section; and a pitch having a pulse sequence corresponding to a pitch period using the fundamental frequency. A pulse sequence generation unit that generates a waveform, an LPC coefficient calculation unit that performs linear prediction analysis based on the speech signal and calculates an LPC (Linear Prediction Analysis) coefficient, and the speech signal An LPC residual calculation unit that calculates an LPC prediction residual waveform by an inverse filter having the LPC coefficient as a filter coefficient, an LSP coefficient calculation unit that calculates an LSP (Line Spectrum Pair) coefficient from the LPC coefficient, and the detected A pitch mark calculation unit that extracts a pitch mark that maximizes the cross-correlation between the absolute value of the LPC prediction residual waveform and the absolute value of the pitch waveform within the speech interval, and within the detected speech interval, A phase-equalized speech calculation unit that generates phase-equalized speech in which a phase component of the speech signal is equalized to a constant phase based on the speech signal, the pitch mark, and the LPC prediction residual waveform; and the detection A phase equalization pulse sound source filter calculation unit for calculating a filter coefficient of a phase equalization pulse sound source model based on the phase equalized sound within the detected voice interval; A voiced intensity calculation unit that calculates a voice intensity for each frequency band by a predetermined calculation method based on an audio signal, and white noise by a predetermined calculation method based on the audio signal in the detected voice section A white noise gain calculation unit for calculating a gain, and the audio conversion unit calculates an articulation parameter speed as an articulation speed based on the measurement data of the articulation motion, and the articulation speed In response, an LSP coefficient converter that performs a predetermined conversion on the LSP coefficient, a basic frequency converter that performs a predetermined conversion on the fundamental frequency according to the articulation speed, and the phase equalized pulse sound source according to the articulation speed A phase equalization pulse sound source filter converter that performs a predetermined conversion on the filter coefficient of the model, and a white noise gain converter that performs a predetermined conversion on the white noise gain according to the articulation speed; A voiced intensity conversion unit that performs a predetermined conversion on the sound intensity for each frequency band according to the articulation speed, and the speech synthesis unit includes the fundamental frequency converted by the fundamental frequency conversion unit, and the phase A phase equalization pulse sound source is generated based on the filter coefficient converted by the equalization pulse sound source filter conversion unit and the phase equalization pulse sound source model, and for each frequency band converted by the voiced intensity conversion unit. Based on the voice intensity, the generated phase equalization pulse sound source is mixed in the voiced band, and the driving sound source generating unit for generating the driving sound source in which the white noise is mixed in the unvoiced band, and converted by the LSP coefficient converting unit. comprises a convolution unit for synthesizing an audio signal from the LSP coefficients and the output signal of the excitation, the, if the voiced strength calculation unit calculates the voice level If the white noise gain calculator calculates the white noise gain, and, when said excitation generating unit generates said phase equalization pulse excitation, to calculate or generate analysis window length as two pitches cycles, it Is a speech analysis and synthesis device characterized by

  The speech synthesis unit may further include an LPC coefficient calculation unit that converts the LSP coefficient converted by the LSP coefficient conversion unit into an LPC coefficient, and the convolution operation unit converts the LPC coefficient in the LPC coefficient calculation unit. The speech analysis and synthesis apparatus according to claim 1, wherein the speech signal is synthesized by convolving the LPC coefficient thus generated and the output signal of the driving sound source.

Further, according to the present invention, the LSP coefficient conversion unit, the fundamental frequency conversion unit, the phase equalization pulse sound source filter conversion unit, the white noise gain conversion unit, and the voiced intensity conversion unit of the sound conversion unit are each an articulation speed at time t. Assuming that the articulation parameter is x _{t, i} (i = 1,..., N: horizontal and vertical positions such as lips and tongue), the RMS distance dx _t is used, and “dx _t = sqrt (Σ _i (X _{t, i} −x _{t−1, i} ) × (x _{t, i} −x _{t−1, i} ) / n), where sqrt is the root number and the unit of articulation speed is mm ” An average articulation speed avedx is calculated by dividing the sum of articulation speeds of the entire section by the length (number of frames) of the entire speech section. Further, at all times t, “dx _k ≦ t × avedx and dx _{k +} 1> t × _avedx "to become asked for k, time The parameters in, to linear interpolation by the following equation, _{"((dx k + 1 -t ×} avedx) × p k + (t × avedx-dx k) × p k + 1) / (dx k + 1 -dx k), where , P _k is a speech analysis / synthesis device characterized by the LSP coefficient, the fundamental frequency, the filter coefficient of the phase equalization pulse sound source, the white noise gain, or the voiced intensity for each frequency band at time k ”.

The present invention also provides a data input unit that measures simultaneously measured speech as a speech signal and collects measurement data of articulatory movement, a speech analysis unit that analyzes the measurement data, and an analysis result of the speech analysis unit. A speech analysis / synthesis method in a speech analysis / synthesis apparatus comprising: a speech conversion unit that performs predetermined conversion; and a vocoder-type speech synthesis unit that synthesizes speech based on a conversion result of the speech conversion unit, wherein the speech analysis unit To detect a speech section from the speech signal, calculate a fundamental frequency in the speech section, and generate a pitch waveform having a pulse sequence according to a pitch period using the fundamental frequency A sequence generation procedure, an LPC coefficient calculation procedure for performing an LPC (Linear Prediction Analysis) coefficient by performing linear prediction analysis based on the speech signal, the speech signal, An LPC residual calculation procedure for calculating an LPC prediction residual waveform using an inverse filter having the LPC coefficient as a filter coefficient, an LSP coefficient calculation procedure for calculating an LSP (Line Spectrum Pair) coefficient from the LPC coefficient, and the detected A pitch mark calculation procedure for extracting a pitch mark that maximizes the cross-correlation between the absolute value of the LPC prediction residual waveform and the absolute value of the pitch waveform within a speech interval, and within the detected speech interval, A phase-equalized speech calculation procedure for generating phase-equalized speech in which the phase component of the speech signal is equalized to a constant phase based on the speech signal, the pitch mark, and the LPC prediction residual waveform; and the detection A phase equalization pulse sound source filter calculation procedure for calculating a filter coefficient of a phase equalization pulse sound source model based on the phase equalization sound, and the detected voice interval And a voiced intensity calculation procedure for calculating the voice intensity for each frequency band by a predetermined calculation method based on the voice signal, and a calculation method predetermined based on the voice signal within the detected voice section. And a white noise gain calculation procedure for calculating a white noise gain at the sound conversion unit, and based on the measurement data of the articulation motion, the articulation speed calculation procedure for calculating the speed of the articulation parameter as the articulation speed, An LSP coefficient conversion procedure for performing a predetermined conversion on the LSP coefficient according to the articulation speed, a basic frequency conversion procedure for performing a predetermined conversion on the fundamental frequency according to the articulation speed, and the phase according to the articulation speed A phase equalization pulse sound source filter conversion procedure for performing predetermined conversion on the filter coefficient of the equalized pulse sound source model, and a predetermined conversion on the white noise gain according to the articulation speed. A white noise gain conversion procedure to be performed, and a voiced strength conversion procedure to perform a predetermined conversion on the voice strength for each frequency band according to the articulation speed, and the voice synthesis unit performs the conversion in the basic frequency conversion procedure. A phase equalized pulse sound source is generated based on the fundamental frequency, the filter coefficient converted by the phase equalized pulse sound source filter conversion procedure, and the phase equalized pulse sound source model, and the voiced intensity conversion procedure A driving sound source generation procedure for generating a driving sound source in which the generated phase equalization pulse sound source is mixed in the voiced band and white noise is mixed in the unvoiced band, based on the sound intensity for each frequency band converted in A convolution operation procedure for synthesizing an audio signal from the LSP coefficient converted by the LSP coefficient conversion procedure and the output signal of the driving sound source is performed. The case where the voice level in the voicing strength calculation procedure is calculated, the case where the in white noise gain calculation procedure white noise gain is calculated, and the phase equalization pulse excitation in the excitation generation procedure is produced In this case, the speech analysis and synthesis method is characterized in that the analysis window length is calculated or generated as two pitch periods .

  In the present invention, an LPC coefficient calculation procedure for converting an LSP coefficient converted by the LSP coefficient conversion procedure into an LPC coefficient is performed by the speech synthesizer, and the conversion is performed in the LPC coefficient calculation procedure by the convolution calculation procedure. A speech analysis and synthesis method characterized in that a speech signal is synthesized by convolving the LPC coefficient thus generated and the output signal of the driving sound source.

Further, the present invention provides an LSP coefficient conversion procedure, a basic frequency conversion procedure, a phase equalization pulse sound source filter conversion procedure, a white noise gain conversion procedure, and a voiced intensity conversion procedure, respectively, performed by the sound conversion unit. Using the RMS distance dx _t when the articulation parameter is x _{t, i} (i = 1,..., N: horizontal and vertical positions such as lips and tongue) as the articulation speed at time t, “Dx _t = sqrt (Σ _i (x _{t, i} −x _{t−1, i} ) × (x _{t, i} −x _{t−1, i} ) / n), where sqrt is the root sign, the articulation speed The unit is mm ”, and the procedure of calculating the average articulation speed avedx by dividing the sum of the articulation speeds of the entire speech section by the length (number of frames) of the entire speech section, and at all times t, dx _k <= t × avedx, One _{dx k +} 1> t × _avedx "become sought k, the parameters at time t, a step of linear interpolation by the following equation," _{((dx k + 1 -t ×} avedx) × p k + (t × avedx-dx _{k) × p k + 1)} / (dx k + 1 -dx k), where, _{P k} is at time k, the LSP coefficients, the fundamental frequency, phase equalization pulse excitation filter coefficients, white noise gain, or each frequency band The voice analysis and synthesis method is characterized in that “voiced intensity” is performed.

  The present invention also includes a data input unit that collects measurement data of voice and articulation movement, a voice analysis unit that analyzes the measurement data, a voice conversion unit that performs predetermined conversion on the analysis result of the voice analysis unit, A computer program for causing a computer in a speech analysis / synthesis apparatus including a vocoder-type speech synthesis unit that synthesizes speech based on a conversion result of the speech conversion unit to execute the above-described procedure.

  The present invention also provides a computer-readable recording medium that stores the above-described computer program.

  In the speech analysis and synthesis apparatus and speech analysis and synthesis method of the present invention, the speech analysis unit extracts the speech spectrum, sound source parameters, and the like from the speech signal, and the speech conversion unit based on the speed information of the articulation parameters, Since the voice signal is generated by the vocoder-type voice synthesizer, the voice spectrum and the sound source parameters are subjected to predetermined conversion, so that it is possible to synthesize high-quality voice that gives various utterance styles. it can. For this reason, for example, it is possible to output a voice in which the local speech rate for each part of the input voice is converted to a desired speed without changing the voice pitch (pitch) of the input voice. That is, by converting the part where the speech rate is fast and slowing the slow part, it is possible to reduce the local variation in the speech rate of the input speech and obtain a speech that is easy to hear.

In the speech analysis / synthesis apparatus and speech analysis / synthesis method of the present invention, the LSP coefficient, fundamental frequency, phase equalization pulse sound source filter coefficient, white noise gain, or frequency band obtained by the speech analysis unit is obtained. For the voiced intensity, the average articulation speed averagex is calculated based on the speed information of the articulation parameter, and _{k that} satisfies “dx _k ≦ t × avedx and dx _{k + 1} > t × avedx” is obtained at all times t. , “((Dx _{k + 1} −t × avedx) × p _k + (t × avedx−dx _k ) × p _{k + 1} ) / (dx _{k + 1} −dx _k ), where P = _k is an LSP coefficient at time k, a fundamental frequency, a phase equalization pulse sound source filter coefficient, a white noise gain, or a voiced intensity for each frequency band ”. Therefore, it is possible to generate a voice having an utterance style when a human utters at a constant articulation speed.

FIG. 1 is a diagram showing a configuration of a speech analysis / synthesis apparatus according to an embodiment of the present invention.
The speech analysis / synthesis apparatus of the present invention shown in FIG. 1 is configured by connecting a microphone (microphone) 2 and a two-dimensional magnetic sensor system 3 to a speech analysis / synthesis apparatus 1.

  The speech analysis / synthesis apparatus 1 includes a main control unit 11 having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. It is a control unit for controlling the processing operation of each processing unit in the apparatus 1 in an integrated manner. A computer system is constituted by the CPU in the main control 11.

  The data input unit 12 is connected to the microphone 2 and the two-dimensional magnetic sensor system 3 via the interface unit 13. The data input unit 12 measures simultaneous measurement data of an audio signal measured by the microphone 2 and articulatory movement (lip or tongue movement) measured by the two-dimensional magnetic sensor system 3. In the speech analysis / synthesis apparatus 1 shown in FIG. 1, for example, the speech signal is measured at 16 kHz sampling, and the articulation parameters are 1 point near the lower gum, 1 point on each of the upper and lower lips, and 3 points on the tongue. The horizontal and vertical positions of the points are measured at a rate of 250 times per second (12th order vector).

  The speech analysis / synthesis apparatus 1 includes a speech analysis unit 100, a speech conversion unit 200, and a vocoder-type speech synthesis unit 300.

  The speech analysis unit 100 is based on the speech signal collected by the data input unit 12, LPC (Linear Prediction Coefficient), LSP (Line Spectrum Pair) Coefficient, Phase Equalized Speech, Phase Equalized Pulse Sound Source Model Filter Coefficient, Calculation processing such as voiced intensity for each frequency band and white noise gain is performed.

  Based on the measurement data of the articulatory motion, the voice conversion unit 200 converts LSP coefficients, fundamental frequencies, filter coefficients of the phase equalization pulse sound source model, white noise gain, and voiced intensity for each frequency band (for example, linear Interpolation process). The voice synthesizer 300 generates a driving sound source using the parameters converted by the voice converter 200 and synthesizes a voice signal based on a signal from the driving sound source. The voice output unit 14 outputs a voice based on the voice signal synthesized by the voice synthesis unit 300.

FIG. 2 is a diagram illustrating a configuration of the voice analysis unit 100.
The fundamental frequency calculation unit 101 in the speech analysis unit 100 shown in FIG. 2 performs a process of detecting a speech section and extracting a fundamental frequency based on the power of the speech signal.

  The pulse sequence generation unit 102 performs a process of generating a pulse sequence signal ex having a pulse sequence corresponding to the pitch period using the fundamental frequency obtained by the fundamental frequency calculation unit 101 within the speech section. The pulse sequence signal ex having a pulse sequence corresponding to this pitch period is called a pitch waveform (see, for example, the pitch waveform shown in FIG. 8B).

  The LPC coefficient calculation unit 103 performs normal linear prediction analysis using the audio signal, and calculates an LPC (linear prediction analysis) coefficient. The LPC residual calculation unit 104 obtains an LPC prediction residual waveform res by an inverse filter using the LPC coefficient as a filter coefficient (see, for example, the LPC prediction residual waveform shown in FIG. 8C).

  The LSP coefficient calculation unit 105 calculates an LSP (line spectrum pair) coefficient based on the LPC coefficient calculated by the LPC coefficient calculation unit 103 and performs processing for holding the LSP coefficient.

  The pitch mark calculation unit 106 performs a process of extracting a pitch mark based on the LPC prediction residual signal res and the pulse sequence signal ex. (For example, see the pitch mark shown in FIG. 8D).

  The phase-equalized speech calculation unit 107 uses a speech signal, a pitch mark obtained by the pitch mark calculation unit 106, and a waveform of the LPC prediction residual signal within a speech interval to obtain a constant phase component of the speech signal. A phase equalized sound equalized to the phase is generated (see, for example, Patent Document 2).

The phase equalization pulse sound source filter calculation unit 108 generates a phase equalization pulse sound source model that minimizes the perceptually weighted error between the phase equalized speech and the synthesized speech signal within the speech interval. Processing for obtaining parameters (FIR filter coefficient v _k ) of the equalized pulse source model is performed. (For example, see Patent Document 2).

  The voiced intensity calculation unit 109 performs processing for calculating the voiced intensity by passing the audio signal through a bandpass filter and calculating, for example, an autocorrelation function or a harmonic structure index for each frame period of 4 ms (for example, non-voiced intensity). (See Patent Document 4). The white noise gain calculation unit 110 performs a process of calculating the white noise gain in a portion other than the speech section.

  The flow of processing in the voice analysis unit 100 is shown in FIG. Hereinafter, with reference to FIG. 5, a flow of processing in the voice analysis unit 100 will be described.

  First, an audio signal from the microphone is measured from the data input unit 12 (step S101). For example, FIG. 8A shows an example of the waveform of an audio signal.

  Subsequently, the fundamental frequency calculation unit 101 detects a voice section from the obtained voice signal based on the power of the voice signal. For example, in the present embodiment, an analysis window length (analysis interval) of about 30 ms and an analysis shift length of about 4 ms are matched to the characteristics of the human vocal tract, and the fundamental frequency (FO or pitch) is based on the instantaneous frequency amplitude spectrum. (Cycle) is obtained (step S102).

  For calculating the fundamental frequency, for example, IEICE document (5) “Arifiant, D., Tanaka, T., Masuko, T., and Kobayashi, T.,“ Robust FO estimation of speech signal using harmonicity measure based on An instantaneous frequency, “IEICE Trans. Information and Systems, E87-D, 12, pp. 2812-2820 (2004)” can be used.

  Note that another method such as a modified autocorrelation method can be used for the extraction of the fundamental frequency. However, in the present invention, the extraction error of the fundamental frequency greatly affects the accuracy of speech analysis and synthesis. It is important to use a technique with as few extraction errors as possible.

  Next, the pulse sequence generation unit 102 generates a pulse sequence signal (pitch waveform) ex corresponding to the pitch period using the fundamental frequency obtained by the fundamental frequency calculation unit 101 within the speech section (step S103). An example of the pulse series signal ex corresponding to this pitch period is shown in FIG.

  Subsequently, the LPC coefficient calculation unit 103 performs linear prediction analysis using the speech signal, and the LPC residual calculation unit 104 obtains an LPC prediction residual waveform res using an LPC inverse filter. An example of this LPC prediction residual waveform res is shown in FIG.

  In this embodiment, as described above, an LPC analysis window length of 30 ms and an analysis shift length of 4 ms are obtained by the 28th-order autocorrelation method, and a lag window is used to avoid the influence of the fundamental frequency. . Here, the LPC coefficient is converted into a line spectrum pair (LSP) coefficient by the LSP coefficient calculation unit 105 and stored (step S104).

Subsequently, the pitch mark calculation unit 106 calculates the absolute value of the pitch waveform ex (t, k) generated in step S103 at the frame number t (frame period 4 ms) and time k (window length 30 ms) within the voice section. A cross-correlation function between the absolute values of the LPC prediction residual waveform res (t, k) obtained in step S104 for each frame t,
r (t, j) = Σ _k | res (t, k) | × | ex (t, k + j) |

Was calculated, the Σ _{t r (t,} j) is maximized such j sequences, determined using dynamic programming. Here, | * | is an absolute value. The obtained sequence j is a pitch mark candidate because the absolute value of the LPC prediction residual signal indicates a large time. Finally, a time at which | res | is maximized is searched again in the vicinity of the obtained pitch mark candidate and extracted as a pitch mark (step S105). An example of this pitch mark is shown in FIG.

  Next, the speech signal obtained in step S101, the pitch mark obtained in step S105, and the LPC prediction residual signal obtained in step S104 are used by the phase equalized speech calculation unit 107 within the speech interval. Thus, phase equalized sound is generated by equalizing the phase component of the sound signal to a constant phase (see, for example, Patent Document 2) (step S106).

  Then, the phase equalization pulse sound source filter calculation unit 108 uses the phase equalization pulse sound source model and the phase equalization pulse sound source model so that the auditory weighted error with the phase equalization sound is minimized within the speech interval. A parameter (FIR filter coefficient vk) is obtained (step S107). Here, the transfer characteristic of the FIR filter (6 taps) is expressed as follows, as in Patent Document 2.

  Here, Ti is a pitch period in the pitch mark i.

  Further, the white noise gain calculated by the white noise gain calculation unit 110 is obtained by calculating the autocorrelation function of the audio signal s.

  When

  Given in. Here, P is the order of LPC analysis, αk is the LPC coefficient, n is the frame number, and N is the window length.

  Next, the voiced intensity calculation unit 109 converts the audio signal to 0-500, 500-1000, 1000-2000, 2000-3000, 3000-4000, 4000-5000, 5000-6000, 6000-7000, 7000-8000Hz. An autocorrelation function or a harmonic structure index (for example, see the above-mentioned IEICE document (5)) is calculated for each 4 ms frame period through each bandpass filter, and this is used as a voiced intensity (for example, non (See Patent Document 4) (Step S108).

  In calculating the phase equalization pulse sound source, the white noise gain, and the voiced intensity, each pitch mark position is set as the analysis start time, and the analysis window length is calculated as two pitch periods. In this embodiment, pitch synchronization analysis is used for analysis, but since 4 ms frame shift is used for synthesis, there is a problem that the pitch mark position and the start time of the fixed-length frame are different. Therefore, in the present embodiment, the parameters in each frame are obtained by linear interpolation. It should be noted that the gain of white noise outside the speech section is obtained using a fixed window length of 15 ms and a fixed frame shift length of 4 ms (step S109).

FIG. 3 is a diagram illustrating a configuration example of the audio conversion unit 200.
As shown in FIG. 3, the voice conversion unit 200 includes an articulation speed calculation unit 201 that calculates the speed of the articulation parameter (articulation speed). Further, based on the articulation speed, an LSP coefficient conversion unit 202, a basic frequency conversion unit 203, a phase equalization pulse sound source filter conversion unit 204, a white noise gain, which convert (linear interpolation) each parameter obtained by the speech analysis unit 100. A conversion unit 205 and a voiced intensity conversion unit 206 for each frequency band are included. The linear interpolation will be described later.

  FIG. 6 is a diagram showing the flow of processing in the voice conversion unit. Hereinafter, the flow of the processing will be described with reference to FIG.

The articulation speed calculation unit 201 calculates the speed (articulation speed) of the articulation parameter measured using the two-dimensional magnetic sensor system in the voice section (steps S201 and S202).
A waveform example of this articulation speed is shown in the lowermost waveform (thin line waveform) in FIG.

When calculating the articulatory speed, the articulatory speed at time t is the RMS distance dx _{t with the} articulation parameters _{xt, i} (i = 1,..., 12: horizontal and vertical positions such as lips and tongue). ,
dx _t = sqrt (Σ _i (x _{t, i} −x _{t−1, i} ) × (x _{t, i} −x _{t−1, i} ) / 12),
Is used. Here, sqrt is a root number, and the unit of the articulation speed is mm.

  Then, the articulation speed calculation unit 201 calculates an average articulation speed avedx obtained by dividing the sum of the articulation speeds of the entire speech section by the length (number of frames) of the entire speech section (step S202).

And at all times t
_{K is obtained as} “dx _k ≦ t × avedx and dx _{k + 1} > t × avedx”, and finally the parameter at time t is linearly interpolated.
((Dx _{k + 1} −t × avedx) × p _k + (t × avedx−dx _k ) × p _{k + 1} ) / (dx _{k + 1} −dx _k ),
Is calculated.

Here p _k is at time k, LSP coefficients, the fundamental frequency, phase equalization pulse excitation filter coefficients, a voiced strength for each white noise gain or frequency bands,, LSP coefficients are calculated by the LSP coefficient converter 202 (Step S203), the fundamental frequency is calculated by the fundamental frequency converter 203 (Step S204), and the phase equalization pulse excitation filter coefficient is calculated by the phase equalization pulse excitation filter converter 204 (Step S205). The white noise gain is calculated by the white noise gain conversion unit 205 (step S206), and the voiced intensity for each frequency band is calculated by the voiced intensity conversion unit 206 for each frequency band. Calculated by the LSP coefficient converter 202 (step S207),

  As a result, it is possible to generate a voice having a speech style when a human speaks at a constant articulation speed (this is referred to as articulation speed equalized voice).

  An example of this articulation speed equalized voice is shown as the articulation speed equalized voice (second waveform from the top) in FIG.

  On the other hand, by arranging parameters for each average of the reciprocal of the articulation speed, it is possible to generate speech having an utterance style in which the initial speed of the articulation speed is fast and gradually slows, which is difficult for humans. These articulation speed conversion methods are examples, and various methods are conceivable.

FIG. 4 is a diagram illustrating a configuration example of the speech synthesis unit 300.
As shown in FIG. 4, the speech synthesis unit 300 includes a drive sound source generation unit 301 that generates a phase equalization pulse sound source and a white noise sound source, and an LPC coefficient calculation unit 302 that calculates an LPC coefficient from the LSP coefficient. ing. In addition, a convolution operation unit 303 that synthesizes the final audio signal 304 based on the phase equalization pulse sound source, the white noise, and the LPC coefficient is provided.

  A flow of processing in the speech synthesizer 300 is shown in FIG. Hereinafter, with reference to FIG. 7, the flow of the speech synthesis unit will be described.

  In the speech synthesizer 300, the parameters that are linearly interpolated based on the articulation speed in the speech converter 200 are acquired (step S301).

  Also, in this step S301, the phase equalization pulse sound source filter coefficient is obtained by applying equation (1) to the fundamental frequency linearly interpolated based on the articulation speed by the sound conversion unit 200 by the driving sound source generation unit 301. Obtain a phase equalization pulse sound source.

  Further, the white noise is multiplied by the gain of the white noise interpolated based on the articulation speed in the voice conversion unit 200. Then, based on the voiced intensity interpolated based on the articulation speed in the voice conversion unit 200, a band larger than a certain threshold is set as a voiced band, and a small band is set as a voiceless band. A drive sound source mixed with is created.

  Finally, the LPC coefficient calculated by the LSP coefficient converter 202 is converted into an LPC coefficient by the LPC coefficient calculator 302, and the LPC coefficient converted by the LPC coefficient calculator 302 is converted by the convolution calculator 303. The voice is synthesized by convolving the output signal of the driving sound source (step S302).

  The configuration and processing flow of the speech analysis / synthesis apparatus of the present invention have been described above. As specific examples, the waveforms of signals processed in the speech analysis unit 100, speech conversion unit 200, and speech synthesis unit 300 are described. Examples are shown in FIGS.

  FIG. 8 shows a partial waveform example of the utterance material “skill”. FIG. 8A shows an audio signal obtained by sampling an audio signal input from a microphone at 16 kHz, and FIG. 8B shows a pitch waveform that is a signal having a pulse sequence corresponding to the pitch period using the fundamental frequency. Show.

  FIG. 8C shows the waveform of the LPC prediction residual signal res obtained by performing linear prediction analysis (LPC) and using an LPC inverse filter. FIG. 8D shows the frame number t in the speech interval. A pitch mark generated based on the pitch waveform and the LPC prediction residual signal is shown at (frame period 4 ms) and time k (window length 30 ms). FIG. 8E shows a re-synthesis waveform synthesized based on the average articulation speed averagex calculated from the speed of the articulation parameter (articulation speed).

  As shown in FIG. 8, when the threshold value processing of the absolute value of the LPC prediction residual signal, which is a conventional method, is used, the portion surrounded by the dotted circle in the waveform of FIG. 8C is mistaken as a pitch mark. However, there are few such errors when this method is used.

FIG. 9 is a diagram illustrating an example of white noise gain of the utterance material “skill”.
FIG. 9A shows the case where the pitch synchronization analysis is not applied, and FIG. 9B shows the case where the pitch synchronization analysis is applied. As shown in FIG. 9, by performing pitch synchronization analysis, a smoothly changing gain that is not affected by the fundamental wave can be obtained.

  FIG. 10 is a diagram showing an example of a driving sound source. FIG. 10A shows a phase equalization pulse sound source, and FIG. 10B shows white noise with gain. FIG. 10C shows a signal obtained by mixing FIG. 10A and FIG. 10B based on voiced / unvoiced determination for each frequency band.

  Moreover, FIG. 11 is a figure which shows the example of the articulation speed equalization audio | voice in this invention. The utterance is an example of “a thunderhead in the blue sky”.

The signal waveforms shown in FIG. 11 are, in order from the top, original voice, articulation speed equalized voice, fundamental frequency, articulation position (lower gum, horizontal direction), articulation position (upper lip, horizontal direction), articulation position (lower lip, Horizontal direction), articulation position (tongue 1, horizontal direction), articulation position (tongue 2, horizontal direction), articulation position (tongue 3, horizontal direction),
Articulation position (lower gum, vertical direction), articulation position (upper lip, vertical direction), articulation position (lower lip, vertical direction), articulation position (tongue 1, vertical direction), articulation position (tongue 2, vertical direction), articulation The position (tongue 3, vertical direction) and articulation speed are shown. The fine lines in the fundamental frequency, the articulation position, and the articulation speed are the original sound, and the thick line is the articulation speed equalized sound.

  As shown in FIG. 11, the articulation speed in the articulation speed equalized voice is kept constant in the voice section, and the effectiveness of the method of the present invention can be confirmed.

  As a result of the listening test, it was confirmed that there is almost no perceptual distortion between the re-synthesized audio signal and the original audio signal.

  In the above description, in step S302 of FIG. 7, the LPC coefficient calculation unit 302 converts the LSP coefficient calculated by the LSP coefficient conversion unit 202 into an LPC coefficient, and the convolution calculation unit 303 converts the LPC coefficient calculation unit. Speech was synthesized by convolving the LPC coefficient converted by 302 and the output signal of the driving sound source. However, the present invention is not limited to this, and the convolution operation unit 303 generates an LSP synthesis filter from the LSP coefficient calculated by the LSP coefficient conversion unit 202, and convolves the generated LSP synthesis filter and the output signal of the driving sound source. You may synthesize speech.

  The speech analysis / synthesis apparatus of the present invention has been described above. The speech analysis / synthesis apparatus 1 shown in FIG. 1 has a computer system therein. The processing in the data input unit 12, the voice analysis unit 100, the voice conversion unit 200, the voice synthesis unit 300, and the like is realized by the CPU reading and executing the program (of course, the dedicated function). It may be realized by hardware).

  The program is stored in a computer-readable recording medium such as a hard disk or ROM. The computer reads out and executes the program, and the above process is performed.

  That is, in each process in the data input unit 12, the voice analysis unit 100, the voice conversion unit 200, the voice synthesis unit 300, etc., a central processing unit such as a CPU reads the program and executes information processing and calculation processing. By doing so, it is realized.

  Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

  Further, it is assumed that an input device, a display device, and the like (none of them are displayed) are connected to the speech analysis / synthesis device 1 shown in FIG. Here, the input device refers to an input device such as a keyboard and a mouse. The display device refers to a CRT (Cathode Ray Tube), a liquid crystal display device, or the like.

  Although the embodiments of the present invention have been described above, the speech analysis / synthesis apparatus of the present invention is not limited to the above illustrated examples, and various modifications can be made without departing from the scope of the present invention. Of course.

It is a figure which shows the structure of the speech analysis synthesis apparatus concerning embodiment of this invention. It is a figure which shows the structural example of a speech analysis part. It is a figure which shows the structural example of an audio | voice conversion part. It is a figure which shows the structural example of a speech synthesizer. It is a figure which shows the flow of a process in an audio | voice analysis part. It is a figure which shows the flow of the process in an audio | voice conversion part. It is a figure which shows the flow of a process in a speech synthesizer. It is a figure which shows the example of a part of waveform of utterance material "the skill". It is a figure which shows the example of the gain of the white noise of utterance material "the skill". It is a figure which shows the example of a drive sound source. It is a figure which shows the example of the articulation speed equalization audio | voice in this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Speech analysis synthesis apparatus, 2 ... Microphone, 3 ... Two-dimensional magnetic sensor system, 11 ... Main control part, 12 ... Data input part, 13 ... Interface part, 14 * ..Voice output unit 100 ... Voice analysis unit 101 ... Frequency frequency calculation unit 102 ... Pulse sequence generation unit 103 ... LPC coefficient calculation unit 104 ... LPC residual calculation unit , 105 ... LSP coefficient calculation unit, 106 ... Pitch mark calculation unit, 107 ... Phase equalization sound calculation unit, 108 ... Phase equalization pulse sound source filter calculation unit, 109 ... Voiced intensity calculation 110, white noise gain calculation unit, 200 ... audio conversion unit, 201 ... articulation speed calculation unit, 202 ... LSP coefficient conversion unit, 203 ... fundamental frequency conversion unit, 204 ...・ Phase equalization pulse source filter Conversion unit 205 ... white noise gain conversion unit 206 ... voiced intensity conversion unit 300 ... voice synthesis unit 301 ... drive sound source generation unit 302 ... LPC coefficient calculation unit 303 ..Convolution operation unit

Claims (8)

A data input unit that measures simultaneously measured voice as a voice signal and collects measurement data of articulatory movement, a voice analysis unit that analyzes the measurement data, and a voice that performs predetermined conversion on the analysis result of the voice analysis unit A speech analysis and synthesis apparatus comprising a conversion unit and a vocoder type speech synthesis unit that synthesizes speech based on a conversion result of the speech conversion unit,
The voice analysis unit
Detecting a voice section from the voice signal and calculating a fundamental frequency in the voice section;
Using the fundamental frequency, a pulse sequence generation unit that generates a pitch waveform having a pulse sequence according to a pitch period;
An LPC coefficient calculation unit that performs linear prediction analysis based on the speech signal and calculates an LPC (Linear Prediction Analysis) coefficient;
An LPC residual calculation unit that calculates an LPC prediction residual waveform using the speech signal and an inverse filter having the LPC coefficient as a filter coefficient;
An LSP coefficient calculation unit for calculating an LSP (Line Spectrum Pair) coefficient from the LPC coefficient;
A pitch mark calculator for extracting a pitch mark that maximizes the cross-correlation between the absolute value of the LPC prediction residual waveform and the absolute value of the pitch waveform within the detected speech section;
A phase that generates phase-equalized speech in which the phase component of the speech signal is equalized to a constant phase based on the speech signal, the pitch mark, and the LPC prediction residual waveform within the detected speech section. An equalized speech calculator;
A phase equalization pulse sound source filter calculation unit for calculating a filter coefficient of a phase equalization pulse sound source model based on the phase equalization sound within the detected voice interval;
Within the detected speech section, a voiced strength calculation unit that calculates speech strength for each frequency band by a predetermined calculation method based on the speech signal;
A white noise gain calculation unit that calculates a white noise gain by a predetermined calculation method based on the audio signal within the detected audio section;
With
The voice conversion unit
Based on the measurement data of the articulatory movement, an articulation speed calculation unit that calculates the speed of the articulation parameter as the articulation speed,
An LSP coefficient converter that performs a predetermined conversion on the LSP coefficient according to the articulation speed;
A fundamental frequency converter that performs a predetermined conversion on the fundamental frequency according to the articulation speed;
A phase equalization pulse sound source filter conversion unit that performs a predetermined conversion on the filter coefficient of the phase equalization pulse sound source model according to the articulation speed;
A white noise gain conversion unit that performs a predetermined conversion on the white noise gain according to the articulation speed;
A voiced intensity conversion unit that performs a predetermined conversion on the sound intensity for each frequency band according to the articulation speed;
With
The speech synthesizer
A phase equalization pulse sound source is generated based on the fundamental frequency converted by the basic frequency conversion unit, the filter coefficient converted by the phase equalization pulse sound source filter conversion unit, and the phase equalization pulse sound source model. At the same time, based on the voice intensity for each frequency band converted by the voiced intensity converter, the generated phase equalization pulse sound source is mixed in the voiced band and the driving sound source is mixed in the unvoiced band with white noise. A driving sound generator,
A convolution operation unit that synthesizes an audio signal from the LSP coefficient converted by the LSP coefficient conversion unit and the output signal of the driving sound source;
Equipped with a,
Analysis when the voiced intensity calculation unit calculates the voice intensity, when the white noise gain calculation unit calculates the white noise gain, and when the drive sound source generation unit generates the phase equalized pulse sound source Calculate or generate the window length as 2 pitch periods,
A speech analysis and synthesis apparatus characterized by the above. The speech synthesizer
An LPC coefficient calculator that converts the LSP coefficient converted by the LSP coefficient converter into an LPC coefficient;
The convolution unit is
A voice signal is synthesized by convolving the LPC coefficient converted in the LPC coefficient calculation unit and the output signal of the driving sound source;
The speech analysis / synthesis apparatus according to claim 1. The LSP coefficient conversion unit, the fundamental frequency conversion unit, the phase equalization pulse sound source filter conversion unit, the white noise gain conversion unit, and the voiced intensity conversion unit of the sound conversion unit, respectively,
As the articulation speed at time t, the RMS distance dxt when the articulation parameter is xt, i (i = 1,..., N: horizontal and vertical positions such as lips and tongue) is used.
“Dxt = sqrt (Σi (xt, i−xt−1, i) × (xt, i−xt−1, i) / n),
Where sqrt is the root number and the unit of articulation speed is mm "
Further, an average articulation speed avedx is calculated by dividing the sum of articulation speeds of the entire speech section by the length (number of frames) of the entire speech section.
Furthermore, at all times t
Find k such that “dxk <= t × avedx and dxk + 1> t × avedx”,
Linearly interpolating the parameter at time t by the following equation:
“((Dxk + 1−t × avedx) × pk + (t × avedx−dxk) × pk + 1) / (dxk + 1−dxk), where Pk is the LSP coefficient, fundamental frequency, phase equalization pulse sound source at time k Filter coefficient, white noise gain, or voiced intensity per frequency band ",
The speech analysis / synthesis apparatus according to claim 1 or 2, characterized in that: A data input unit that measures simultaneously measured voice as a voice signal and collects measurement data of articulatory movement, a voice analysis unit that analyzes the measurement data, and a voice that performs predetermined conversion on the analysis result of the voice analysis unit A speech analysis / synthesis method in a speech analysis / synthesis apparatus comprising: a conversion unit; and a vocoder-type speech synthesis unit that synthesizes speech based on a conversion result of the speech conversion unit,
By the voice analysis unit,
Detecting a voice section from the voice signal, and calculating a fundamental frequency in the voice section;
Using the fundamental frequency, a pulse sequence generation procedure for generating a pitch waveform having a pulse sequence according to the pitch period;
An LPC coefficient calculation procedure for performing linear prediction analysis based on the speech signal and calculating an LPC (Linear Prediction Analysis) coefficient;
An LPC residual calculation procedure for calculating an LPC prediction residual waveform by the speech signal and an inverse filter having the LPC coefficient as a filter coefficient;
An LSP coefficient calculation procedure for calculating an LSP (Line Spectrum Pair) coefficient from the LPC coefficient;
A pitch mark calculation procedure for extracting a pitch mark that maximizes a cross-correlation between an absolute value of the LPC prediction residual waveform and an absolute value of the pitch waveform within the detected speech section;
A phase that generates phase-equalized speech in which the phase component of the speech signal is equalized to a constant phase based on the speech signal, the pitch mark, and the LPC prediction residual waveform within the detected speech section. Equalized speech calculation procedure;
A phase equalization pulse sound source filter calculation procedure for calculating a filter coefficient of a phase equalization pulse sound source model based on the phase equalization sound within the detected voice interval;
Within the detected speech section, a voiced strength calculation procedure for calculating speech strength for each frequency band by a predetermined calculation method based on the speech signal;
Within the detected speech section, a white noise gain calculation procedure for calculating a white noise gain by a predetermined calculation method based on the speech signal;
Is done,
By the voice conversion unit,
Based on the measurement data of the articulatory movement, the articulation speed calculation procedure for calculating the speed of the articulation parameter as the articulation speed,
An LSP coefficient conversion procedure for performing a predetermined conversion on the LSP coefficient according to the articulation speed;
A fundamental frequency conversion procedure for performing a predetermined conversion on the fundamental frequency according to the articulation speed;
A phase equalization pulse sound source filter conversion procedure for performing a predetermined conversion on the filter coefficient of the phase equalization pulse sound source model according to the articulation speed;
A white noise gain conversion procedure for performing a predetermined conversion on the white noise gain according to the articulation speed;
A voiced intensity conversion procedure for performing a predetermined conversion on the sound intensity for each frequency band according to the articulation speed;
Done
By the speech synthesizer,
A phase equalized pulse sound source is generated based on the fundamental frequency converted in the basic frequency conversion procedure, the filter coefficient converted by the phase equalized pulse sound source filter conversion procedure, and the phase equalized pulse sound source model. At the same time, based on the voice intensity for each frequency band converted in the voiced intensity conversion procedure, the generated phase equalization pulse sound source is mixed in the voiced band and the driving sound source is mixed in the unvoiced band with white noise. Driving sound source generation procedure,
A convolution calculation procedure for synthesizing an audio signal from the LSP coefficient converted by the LSP coefficient conversion procedure and the output signal of the driving sound source;
Is done ,
When the voice intensity is calculated in the voiced intensity calculation procedure, the white noise gain is calculated in the white noise gain calculation procedure, and the phase equalization pulse sound source is generated in the driving sound source generation procedure In this case, the analysis window length is calculated or generated as two pitch periods.
A speech analysis and synthesis method characterized by the above. By the speech synthesizer,
An LPC coefficient calculation procedure for converting the LSP coefficient converted by the LSP coefficient conversion procedure into an LPC coefficient is performed.
In the convolution calculation procedure,
A voice signal is synthesized by convolving the LPC coefficient converted in the LPC coefficient calculation procedure with the output signal of the driving sound source;
The speech analysis and synthesis method according to claim 4. In the LSP coefficient conversion procedure, the fundamental frequency conversion procedure, the phase equalization pulse sound source filter conversion procedure, the white noise gain conversion procedure, and the voiced intensity conversion procedure by the sound conversion unit,
A procedure using the RMS distance dxt when the articulation parameter is xt, i (i = 1,..., N: horizontal and vertical positions such as lips and tongue) as the articulation speed at time t;
“Dxt = sqrt (Σi (xt, i−xt−1, i) × (xt, i−xt−1, i) / n),
Where sqrt is the root number and the unit of articulation speed is mm "
Further, a procedure for calculating an average articulation speed avedx obtained by dividing the sum of the articulation speeds of the entire speech section by the length (number of frames) of the entire speech section;
Furthermore, at all times t
Find k such that “dxk <= t × avedx and dxk + 1> t × avedx”,
A procedure for linearly interpolating the parameter at time t by the following equation;
“((Dxk + 1−t × avedx) × pk + (t × avedx−dxk) × pk + 1) / (dxk + 1−dxk), where Pk is the LSP coefficient, fundamental frequency, phase equalization pulse sound source at time k Filter coefficient, white noise gain, or voiced intensity per frequency band ",
6. The speech analysis / synthesis method according to claim 4 or 5, wherein: A data input unit that collects measurement data of voice and articulation movement, a voice analysis unit that analyzes the measurement data, a voice conversion unit that performs predetermined conversion on the analysis result of the voice analysis unit, and a conversion of the voice conversion unit To a computer in a speech analysis and synthesis device comprising a vocoder type speech synthesis unit that synthesizes speech based on the results,
The computer program for performing the procedure in any one of Claims 4-6.   A computer-readable recording medium storing the computer program according to claim 7.

Images