JP4762553B2 - Text-to-speech synthesis method and apparatus, text-to-speech synthesis program, and computer-readable recording medium recording the program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that the speech synthesis of high quality is heretofore unobtainable as the waveform distortion calculation in speech phoneme piece selection during synthesized speech generation is in a speech synthesis unit and therefore the sound quality of a consonant part is extremely good but if the sound quality of a vowel part is poor, the phonemes are precluded from the representative phoneme selection and the goodness of the consonant part is not reflected and the synthesized speech of average quality which is "modestly good in both of the consonants and vowels" is resulted. <P>SOLUTION: A plurality of signal waveforms are segmented from a plurality of training phonemes and a plurality of fused phonemes are generated by combining the segmented signal waveforms. At least either of the pitch and duration length of the fused phonemes is changed according to the corresponding parameters of the prescribed phonemes to generate a plurality of the synthesized phonemes. The distances between the plurality of the generated synthesized phonemes and the prescribed phonemes are evaluated and the fused phonemes are stored into a phoneme dictionary, based on the evaluation and the fused phonemes corresponding to the input phonemes obtained by analyzing the input text are selected from the phonemes dictionary and are connected and the synthesized speech is output. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to text-to-speech synthesis, and more particularly to a text-to-speech synthesis technique for improving the quality of synthesized speech generated from information such as pitch length and duration.

Synthesizing speech signals artificially from arbitrary sentences is called text-to-speech synthesis. Text-to-speech synthesis is generally performed in three stages: a language processing unit, a phoneme processing unit (prosodic setting), and a speech synthesis unit.
The input text is first subjected to morphological analysis and syntactic analysis in the language processing unit, and then subjected to accent and intonation processing in the phonological processing unit, so that the phoneme environment such as phonological symbol, pitch length, duration length, etc. Information is output. Then, based on the phoneme environment information, a speech unit registered in the speech unit dictionary is selected. Finally, the speech synthesizer synthesizes speech from information such as phonological symbols, pitch length, and duration length.
In such a speech synthesis technical field, according to at least one of the pitch and duration length of a plurality of training speech segments, at least one of the pitch and duration length of a representative speech segment that has already been generated is changed. Generate synthesized speech segments. The one that evaluates the distortion between the generated synthesized speech unit and the training unit, and outputs the synthesized speech by selecting and connecting the speech unit that minimizes the distortion (this is called the representative speech unit). is there. (For example, refer to Patent Document 1).

  Here, the speech segment is a segment that is cut out from a speech signal in units of speech synthesis such as CV, VC, and VCV, where V is a vowel and C is a consonant. Represents a parameter series extracted by any method. The phoneme environment is an environmental factor of the speech unit. For example, the phoneme name of the speech unit, the preceding phoneme, the following phoneme, the pitch period, the pitch pattern, the duration, the phoneme boundary position between C and V, Examples include power, number of mora, and accent position.

JP-A-9-319391 (pages 4-8, FIG. 1)

  The conventional speech synthesis method is configured as described above. However, since the waveform distortion calculation at the time of speech segment selection is a speech synthesis unit such as CV, VC, VCV, etc., the speech configured in CV units. Taking the case of a segment as an example, in a representative speech segment candidate, the sound quality of the C (consonant) part is very good (or resistant to deformation), but the sound quality of the V (vowel) part is bad (or deformed). Weak)), the speech unit candidate is excluded from the representative speech unit to be finally selected, and the good quality of the C part is not reflected in the representative speech unit. As a result, the representative speech unit There is a problem that only a speech unit having an average quality can be obtained as a speech segment selected as “good consonants and vowels”.

  The present invention has been made to solve the above problems, and in the unit selection process of selecting the optimum speech unit from the training speech unit or speech unit dictionary, any plurality of speech waveforms or speech waveforms It is an object of the present invention to obtain a speech synthesis method and apparatus that enables a high-quality synthesized speech by newly generating an optimal speech segment by combining the parameters that constitute the above.

  Further, the present invention provides a common part in a plurality of speech units, for example, / ma /, / ka /, /, in a unit selection process for selecting an optimal speech unit from a training speech unit or a speech unit dictionary. The vowel phoneme / a / in ba /, the consonant phoneme / n / in / na /, / ni /, / nu /, etc. A speech synthesis method that generates a high-quality synthesized speech by generating an optimal common part of speech units and degenerates the common part, while enabling a significant reduction in the storage capacity of speech units. And aim to obtain equipment.

A text-to-speech synthesis method according to the present invention includes:
Two training speech segments are prepared from a plurality of training speech segments, and the waveform separation position of one speech segment is cut out from one training speech segment at a predetermined phoneme boundary to generate a plurality of signal waveforms A separation step;
A waveform fusion step of generating a plurality of fused speech segments by combining and fusing any one or a plurality of signal waveforms from a plurality of signal waveforms cut out at the waveform separation position;
Speech that generates a plurality of synthesized speech segments in which at least one of the pitch and duration length of the generated fused speech segment is changed according to at least one of the pitch and duration length of the other training speech segment prepared Fragment synthesis step;
In the waveform separation step, the distance between each of the prepared other training speech units and each of the generated synthesized speech units is evaluated, and the distance is minimized based on the evaluation . A distortion evaluation step for holding or storing in a speech segment dictionary a fusion speech unit in which a plurality of signal waveforms obtained by finely adjusting the cut-out position before and after the phoneme boundary are combined ,
Synthesis by outputting synthesized speech by selecting and connecting fused speech units corresponding to input phonemes obtained by analyzing input text from a plurality of fused speech units held or stored in the speech unit dictionary A voice generation step.

A text-to-speech synthesizer according to the present invention provides:
A prosody setting unit that analyzes input text and obtains input phonemes;
Preparing two training speech segments from a plurality of training speech segments , cutting out waveform separation positions of one speech segment from one training speech segment at a predetermined phoneme boundary, and generating a plurality of signal waveforms; A plurality of fused speech unit candidates are generated by arbitrarily combining the plurality of signal waveforms, and at least one of the pitch and duration of the fused speech unit candidate is one of the other training speech units prepared above. Generating a plurality of synthesized speech segments modified according to at least one of one pitch and duration, and evaluating a distance between the plurality of synthesized speech segments and the other prepared training speech segment; melting the distance based on the evaluation is fused by combining a plurality of signal waveforms were finely adjusted before and after the phoneme boundary extraction position of the waveform separation step so as to minimize And fused speech unit generating means for generating a speech segment,
Fusion speech unit storage means for holding or storing the fusion speech unit;
Unit selection means for selecting a fusion speech unit corresponding to the input phoneme from the fusion speech units held or stored by the fusion speech unit storage unit;
Speech synthesis means for connecting the selected fused speech segments and generating synthesized speech.

The text-to-speech synthesis program according to the present invention is
Prosody setting means to obtain input phonemes by analyzing input text on a computer,
Preparing two training speech segments from a plurality of training speech segments , cutting out waveform separation positions of one speech segment from one training speech segment at a predetermined phoneme boundary, and generating a plurality of signal waveforms; Generating a plurality of synthesized speech segments in which at least one of the pitch and duration length of the fusion speech segment candidate is changed according to at least one pitch and duration length of the other prepared training speech segment Then, the distance between the plurality of synthesized speech segments and the other prepared training speech segment is evaluated, and the cut-out position in the waveform separation step is determined so as to minimize the distance based on the evaluation. A fused speech unit generating means for generating a fused speech unit by combining a plurality of signal waveforms finely adjusted before and after
Fused speech unit storage means for holding or storing the fused speech unit;
Unit selection means for selecting a fusion speech unit corresponding to the input phoneme from the fusion speech units held or stored by the fusion speech unit storage unit;
The selected fused speech segments are connected to function as speech synthesis means for generating synthesized speech.

Further, a computer-readable recording medium in which the text-to-speech synthesis program according to the present invention is recorded,
Prosody setting means to obtain input phonemes by analyzing input text by computer,
Preparing two training speech segments from a plurality of training speech segments , cutting out waveform separation positions of one speech segment from one training speech segment at a predetermined phoneme boundary, and generating a plurality of signal waveforms; A plurality of fused speech unit candidates are generated by arbitrarily combining the plurality of signal waveforms, and at least one of the pitch and duration of the fused speech unit candidate is one of the other training speech units prepared above. Generating a plurality of synthesized speech segments modified according to at least one of one pitch and duration, and evaluating a distance between the plurality of synthesized speech segments and the other prepared training speech segment; melting the distance based on the evaluation is fused by combining a plurality of signal waveforms were finely adjusted before and after the phoneme boundary waveform separation position of the waveform separation step so as to minimize Fused speech unit generating means for generating a speech segment,
Fused speech unit storage means for holding or storing the fused speech unit;
Unit selection means for selecting a fusion speech unit corresponding to the input phoneme from the fusion speech units held or stored by the fusion speech unit storage unit;
A text-to-speech synthesis program for connecting the selected fused speech units to function as speech synthesis means for generating synthesized speech was recorded.

  According to the present invention, it is possible to generate a high-quality synthesized speech by combining a plurality of arbitrary speech waveforms from a training speech unit and generating a new speech unit having good sound quality at a synthesized sound level. Can do.

  In addition, according to the present invention, an optimal common part of a plurality of speech segments is generated from a plurality of cut out speech waveforms by combining waveforms that minimize distortion, and the common part is degenerated, thereby achieving high quality. It is possible to greatly reduce the storage capacity of the speech segment dictionary while providing a simple synthesized speech.

Embodiments of the present invention will be described below with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of a speech synthesis apparatus that implements the speech synthesis method according to Embodiment 1 of the present invention. In FIG. 1, 1 is an input terminal, 2 is a language processing unit, 3 is a language dictionary, 4 is a prosody setting unit, 5 is a fused speech unit generation unit, 6 is a speech unit dictionary, 7 is a unit selection unit, 8 Is a speech synthesizer, and 9 is an output terminal.

  In FIG. 1, an input text 101 input from an input terminal 1 is subjected to morphological analysis and syntax analysis by cross-referencing the language dictionary 3 in the language processing unit 2 and outputs an analysis result 102 such as reading and part-of-speech information.

  Next, based on the analysis result 102 output from the language processing unit 2, the prosody setting unit 4 performs control processing of phoneme sequences, accents, and intonations, and parameters of acoustic features such as phoneme symbol strings, phonemes Prosody information 103, which is a parameter of a pitch pattern, pitch period, pitch mark, duration, or prosody, is set.

In the fused speech unit generation unit 5, the first training speech unit 104 and the second training speech unit 105 are input, and for example, a plurality of signal waveforms belonging to a phoneme from the second training speech unit 105 are obtained. A plurality of fused speech segments are generated by cutting out and fusing any one or a plurality of signal waveforms in combination. Further, the first training speech unit 104 belonging to the same phoneme as the second training speech unit is selected, and the second training speech is determined according to the information such as the pitch period and the duration length included in these speech units. A plurality of synthesized speech segments are generated by changing the pitch period and duration of the plurality of fused speech segments generated from the segment 105. Subsequently, distortion evaluation of each of the plurality of synthesized speech elements and each of the first training speech elements 104 is performed, and the fusion speech element 106 that minimizes the distortion is stored in the speech element dictionary 6.
In this embodiment, the training speech segment uses a natural speech signal uttered by a human.
The processing of the fused speech element generation unit 5 will be described in detail later.

  Next, the unit selection unit 7 refers to the prosody information 103 and the speech unit dictionary 6 that holds or stores a plurality of fused speech units 106 to represent representative speech units that are fused speech units used for speech synthesis. A piece 107 is selected.

  The speech synthesizer 8 changes the pitch period and phoneme duration for the representative speech segment 107 selected from the speech segment dictionary 6 according to the prosodic information 103, and connects the segments to synthesize speech. The signal 108 is output to the output terminal 9. Here, as a method of synthesizing speech by changing the pitch period and the phoneme duration, for example, a residual drive LSP method for synthesizing on an LSP (Line Spectral Pair) parameter, an MBE (Multi Band) for synthesizing on a spectral parameter, and the like. Excitation), a pitch waveform superposition method for superposing and synthesizing two pitch long waveforms, and a waveform editing method for connecting and synthesizing signal waveforms such as phoneme units can be used.

  Next, an embodiment of the processing of the fused speech unit generation unit 5 that characterizes the present invention will be specifically described. The flowchart of FIG. 2 shows the processing procedure in the first embodiment of the fused speech segment generation unit 5.

  In the fusion speech segment generation processing in the first embodiment, first, a first training speech segment S1 is obtained by labeling a large number of speech data for each phoneme and cutting out according to a synthesis unit such as CV, VC, VCV, and the like. [i] (i = 1, 2, 3,..., Ns1) and a second training speech segment S2 [i] (i = 1, 2, 3,..., Ns2) cut out in the same manner are prepared. . Here, Ns1 and Ns2 are the number of segments belonging to the same phoneme of each training speech segment. Further, at the time of labeling, phoneme information, pitch information, duration time, phoneme boundary information, and other information such as front and back phoneme environments as necessary are extracted and stored.

  After preparing the first and second training speech segments as described above, first, waveform separation is performed in the waveform separation step S11. In this embodiment, in order to simplify the description, the synthesis unit is CV, and the signal waveform separation position of the speech unit is set to the phoneme boundary position between C and V (hereinafter referred to as CV boundary). The following description will be given on the assumption that the signal waveform of the part (hereinafter referred to as C segment) and the signal waveform of the V part (hereinafter referred to as V segment) are separated into two parts.

  In the waveform separation step S11, C segments SC [j] (j = 1, 2, 3,..., Ns2), V segments in accordance with the CV boundary of each speech segment from the second training speech segment S2 [i]. Separated into SV [k] (k = 1, 2, 3,..., Ns2). For example, when the phoneme of the CV segment is / ma /, the C segment has a phoneme waveform of / m /, and the V segment has a phoneme waveform of / a /. Further, along with the separation of the C element and the V element, information such as pitch information, duration time, front and rear phoneme environment of the C element and the V element is stored.

  Subsequently, in the waveform fusion step S12, those belonging to the same phoneme are arbitrarily selected from the C segment SC [j] and the V segment SV [k] separated in the waveform separation step S11, and the fused speech segment SM [ jk] (j = 1, 2, 3,..., Ns2, k = 1, 2, 3,..., Ns2). Here, SM [jk] is a CV speech element obtained by connecting and merging the jth C element and the kth V element. As for the pitch of the fusion speech element SM [jk], the pitch information of the C element and the V element is inherited as the pitch of the C part and the pitch of the V part, respectively. The total value of the duration length of V and the duration length of the V segment is used as the duration length of the fused speech segment.

  When generating the fused speech unit SM [jk], an interpolation process may be performed to reduce discontinuity in the connection between the C unit and the V unit. As an example of the interpolation process, a method using linear interpolation of power and amplitude between frames, a moving average, a Lagrange interpolation polynomial, and the like can be used.

  In the speech unit synthesis step S13, the pitch and duration of the fused speech unit SM [jk] are changed so as to be equal to the pitch and duration of the first training speech unit S1 [i]. Performing synthesis, synthesized speech segment G [jk, i] (j = 1, 2, 3,..., Ns2, k = 1, 2, 3,..., Ns2, i = 1, 2, 3,. Ns1) is generated. Here, when the phoneme of the fusion speech unit is / ma /, the first training speech unit S1 [i] also uses the CV unit of the same phoneme / ma /.

  In distortion evaluation step S14, distortion evaluation of the synthesized speech segment G [jk, i] is performed. This distortion evaluation is based on the distance e [jk, i] (j = 1, 2, 3,..., Ns2, k =) between the synthesized speech unit G [jk, i] and the first training speech unit S1 [i]. 1, 2, 3,..., Ns2, i = 1, 2, 3,. The distance e [jk, i] is, for example, the square error between the signal waveform of the synthesized speech unit G [jk, i] and the signal waveform of the first training speech unit S1 [i], or the synthesized speech unit G [jk, i] and the first training speech segment S1 [i] can be converted into a power spectrum using FFT (Fast Fourier Transform) or the like, and a square error between the spectra can be used. Alternatively, it may be a distance between each element using a known parameter such as an LSP parameter or a cepstrum parameter. Alternatively, the synthesized speech unit and the first training speech unit may be subjected to, for example, band-pass filter processing, and another evaluation method suitable for each band may be used. By performing distortion evaluation using an evaluation method suitable for each band, further detailed distortion evaluation becomes possible, and the quality of synthesized speech can be improved.

  In addition, the distance e [jk, i] is weighted more heavily than the other parts in the vicinity of the CV boundary and in a part where the spectrum greatly fluctuates, for example, in the rising / falling part of the speech such as the beginning / end of the word and the phonological change transition part. You may evaluate. Since the distance e [jk, i] is evaluated by weighting the vicinity of the CV that is the waveform fusion point with a large weight, the distortion due to the waveform discontinuity due to the waveform fusion can be weighted and evaluated. The generation of the fusion speech unit can be suppressed, and the quality of the synthesized speech can be improved.

Further, when evaluating the distance e [jk, i] between the synthesized speech unit G [jk, i] and the first training speech unit S1 [i], the synthesized speech unit G [jk, i] Some perceptual weighting filter processing may be performed on the first training speech segment S1 [i]. As the auditory weighting method, for example, a known method such as a method by inverse filter processing using an LPC (Linear Predictive Coefficient) parameter or the like can be used. This auditory weighting process can save the calculation by processing the training speech segment in advance.
At this time, only the fused speech unit that minimizes the distortion is generated and output to the speech unit dictionary 6 from the training speech unit that is not subjected to auditory weighting processing.
In addition, for the distance e [jk, i], a function constituting an auditory weighting filter may be incorporated as a weighting function for distance calculation. By performing auditory weighting processing, distortion evaluation can be performed with emphasis on auditory important parts, and the quality of synthesized speech can be further improved.

  In the comprehensive evaluation step S15, after all the distortion evaluations of the synthesized speech unit G [jk, i] are performed in the distortion evaluation step S14, the fusion speech unit SM [jk] is performed according to the equations (1) and (2). The total distortion E [jk] (j = 1, 2, 3,..., Ns2, k = 1, 2, 3,..., Ns2) is obtained.

  Here, w (ij) is a weighting function derived from the relationship between the first training speech segment S1 [i] and the second training speech segment S2 [i], and F01 [i] is the first The average pitch period of the training speech segment S1 [i], and F02 [i] is the average pitch period of the second training speech segment S2 [i]. Wc is a predetermined weighting factor, and is set to an experimentally suitable value for each phoneme.

  As described above, the total distortion E of the fusion speech unit SM [jk] obtained from all the combinations of the C unit SC [j] and the V unit SV [k] obtained from the second training speech unit S2 [i]. [jk] is evaluated, and the fusion speech unit 106 composed of a combination of the C unit SC [j] and the V unit SV [k] that minimizes the total distortion is output to the speech unit dictionary 6. For all the phonemes necessary for using the steps S11 to S15 for speech synthesis, the first training speech unit 104 and the second training speech unit 105 are replaced with those of the phoneme. The speech unit dictionary 6 is constructed by sequentially performing the operations.

  In this embodiment, the waveform separation position is exactly the CV boundary for the sake of simplicity of explanation, but the waveform separation position may be moved and adjusted in consideration of articulation coupling for each phoneme.

  Further, the waveform separation position may be tracked (finely adjusted) before and after the CV boundary so that the total distortion E [jk] or the distance e [jk, i] is reduced. FIG. 3 shows a modified example of the processing of the fused speech unit generation unit 5 at this time. A feedback loop is formed from the comprehensive evaluation step S15 to the waveform separation step S11, and the total distortion or distance is minimized in the determination step S16. Until the determination is made, the processing from step S11 to step S15 is sequentially performed.

  In the present embodiment, for the sake of simplification of explanation, the synthesis unit has been described as a CV segment. However, the present invention is naturally applicable to a synthesis unit such as VC, VCV, or CVC. In addition, for example, in a speech unit including a semi-vowel / yo / such as / myo /, a fusion speech unit is created by combining each of them by dividing into / m /, / y /, and / o /. It is also possible. Further, it may be divided into four units of / -m /, / m-y /, / y-o /, / o- / in units of semiphonemes.

  In the present embodiment, the case of voiced consonant / ma / has been illustrated, but for example, the present invention may be applied to CV segments such as unvoiced consonant / sa /, or in a single vowel / a / The present invention can also be applied to a case where a / a / phoneme and another / a / phoneme are connected to generate a long sound of / a / (VV segment). Further, in the case of a single consonant, a long sound can be generated (CC segment) as in the case of a single vowel. That is, since it can deal with C and V single phonemes, it can also be applied to a speech synthesis method using single C and V as synthesis units (single phoneme units).

  Further, a unit that is further subdivided than a synthesis unit such as C, V, and CV, for example, a 2-pitch length waveform used in a 2-pitch length waveform superposition synthesis method is regarded as a unit combination unit, and this 2-pitch length waveform unit Combined to generate a fusion speech unit, or to divide the speech unit time-axis signal into 5ms frames and combine them at the parameter level such as LSP parameters analyzed in that frame unit to generate a fusion speech unit You may do it.

  In addition, when evaluating distortion, we compared the case where a synthesized speech unit was generated using a fusion speech unit for a phoneme and the case where a synthesized speech unit was generated using a conventional speech unit that was not fused. If the distortion is smaller when the conventional fused speech unit is not used, it is possible to select a normal speech unit without using the fused speech unit for the phoneme.

  By adopting the configuration of the first embodiment, for example, even when the number of training speech units cannot be sufficiently prepared, a speech unit can be obtained by generating a fused speech unit by arbitrarily combining waveforms to obtain a speech unit. The variation of the segment can be increased, and high-quality synthesized speech can be generated.

Embodiment 2. FIG.
In the fusion speech unit generation unit 5 shown in the first embodiment, when the number of second training speech units is Ns2, distortion evaluation of fusion speech units of (Ns2) × (Ns2) combinations However, as Ns2 increases, the amount of processing increases dramatically. However, by pre-selecting speech units to be used for the combination of fused speech units, the amount of processing for evaluation of fused speech units can be reduced. it can.

  FIG. 4 is a flowchart showing a processing procedure of another embodiment of the fused speech unit generation unit 5. In FIG. 4, the preliminary selection step S21 is before the processing of the waveform separation step S11, and the others are configured in the same manner as the processing of FIG.

  In the preliminary selection step S21, only a speech unit suitable as a candidate for generating a fusion speech unit is selected from the second training speech unit by a predetermined preliminary selection method, and a waveform is generated for the selected speech unit. Processing of separation and waveform fusion is performed, and the result is output to the speech unit synthesis step S13. Also, the first training speech unit may be preliminarily selected as appropriate, and only a suitable speech unit may be output to the speech unit synthesis step S13.

  As a preliminary selection method, for example, an upper speech unit (a speech unit having a small distortion at the synthesized speech level) selected by a conventional representative speech unit selection method is used, or a pitch period or duration is in a predetermined range. This can be implemented by selecting only speech units having the same front / rear phoneme environment, or speech units having similar spectra (distance on the spectrum is within a certain range, etc.).

  In addition, as another preliminary selection method, it is also possible to eliminate (screen) a speech element having poor sound quality in advance using a known method. Examples of speech units with poor sound quality include speech units with low power, signal waveform / pitch period / power disorder, speech units with incorrect voiced / unvoiced judgment and pitch extraction during labeling, or pitch period May be speech units that are inappropriate for use in speech synthesis due to the fact that is significantly deviated from the average value, or the duration is too short or too long.

  Subsequently, step S11, step S12, step S13, step S14, and step S15 are sequentially executed in the same manner as in the first embodiment on the training speech segment after the preliminary selection output from the preliminary selection step S21, and created. The merged speech unit 106 is output to the speech unit dictionary 6.

  In the second embodiment, the waveform separation position is exactly the CV boundary for simplification of description, but the waveform separation position may be moved and adjusted in consideration of articulation coupling for each phoneme.

  Also in the second embodiment, similarly to the first embodiment, the waveform separation position is set to CV for each speech unit so that the total distortion E [jk] or the distance e [jk, i] is reduced. Tracking (fine adjustment) may be performed before and after the boundary. FIG. 5 shows another modification of the processing of the fused speech unit generation unit 5 at this time. A feedback loop returning from the comprehensive evaluation step S15 to the waveform separation step S11 is formed, and the total distortion or distance is determined in the determination step S16. Until it is determined to be the minimum, the processing from step S11 to step S15 is sequentially performed.

  As described above, by pre-selecting the speech unit used for the combination of the fusion speech units, it is possible to reduce the amount of processing for the fusion speech unit evaluation, and the second training speech unit having poor sound quality can be obtained. Therefore, the quality of synthesized speech can be improved.

  In addition, the first training speech unit that serves as a reference (teacher data) for the fusion speech unit is also preliminarily selected, so that the amount of processing can be reduced at the time of distortion evaluation and the first training with poor sound quality can be achieved. Speech segments can be eliminated, and the quality of synthesized speech can be improved.

Embodiment 3 FIG.
As another form of the second embodiment, after the C piece and the V piece are separated, preliminary selection can be performed for each piece.

  FIG. 6 is a flowchart showing a processing procedure of another embodiment of the fused speech unit generation unit 5. In FIG. 6, first, waveform separation of the second training speech segment is performed in waveform separation step S11, and then preliminary selection independent of C segment and V segment is performed in preliminary selection step S21. Thereafter, step S12, step S13, step S14, and step S15 are sequentially executed in the same manner as in the first embodiment, and the created fused speech unit 106 is output to the speech unit dictionary 6.

  As a preliminary selection method, for example, a method of selecting each superior candidate by a conventional representative segment selection method and generating a fusion speech segment by each superior candidate is used in the preliminary selection step S21 of the second embodiment. The same method can be used.

  Also in the third embodiment, similarly to the second embodiment, the waveform separation position may be moved and adjusted for each phoneme in consideration of articulation coupling or the like, or the total distortion E [jk] or the distance e The waveform separation position may be tracked (finely adjusted) before and after the CV boundary so that [jk, i] becomes small.

  Performing pre-selection independently for each waveform after separation after waveform separation has the effect of reducing the processing amount for distortion evaluation of fused speech segments, and eliminates training speech segments with poor sound quality And the quality of the synthesized speech can be improved.

Embodiment 4 FIG.
As a modification of the first embodiment, by sharing the C and V segments of the CV segment with those of other phonemes, the size of the acoustic dictionary can be greatly increased while maintaining the quality of the synthesized speech. Since it can be reduced or phonemes can be made common, it is possible to stabilize the synthesized speech in terms of hearing.

  FIG. 7 is a block diagram showing a configuration of a text-to-speech synthesizer that implements a speech synthesis method according to Embodiment 4 of the present invention. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. In the present embodiment, a common speech unit generator 21 that generates and outputs a plurality of common speech unit candidates 201 and a plurality of non-common speech unit candidates 202 from the second training speech unit 105, One training speech unit 104, a plurality of common speech unit candidates 201 and a plurality of non-common speech unit candidates 202 are input, and a common speech unit 203 and a non-common speech unit 204 are output. The speech unit generation tool 22 is provided, and the point that the fusion speech unit generation unit 5 is configured by the common speech unit generation tool 21 and the fusion speech unit generation tool 22 is different from the above embodiments. It is a place.

  In the common speech segment generator 21, for example, a plurality of signal waveforms having the same phoneme name in the phoneme are cut out from the second training speech segment 105, and a plurality of common speech segment candidates 201 and a plurality of non-speech segment candidates 201 are extracted. A common speech segment candidate 202 is generated and output. Here, the common speech unit indicates a speech unit waveform of the vowel part / a / which is a common phoneme name in a group of voiced consonants / ma /, / za /, / na /, etc. The speech segment indicates a speech segment waveform of each consonant part / m /, / z /, / n /, etc.

  The fusion speech unit generator 22 includes a first training speech unit 104 and a common speech unit candidate 201 generated from the second training speech unit 105 in the common speech unit generator 21 and a non-common. A speech unit candidate 202 is input. A plurality of fused speech units are generated by combining and combining arbitrary one or a plurality of signal waveforms from the input common speech unit candidate 201 and the non-common speech unit candidate 202. Further, the first training speech unit 104 belonging to the same phoneme as the fusion speech unit is selected, and the pitches of the plurality of fusion speech units are determined according to information such as the pitch period and duration length included in these speech units. A plurality of synthesized speech segments are generated by changing the period and the duration time.

  Subsequently, distortion evaluation of each of the plurality of synthesized speech segments and each of the first training speech segments 104 is performed. The distortion evaluation is performed on all phonemes belonging to the phoneme to be shared, and the phoneme unit of the common part that minimizes the distortion is stored in the phoneme unit dictionary 6 as the common phoneme unit 203. When the common speech element 203 is used, a speech element other than the common part that minimizes distortion in each phoneme unit is stored in the speech element dictionary 6 as a non-common speech element 204.

  The segment selection unit 7 refers to the speech segment dictionary 6 that stores or stores the common speech unit 203 and the non-common speech unit 204 and the prosody information 103, according to the phoneme symbol string of the prosody information 103. The common speech unit 203 and the non-common speech unit 204 are selected to generate a corresponding fused speech unit and output as a representative speech unit 107 used for speech synthesis.

The flowchart of FIG. 8 shows a processing procedure in the fourth embodiment of the common speech unit generator 21 and the fused speech unit generator 22, that is, the fused speech unit generator 5. In the fusion speech unit generation process in the fourth embodiment, as in the process in the fusion speech unit generation unit 5 in the first embodiment, first, a large number of speech data is labeled for each phoneme, Same as the first training speech segment S1 [i] | ph (i = 1, 2, 3,..., Ns1, ph = phoneme name) of a plurality of phonemes extracted according to a synthesis unit such as CV, VC, VCV, etc. A second training speech segment S2 [i] | ph (i = 1, 2, 3,..., Ns2, ph = phoneme name) of a plurality of phonemes extracted in step S is prepared. Here, Ns1 and Ns2 are the number of speech units belonging to the same phoneme of each training speech unit. Further, at the time of labeling, phoneme information, pitch information, duration time, phoneme boundary information, and other information such as front and back phoneme environments as necessary are extracted and stored.
For the convenience of the following description, the numbers of the first and second training speech segments for other phonemes are also Ns1 and Ns2, respectively, but this definition does not narrow the scope of the present invention, and any number Can take.

  After preparing the first and second training speech segments as described above, first, in the waveform separation step S11 and the common segment extraction step S41, the second training speech segment 105 in the common speech segment generator 21 is used. Execute internal processing. In this embodiment, as in the first embodiment, the synthesis unit is CV, and the signal waveform separation position of the speech element is set to the CV boundary, so that the C element and the V element are divided into two. The following explanation will be given as an example of separation.

  In the waveform separation step S11, C segments SC [j] | ph (j = 1, 2, 3,..., Ns2,..., According to the CV boundary of each speech unit from the second training speech unit S2 [i] | ph. ph = phoneme name) and V segment SV [k] | ph (k = 1, 2, 3,..., Ns2, ph = phoneme name). Further, along with the separation of the C element and the V element, information such as pitch information, duration time, front and rear phoneme environment of the C element and the V element is stored. Also, the C segment SC [j] | ph that is the unshared speech segment candidate 202 is output.

  In the common segment extraction step S41, for example, from / a / as a common element, a phoneme in which / a / exists, that is, from speech units such as / ma /, / ba /, / na /, / sa /, etc. The waveform signal of / a / is extracted with reference to the V segment SV [k] | ph separated in the waveform separation step S11, and A [k] (k = 1, 2) which is the common speech segment candidate 201. 3, ..., NA). However, NA is the number of common speech unit candidates.

  Subsequently, in the waveform fusion step S12, the speech unit synthesis step S13, the distortion evaluation step S14, and the comprehensive evaluation step S15, internal processing of the fusion speech unit generator 22 is executed.

  In the waveform fusion step S12, the C segment SC [j] | ph, which is the non-common speech unit candidate 202 separated in the waveform separation step S11, and the common speech unit generated in the common segment extraction step S41. A [k] that is a candidate 201 is arbitrarily selected, and the fused speech element SM [jk] | ph (j = 1, 2, 3,..., Ns2, k = 1, 2, 3,..., NA, ph = Phoneme name). Here, SM [jk] | ph = ma is a CV speech obtained by connecting and merging the jth C segment (/ m /) of / ma / with the kth common segment candidate (/ a /). It represents the unit / ma /. As for the pitch of the fusion speech unit SM [jk] | ph, the pitch information of the C unit and the common speech unit candidate is inherited as the pitch of the C part and the pitch of the V part, respectively, Uses the total value of the duration length of the C segment and the duration length of the common speech segment candidate as the duration length of the fused speech segment.

  When generating the fused speech unit SM [jk] | ph, an interpolation process may be performed to reduce discontinuity in the connection between the C unit and the common speech unit candidate. As an example of the interpolation process, a method using linear interpolation of power and amplitude between frames, a moving average, a Lagrange interpolation polynomial, and the like can be used.

  In the speech unit synthesis step S13, the pitch and duration of the fusion speech unit SM [jk] | ph are set to be equal to the pitch and duration of the first training speech unit S1 [i] | ph. The synthesized speech unit G [jk, i] | ph (j = 1, 2, 3,..., Ns2, k = 1, 2, 3,..., NA, i = 1, 2, 3,..., Ns1, ph = phoneme name). Here, when the phoneme of the fusion speech unit is / ma /, speech synthesis is performed using the first training speech unit S1 [i] | ph = ma which is a CV unit of the same phoneme / ma /. And a synthesized speech segment G [jk, i] | ph = ma. Similarly, when the fusion speech unit is / ba /, the first training speech unit is also written as a synthesized speech unit G [jk, i] | ph = ba using the CV unit of / ba /. Generate synthetic speech segments for all phonemes.

  In distortion evaluation step S14, distortion evaluation of the synthesized speech segment G [jk, i] | ph is performed. This distortion evaluation is performed by evaluating a distance e [jk, i] | ph between the synthesized speech element G [jk, i] | ph and the first training speech element S1 [i] | ph. The distance e [jk, i] | ph is, for example, the square error between the signal waveform of the synthesized speech unit G [jk, i] | ph and the signal waveform of the first training speech unit S1 [i] | ph. , The synthesized speech unit G [jk, i] | ph and the first training speech unit S1 [i] | ph are converted into a power spectrum using FFT (Fast Fourier Transform) or the like, and the square between the spectra Errors can be used. Alternatively, it may be a distance between each element using a known parameter such as an LSP parameter or a cepstrum parameter. Alternatively, the synthesized speech unit and the first training speech unit may be subjected to, for example, band-pass filter processing, and another evaluation method suitable for each band may be used. By performing distortion evaluation using an evaluation method suitable for each band, further detailed distortion evaluation becomes possible, and the quality of synthesized speech can be improved.

  In addition, in the vicinity of the CV boundary and the part where the spectrum greatly fluctuates, for example, the rising / falling part of the speech such as the beginning / end of the word or the phonological change transition part, the distance e [jk, i] | You may evaluate ph. By evaluating the distance e [jk, i] | ph by weighting the vicinity of the CV that is the waveform fusion point and the like, the distortion caused by the waveform discontinuity due to waveform fusion can be weighted and evaluated. Generation of deteriorated fused speech segments can be suppressed, and the quality of synthesized speech can be improved.

  Further, when evaluating the distance e [jk, i] | ph between the synthesized speech unit G [jk, i] | ph and the first training speech unit S1 [i] | ph, the synthesized speech unit G Some perceptual weighting filter processing may be performed on [jk, i] | ph and the first training speech element S1 [i] | ph. As the auditory weighting method, for example, a known method such as a method by inverse filter processing using an LPC (Linear Predictive Coefficient) parameter or the like can be used. This auditory weighting process can save the calculation by processing the training speech segment in advance. At this time, only the fused speech unit that minimizes the distortion is generated and output to the speech unit dictionary 6 from the training speech unit that is not subjected to auditory weighting processing. In addition, a function constituting an auditory weighting filter may be incorporated as a weighting function for distance calculation with respect to the distance e [jk, i] | ph. By performing auditory weighting processing, distortion evaluation can be performed with emphasis on auditory important parts, and the quality of synthesized speech can be further improved.

  In the comprehensive evaluation step S15, after all the distortion evaluations of the synthesized speech unit G [jk, i] | ph are performed in the distortion evaluation step S14, the fusion speech unit SM [ The waveform deformation distortion of jk] | ph is evaluated, and the common speech unit 203 and the non-common speech unit 204 are output to the speech unit dictionary 6.

  First, in order to determine the common speech unit, the C unit SC [j] | ph and the common speech unit A [k] obtained from the second training speech unit S2 [i] | ph The phonetic-specific distortion Ep [jk] | ph of the fusion speech unit SM [jk] | ph by all combinations of the above is obtained from the equation (3). After obtaining the phoneme-specific distortion Ep [jk] | ph, the overall distortion EA [k] for all phonemes is obtained by Equation (4), and the common speech element candidate A [ k] is stored in the speech unit dictionary 6 as the common speech unit 203.

  Subsequently, after the common speech element A [k] is obtained, the synthesized speech element G [jk, i] | ph corresponding to the common speech element determined in each phoneme is re-evaluated, The C segment that minimizes the distortion is stored in the speech segment dictionary as the unshared speech segment 204. As described above, the speech segment dictionary 6 is constructed by sequentially performing the steps S11, S41, and S12 to S15 for all the common phonemes.

  In the fourth embodiment, the waveform separation position is exactly the CV boundary for simplification of description, but the waveform separation position may be moved and adjusted in consideration of articulation coupling for each phoneme.

  Also in the fourth embodiment, similarly to the first embodiment, the waveform separation position is set to CV for each speech unit so that the total distortion EA [k] or the distance e [jk, i] | ph becomes small. Tracking (fine adjustment) may be performed before and after the boundary.

  FIG. 9 shows another modification of the processing of the common speech unit generator 21 and the fusion speech unit generator 22 at this time, and a feedback loop is formed between the comprehensive evaluation step S15 and the waveform separation step S11. Until the total distortion or the distance is determined to be minimum in the determination step S16, the processes in steps S11, S41 and S12 to S15 are sequentially performed.

  In the fourth embodiment, an example in which the vowel / a / is shared is presented, but for example, voiced voices such as / ma /, / mi /, / mu /, / me /, / mo / The consonant consonant part / m / etc. Can also be shared. The present invention is also applicable to unvoiced consonants such as unvoiced consonants / sa /, / shi /, / su /, / se /, / so /. In addition, the C element such as / m / and the V element such as / a / are made common, and the transition part (/ ma /) of the C element and the V element, that is, only the phoneme transition part is not common. It is also possible to use a speech unit.

  In the fourth embodiment, for the sake of simplification of explanation, the synthesis unit has been described as a CV segment. However, the present invention can also be applied to synthesis units such as VC, VCV, and CVC. In addition, for example, in a speech unit including a semi-vowel / yo / such as / myo /, a fusion speech unit is created by combining each of them by dividing into / m /, / y /, and / o /. It is also possible. Further, it may be divided into four units of / -m /, / m-y /, / y-o /, / o- / in units of semiphonemes.

  Further, a unit that is further subdivided than a synthesis unit such as C, V, and CV, for example, a 2-pitch length waveform used in a 2-pitch length waveform superposition synthesis method is regarded as a unit combination unit, Combined to generate a common speech unit, or to divide the time axis signal of the speech unit into frames of 5 ms units and combine them at the parameter level such as LSP parameters analyzed in units of the frames. May be generated.

  In addition, when evaluating a distortion, a synthesized speech unit is generated using a common speech unit in a certain phoneme, and a synthesized speech is generated using a common fusion speech unit without using a shared speech unit. Compared to the case of generating a segment, if the distortion is smaller when the common speech unit is not used, the normal fusion speech unit is not used for the phoneme without using the common speech unit. It is also possible to select.

  By adopting the configuration of the fourth embodiment, for example, even when the number of training segments cannot be sufficiently prepared, a speech unit can be generated by generating a combined speech unit by arbitrarily combining waveforms to form a speech unit. The variation of the piece can be increased, and a high-quality synthesized speech can be generated.

  Further, by adopting the configuration of the fourth embodiment, the C segment and the V segment of the CV segment are shared with those of other phonemes, so that the common part is degenerated. It is possible to significantly reduce the amount of memory of the acoustic dictionary while maintaining the above, and to stabilize the synthesized speech for audibility because the phonemes can be shared.

Embodiment 5 FIG.
As another fifth embodiment of the fourth embodiment, as in the second embodiment, speech units used for the combination of the common speech unit and the fused speech unit may be preselected. Preliminary selection can reduce the amount of processing for the fusion speech unit evaluation, can eliminate training speech units having poor sound quality, and can improve the quality of synthesized speech.

Embodiment 6 FIG.
In the first embodiment, the fusion speech unit 106 is generated according to the pitch period and phoneme duration length held by the speech unit in the first training speech unit 104, but the pitch and the pitch generated by a predetermined rule are generated. The synthesized speech segment is generated by transforming the fusion speech unit according to the duration, for example, the pitch period of the input text output by the prosody setting unit 4 and the phoneme duration, and the pitch period output by the prosody setting unit 4 The speech unit having the smallest difference from the phoneme duration is extracted from the first training speech unit 104, and the pitch and duration of the extracted first training speech unit and the synthesized speech unit are extracted. It is also possible to perform distortion evaluation.

  FIG. 10 is a block diagram showing the configuration of a speech synthesizer that implements the speech synthesis method according to Embodiment 6 of the present invention. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. In the present embodiment, the prosody information 103 output from the prosody setting unit 4 is input to the fused speech unit generation unit 5, which is different from the previous embodiments.

  First, for example, “view mountain scenery” is input from the input terminal 1 as the input text 101. The language processing unit 2 analyzes the input text 101 by cross-referencing the language dictionary 3 and outputs an analysis result 102. The prosody setting unit 4 performs control processing of phoneme series, accent and intonation, and parameters of acoustic features such as phoneme symbol string, pitch pattern of speech segment, pitch period, pitch mark, duration length or prosody parameter The prosodic information 103 is set. It should be noted that the “view mountain scenery” input as the input text is decomposed into a phoneme symbol string as shown in Equation 5 when CV is used as a synthesis unit.

  The fused speech segment generation unit 5 sequentially generates fused speech segments from the second training speech segment 105 according to the prosodic information 103 corresponding to each phoneme / ya /, / ma /,. Then, each speech unit most suitable for the prosodic information 103 is selected from the first training speech unit 104, distortion evaluation with the fusion speech unit is performed, and the fusion speech unit 106 that minimizes the distortion is performed. Is stored in the speech segment dictionary 6.

  FIG. 11 is a flowchart showing a processing procedure of the fused speech unit generation unit 5 in the present embodiment. The flowchart of FIG. 11 includes step S11, step S12, step S13, step S14, and step S15 described in FIG. 2, and an evaluation element selection step S51 that is a new element.

  From FIG. 11, first, waveform separation of the second training speech unit 105 is performed in waveform separation step S11, and a fusion speech unit is generated in waveform fusion step S12. In the speech unit synthesis step S13, synthesis is performed by changing the pitch period and duration of the generated fused speech unit in accordance with the pitch period and duration included in the prosodic information 103 corresponding to each phoneme. Generate speech segments.

  Subsequently, in the evaluation unit selection step S51, the pitch period included in the prosody information 103 corresponding to each phoneme is extracted from the first training speech unit 104 in order to evaluate the distortion of the generated synthesized speech unit. A speech unit that approximates the duration is selected and extracted. In other words, a speech unit that approximates the pitch pattern and duration of the synthesized speech unit is selected.

  As a method of selecting a speech unit to be used for distortion evaluation from the first training speech unit 104, for example, using a weighted square error Ed [i] of a pitch period and a duration length shown in the following equation 6, Ed This can be implemented by selecting a speech segment whose [i] is less than or equal to a predetermined threshold.

  Here, F0rule is an M array indicating the pitch period sequence included in the prosodic information 103, and F0rule [j] indicates the j-th element. F0 [i] (i = 1, 2, 3,..., Ns1) is a pitch period sequence of the first training speech segment 104 normalized (M-dimensionalized) according to the array length of F0rule. It is an array, and F0 [i] [j] indicates the jth element of F0 [i]. Similarly, DURrule indicates the duration of the prosody information 103, and DUR [i] (i = 1, 2, 3,..., Ns1) is the duration of the first training speech segment 104. wf and wd are predetermined weighting factors, for example, wf = 0.8 and wd = 0.2.

  In the distortion evaluation step S14, distortion evaluation between the speech unit selected in the evaluation unit selection step S51 and the synthesized speech unit generated in the speech unit synthesis step S13 is performed for each phoneme.

  As described above, following the “viewing the mountain landscape”, a large amount of arbitrary input text is sequentially input, and steps S11 to S13 of the language processing unit 2, the prosody setting unit 4 and the fusion speech unit generation unit 5 are performed. The processes of S51 and S14 to S15 are sequentially executed, and the distortion evaluations obtained in step S14 are tabulated for each phoneme. In the comprehensive evaluation step S15, based on the distortion evaluation obtained in this way, the fusion speech element 106 that finally minimizes the distortion is stored in the speech element dictionary 6 for each phoneme.

  Note that the pitch period sequence and the duration length used in the evaluation segment selection step S51 are, instead of the prosodic information generated by a predetermined rule, the pitch period sequence and the duration length extracted from natural speech, That is, natural prosody can be used.

  By adopting the configuration of the sixth embodiment, it is possible to evaluate only speech segments in accordance with the prosody information 103 generated by the prosody setting unit 4, so that the quality of the synthesized speech can be further improved, Since it is not necessary to perform distortion evaluation with training speech segments that do not correspond to the prosodic information 103, there is an effect of reducing the processing amount.

Embodiment 7 FIG.
As another seventh embodiment of the sixth embodiment, as in the second embodiment, speech units used for the combination of fused speech units may be preselected. Preliminary selection can reduce the amount of processing for the fusion speech unit evaluation, can eliminate training speech units having poor sound quality, and can improve the quality of synthesized speech.

Embodiment 8 FIG.
In the first embodiment, the fusion speech unit stored in the speech unit dictionary 6 may be subjected to compression processing in order to reduce the amount of memory and the amount of communication information.

  FIG. 12 is a block diagram showing the configuration of a speech synthesizer that implements the speech synthesis method according to Embodiment 8 of the present invention. The same parts as those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted. In the present embodiment, an encoding unit 31 that holds in the speech unit dictionary 6 an encoded speech unit 301 obtained by encoding the fused speech unit 106 output by the fused speech unit generation unit 5, and a speech unit dictionary 6 is different from the previous embodiments in that a decoding unit 32 for decoding encoded speech units 301 from 6 is provided.

  The fusion speech unit 106 that is the output of the fusion speech unit generation unit 5 is input to the encoding unit 31 and is subjected to data compression or encoding processing by a predetermined compression method to obtain an encoded speech unit 301. The encoded speech unit 301 is output to the speech unit dictionary 6. The unit selection unit 7 inputs the encoded speech unit 301 held in the speech unit dictionary 6 according to the prosodic information 103 to the decoding unit 32, and the decoding unit 32 performs data expansion or decoding processing. A decoded speech unit 302 is obtained, a unit selection / connection process is performed, and speech synthesis is performed by the speech synthesizer 8 to obtain a synthesized speech 108 which is output from the output terminal 9.

Here, as a method of compressing parameters or waveform signals stored in the speech unit dictionary 6 by the fusion speech unit 106, for example, the Huffman compression, the LZ (Lempel-Ziv) method, or other known data lossless compression methods are used. Lossless compression may be used, and acoustic parameters such as the LSP parameters and spectral parameters may be quantized or encoded to be irreversibly compressed, or the waveform may be compressed using the ADPCM method, ITU-T G.729, or other known speech acoustic codes. It is also possible to perform irreversible compression using a conversion method.
In addition, after irreversible compression by quantization or encoding, it is possible to use a combination of both, such as lossless compression of the lossy compressed data to further reduce the amount of memory. In consideration of the characteristics, compression patterns such as reversible compression only, irreversible compression only, and reversible compression + irreversible compression may be used. Furthermore, the quantization / encoding accuracy (the number of bits allocated to quantization / encoding) and the encoding method may be different for each speech unit.

  When the fused speech unit 106 is compressed and stored / stored in the speech unit dictionary 6, the speech unit dictionary 6 stores the compressed speech unit data in the case of lossless compression and information used for data decompression. Is stored and is used for decoding processing such as quantization table index information and encoding code, and quantization table and codebook constituting quantization of speech unit data. Information will be stored.

  Note that the compression of the fusion speech unit 106 described in the eighth embodiment may be performed after the preliminary selection described in the second embodiment or the like is performed.

  By adopting the configuration of the eighth embodiment, it is possible to compress the fusion speech unit 106 stored in the speech unit dictionary 6, and the memory amount required for the speech unit dictionary 6 and the speech unit dictionary 6 can be reduced. The amount of communication information for downloading and the like can be reduced.

Embodiment 9 FIG.
As a modification of the eighth embodiment, compression / decompression processing for the fusion speech unit 106 is performed on the common speech unit 203 and the non-common speech unit 204 described in the fourth embodiment. Also good.

  FIG. 13 is a block diagram showing the configuration of a speech synthesis apparatus that implements the speech synthesis method according to Embodiment 9 of the present invention. The same parts as those in FIG. 7 are denoted by the same reference numerals, and the description thereof is omitted. To explain the difference, in the present embodiment, the common speech unit 203 and the non-common speech unit 204 output from the fusion speech unit generator 22 are encoded, and the encoded speech unit 301 is encoded. The encoding unit 31 stored in the speech unit dictionary 6 and the decoding unit 32 that decodes the encoded speech unit 301 from the speech unit dictionary 6 to obtain the decoded speech unit 302 are provided. This is different from the fourth embodiment shown in FIG.

The common speech unit 203 and the non-common speech unit 204 generated by the fusion speech unit generator 22 are input to the encoding unit 31 and are described in, for example, the above-described eighth embodiment. Encoding or compression processing is performed by the above method, and the encoded speech unit 301 is output to the speech unit dictionary 6.
The unit selection unit 7 decodes the encoded speech unit 301 corresponding to the common speech unit 203 and the non-common speech unit 204 held in the speech unit dictionary 6 selected according to the prosody information 103. 32, the decoding unit 32 performs data decompression or decoding processing, obtains a decoded speech segment 302, performs segment selection / connection processing, and synthesizes speech by the speech synthesis unit 8 to synthesize the synthesized speech 108. And output from the output terminal 9.

  By adopting the configuration of the ninth embodiment, it is possible to compress the common speech unit 203 and the non-common speech unit 204 which are generated by the fusion speech unit generator 22 and stored in the speech unit dictionary 6. Thus, the amount of memory required for the speech unit dictionary 6 and the amount of communication information for downloading the speech unit dictionary 6 can be reduced.

  Note that the compression of the common speech element and the non-common speech element described in the ninth embodiment may be performed after the preliminary selection described in the fifth embodiment is performed.

  In addition, the information amount compression may be performed separately for the common speech unit and the non-common speech unit by different compression methods. For example, the common speech unit is not compressed and only the non-common speech unit is compressed. It is possible to perform compression and vice versa.

  By adopting the configuration of the ninth embodiment, it becomes possible to compress the common speech unit and the non-common speech unit stored in the speech unit dictionary 6, and the amount of memory required for the speech unit dictionary 6 In addition, it is possible to reduce the amount of communication information for downloading the speech element dictionary 6.

  In the embodiment described above, the first training speech unit 104 S1 [i] and the second training speech unit 105 S2 [j] are separate data. The training speech element may be the same.

In the embodiment, all or part of the morphological analysis, syntax analysis, and prosody setting are processed in advance, and the analysis result is, for example, ROM (Read Only Memory), RAM (Random Access Memory). It is also possible to omit it by storing it in a storage means such as a nonvolatile memory or a magnetic disk and reading the analysis result from the storage means during speech synthesis.
Also, analysis results and prosodic information analyzed by processing means such as a server computer via a communication means such as LAN (Local Area Network), the Internet, infrared communication, mobile phone packet communication, or a hard disk on the server computer, etc. It can also be omitted by reading the analysis results and prosodic information stored in the storage means.

  Furthermore, analysis results and prosodic information may be directly input from input means such as a computer GUI (Graphical User Interface), keyboard, push buttons, 1D / 2D barcode reader, OCR (Optical Character Reader), etc. Absent. This is a fixed text in car navigation systems, mobile phones, PDAs (Personal Digital Assistance), video recorders, surveillance systems, game machines, e-books, toys, etc., such as navigation city names and operation guidance (guidance) sentences, security warnings This is effective when reading out synthesized speech, synthesized character of a game, newspaper sentences, and the like.

  In the above-described embodiment, all or some of the above functions can be achieved by executing a program as software such as a personal computer, or executing a program as embedded software such as a CPU or firmware. It is. Further, it may be realized by an integrated circuit such as an LSI (Large Scale IC), an FPGA (Field Programmable Gate Array), or a logic IC, or may be realized by combining discrete elements. .

  The software may be stored in advance in storage means such as ROM, magnetic disk (hard disk, removable disk, etc.), non-volatile semiconductor memory, etc. , Download from storage means on the server using wired / wireless communication means such as infrared communication, Bluetooth, packet communication of mobile phone, etc., for example, CD-ROM, CD-R, DVD (Digital Versatile Disk), MO disk Further, it may be distributed / provided from a storage medium such as a magnetic disk (hard disk, removable disk, etc.), nonvolatile semiconductor memory, magnetic tape, or a print medium such as a card printed with a barcode. In this case, the program code of the software read from the storage medium or the like realizes the function of the embodiment, and these storage medium and the like constitute the present invention.

  In the above embodiment, the case where each unit is configured on the same computer has been described. However, the present invention is not limited to this, and for example, each unit is divided into computers and processing devices distributed on a network. May be configured.

  In addition, the present invention may be applied to a system composed of one or more devices. The server computer distributes a program or the like for realizing the embodiment of the present invention using a communication means such as a network, and executes a program distributed by a plurality of client computers, mobile terminal devices such as mobile phones and PDAs. Can do.

  The training speech unit used in the above embodiment uses a natural speech signal uttered by a human, but the training speech unit is not only a natural speech but also a speech waveform generated analytically from natural speech, for example, An average waveform, a sub-optimal waveform, a power-corrected speech waveform, etc. selected under a predetermined standard (for example, a mutual distance on the spectrum is equal to or smaller than a predetermined threshold) may be used, or artificially A signal waveform obtained by mixing both the generated waveform and natural speech can also be applied. Further, a pseudo sound signal waveform extracted from a person other than a person, such as an animal call, a musical instrument, or an electronic sound, may be used. Further, a noise waveform may be mixed into the artificially generated speech waveform or the like.

  According to the present invention, since high-quality synthesized speech can be generated, a speech synthesis function for car navigation systems, a mobile phone mail and a voice reading function for information appliances, a municipal disaster prevention radio, a speech synthesis system in a highway radio, an elevator, an escalator, etc. It can be applied to automatic voice guidance.

1 is a block configuration diagram of a speech synthesizer according to Embodiment 1. FIG. 3 is a flowchart of a fused speech unit generation unit in the first embodiment. 10 is a flowchart of a modified example of the fused speech unit generation unit in the first embodiment. 10 is a flowchart of a fusion speech unit generation unit in the second embodiment. 10 is a flowchart of a modified example of the fused speech segment generation unit in the second embodiment. 10 is a flowchart of a fused speech segment generation unit in the third embodiment. FIG. 10 is a block configuration diagram of a speech synthesizer according to a fourth embodiment. 10 is a flowchart of a fused speech unit generation unit in the fourth embodiment. 10 is a flowchart of a modified example of the fused speech element generation unit in the fourth embodiment. FIG. 10 is a block configuration diagram of a speech synthesizer according to a sixth embodiment. 18 is a flowchart of the fusion speech unit generation unit 5 in the sixth embodiment. FIG. 10 is a block configuration diagram of a speech synthesizer according to an eighth embodiment. FIG. 10 is a block configuration diagram of a speech synthesizer according to a ninth embodiment.

Explanation of symbols

  1 input terminal, 2 language analysis unit, 3 language dictionary, 4 prosody setting unit, 5 fused speech unit generation unit, 6 speech unit dictionary, 7 unit selection unit, 8 speech synthesis unit, 9 output terminal, 21 common speech Segment generator, 22 Fusion speech segment generator, 31 Coding unit, 32 Decoding unit, 101 Input text, 102 Analysis result, 103 Prosodic information, 104 First training speech unit, 105 Second training speech Unit, 106 fusion speech unit, 107 representative speech unit, 108 synthesized speech, 201 common speech unit candidate, 202 non-common speech unit candidate, 203 common speech unit, 204 non-common speech unit , 301 encoded speech unit, 302 decoded speech unit.

Claims (12)

Two training speech segments are prepared from a plurality of training speech segments, and the waveform separation position of one speech segment is cut out from one training speech segment at a predetermined phoneme boundary to generate a plurality of signal waveforms A separation step;
A waveform fusion step for generating a plurality of fused speech segments by combining and fusing any one or a plurality of signal waveforms from a plurality of signal waveforms cut out at the waveform separation position;
Speech that generates a plurality of synthesized speech segments in which at least one of the pitch and duration length of the generated fused speech segment is changed according to at least one of the pitch and duration length of the other training speech segment prepared Fragment synthesis step;
In the waveform separation step, the distance between each of the prepared other training speech units and each of the generated synthesized speech units is evaluated, and the distance is minimized based on the evaluation . A distortion evaluation step for holding or storing in a speech segment dictionary a fusion speech unit in which a plurality of signal waveforms obtained by finely adjusting the cut-out position before and after the phoneme boundary are combined ,
Synthesis by outputting synthesized speech by selecting and connecting fused speech units corresponding to input phonemes obtained by analyzing input text from a plurality of fused speech units held or stored in the speech unit dictionary A text-to-speech synthesis method comprising: a speech generation step. The waveform fusion step generates a plurality of signal waveforms that become common parts with other speech units and a plurality of signal waveforms that become non-common parts with other speech units from the plurality of extracted signal waveforms. Then, by combining and fusing any one or a plurality of signal waveforms from the plurality of signal waveforms serving as the common part and the plurality of signal waveforms serving as the non-common part, common with other speech units Generate multiple fusion speech segments containing parts,
The distortion evaluation step evaluates a distance between each of the prepared plurality of synthesized speech segments with respect to each of the other prepared training speech segments, and based on the evaluation, a common speech that minimizes the distance Hold or remember the segment and other speech segments,
The synthesized speech generating step generates a synthesized speech unit corresponding to an input phoneme from the plurality of shared speech units and the other plurality of speech units held or stored, and connects them to generate a synthesized speech The text-to-speech synthesis method according to claim 1, wherein:   3. The pitch and duration length used when generating a synthesized speech segment in the speech segment synthesis step is a pitch and duration length generated according to a predetermined rule. Text-to-speech synthesis method.   The waveform separation step is a process of selecting a speech unit for two prepared training speech units based on a predetermined criterion and cutting out a plurality of signal waveforms from the selected speech unit. The text-to-speech synthesis method according to any one of claims 1 to 3. The amount of information of the fusion speech unit stored or held in the speech unit dictionary is compressed by a predetermined compression method,
5. The synthesized speech is generated by expanding the information amount of the fused speech unit compressed from the acoustic unit dictionary when selecting the fused speech unit corresponding to the input phoneme. The text-to-speech synthesis method according to any one of the above. A prosody setting unit that analyzes input text and obtains input phonemes;
Preparing two training speech segments from a plurality of training speech segments , cutting out waveform separation positions of one speech segment from one training speech segment at a predetermined phoneme boundary, and generating a plurality of signal waveforms; A plurality of fused speech unit candidates are generated by arbitrarily combining the plurality of signal waveforms, and at least one of the pitch and duration of the fused speech unit candidate is one of the other training speech units prepared above. Generating a plurality of synthesized speech segments modified according to at least one of one pitch and duration, and evaluating a distance between the plurality of synthesized speech segments and the other prepared training speech segment; melting the distance based on the evaluation is fused by combining a plurality of signal waveforms were finely adjusted before and after the phoneme boundary extraction position of the waveform separation step so as to minimize And fused speech unit generating means for generating a speech segment,
Fusion speech unit storage means for holding or storing the fusion speech unit;
Unit selection means for selecting a fusion speech unit corresponding to the input phoneme from the fusion speech units held or stored by the fusion speech unit storage unit;
A text-to-speech synthesizer comprising speech synthesis means for connecting the selected fused speech units and generating synthesized speech. The fused speech segment generation means includes:
Intersection of the waveform separation position of one speech units from one training speech units by cutting out a plurality of signal waveforms in a predetermined phoneme boundary, from the cut-out a plurality of signal waveforms, and other speech unit A plurality of signal waveforms, and a plurality of signal waveforms that are non-common parts with other speech units, and a common speech unit generator that outputs to the fused speech unit generation means,
A plurality of fused voices including a common part with another speech unit by combining and combining arbitrary signal waveforms from the plurality of signal waveforms serving as the common part and the plurality of signal waveforms serving as the non-common part. A plurality of synthesized speech elements generated by generating a segment and changing at least one of the pitch and duration of the generated fused speech segment according to at least one of the pitch and duration of the other training speech segment prepared Generate a segment, evaluate the distance between the other prepared training speech segment and the corresponding plurality of generated synthesized speech segments, and separate the waveforms so that the distance is minimized based on the evaluation the extraction position in step is composed of a fused speech unit generation device for outputting fused speech unit fused by combining a plurality of signal waveforms were finely adjusted before and after the phoneme boundary Text to speech synthesis apparatus according to claim 6, wherein a. The fused speech segment generation means includes:
The pitch and duration used when generating a plurality of synthesized speech segments by changing at least one of the pitch and duration length of the fused speech segment is generated in advance according to a predetermined rule. A text-to-speech synthesizer according to claim 6 or 7. The fused speech segment generation means includes:
For the two training speech units prepared, a speech unit is selected based on a predetermined criterion, and a plurality of fused speech units are generated and evaluated using the selected speech unit. The text-to-speech synthesizer according to any one of claims 6 to 8, wherein An encoding unit that compresses the information amount of the fusion speech unit stored or held in the speech unit dictionary by a predetermined compression method;
The text-to-speech synthesizer according to any one of claims 6 to 9, further comprising a decoding unit that expands the amount of information of the compressed fused speech unit in the speech unit dictionary. Prosody setting means to obtain input phonemes by analyzing input text on a computer,
Preparing two training speech segments from a plurality of training speech segments , cutting out waveform separation positions of one speech segment from one training speech segment at a predetermined phoneme boundary, and generating a plurality of signal waveforms; Generating a plurality of synthesized speech segments in which at least one of the pitch and duration length of the fusion speech segment candidate is changed according to at least one pitch and duration length of the other prepared training speech segment Then, the distance between the plurality of synthesized speech segments and the other prepared training speech segment is evaluated, and the cut-out position in the waveform separation step is determined so as to minimize the distance based on the evaluation. A fused speech unit generating means for generating a fused speech unit by combining a plurality of signal waveforms finely adjusted before and after
Fused speech unit storage means for holding or storing the fused speech unit;
Unit selection means for selecting a fusion speech unit corresponding to the input phoneme from the fusion speech units held or stored by the fusion speech unit storage unit;
A text-to-speech synthesis program for connecting selected fusion speech units to function as speech synthesis means for generating synthesized speech. Prosody setting means to obtain input phonemes by analyzing input text by computer,
Preparing two training speech segments from a plurality of training speech segments , cutting out waveform separation positions of one speech segment from one training speech segment at a predetermined phoneme boundary, and generating a plurality of signal waveforms; A plurality of fused speech unit candidates are generated by arbitrarily combining the plurality of signal waveforms, and at least one of the pitch and duration of the fused speech unit candidate is one of the other training speech units prepared above. Generating a plurality of synthesized speech segments modified according to at least one of one pitch and duration, and evaluating a distance between the plurality of synthesized speech segments and the other prepared training speech segment; melting the distance based on the evaluation is fused by combining a plurality of signal waveforms were finely adjusted before and after the phoneme boundary waveform separation position of the waveform separation step so as to minimize Fused speech unit generating means for generating a speech segment,
Fused speech unit storage means for holding or storing the fused speech unit;
Unit selection means for selecting a fusion speech unit corresponding to the input phoneme from the fusion speech units held or stored by the fusion speech unit storage unit;
A computer-readable recording medium recording a text-to-speech synthesis program for connecting selected fusion speech units to function as speech synthesis means for generating synthesized speech.

Images