JP4424023B2 - Segment-connected speech synthesizer - Google Patents

Description

  The present invention relates to a speech synthesizer, and more particularly to a unit connection type speech synthesizer that performs speech synthesis that matches a synthesizer command by selecting and connecting speech units based on a predetermined cost function.

  Speech recognition and speech synthesis are important technologies for realizing interfaces between humans and various systems using computers. By using these and artificial intelligence technology together, the user can use various services without being aware that the other party is a computer system.

  In particular, speech synthesis is very important as an interface for system output to humans. Humans are sensitive to the unnaturalness of synthesized speech. If the user feels that the synthesized speech is unnatural, it will affect the utterance, and as a result, the dialogue between the human and the system may not be successful.

  As a recent speech synthesis technology, a speech unit for each phoneme is made into a database from an utterance corpus in which many human utterances are collected in advance, and at the time of synthesis, the most appropriate speech unit corresponding to a specified phoneme There is known what selects and connects what seems to be. This is referred to as segment-connected speech synthesis in this specification.

  In unit-connected speech synthesis, there is a problem of how to extract an appropriate speech unit from a database based on a given synthesis target.

  The data constituting the synthesis target typically includes a phoneme, and a speech frequency such as a fundamental frequency (F0), a duration, an MFCC (Mel-Frequency Cepstrum Coefficient), and power. These are hereinafter referred to as “synthesizer commands”.

  In unit-connected speech synthesis, an evaluation function called “cost” is defined to express the difference between the synthesizer command and the F0, duration, MFCC, power, etc. of speech units, and the natural degradation associated with the connection. Then, an optimum speech unit sequence is determined by obtaining a speech unit that minimizes the cost.

  The applicant of the present application decomposes the above-mentioned “cost” into “sub-costs” corresponding to certain voice features, and combines them (for example, linear sum) to define a unit-connected speech synthesis. Has proposed. For example, see Patent Document 1.

  Sub-costs can be broadly classified into two groups: target costs and connection costs. The target cost represents an error between the synthesizer command and the segment candidate. The connection cost represents a discontinuity between adjacent segments in the synthesized speech.

JP 2003-208188 A (paragraphs 0014 to 0047)

  In the unit-connected speech synthesis technology as described above, the larger the speech unit database, the higher the possibility of finding a candidate that can reduce the cost during synthesis, and the quality of speech synthesis increases. However, when the speech unit database is large, there is a problem that the amount of calculation necessary for determining candidates increases.

  As one measure for reducing the amount of calculation, it is conceivable to pre-select a unit candidate by performing a cost calculation that requires a smaller amount of calculation prior to selecting a unit by cost calculation. For example, the calculation of the connection cost requires not only a segment candidate but also a relationship with the phonemes before and after it, which increases the amount of calculation. On the other hand, the target cost calculation only requires a segment candidate. Therefore, it is conceivable to preselect a segment candidate using only the target cost without using the connection cost.

  However, even in that case, the problem remains that the amount of calculation for preliminary selection increases as the speech unit database increases. Regardless of the size of the speech segment database, it is desirable to be able to perform segment selection at a high speed and at a constant speed. Even in such a case, it should be avoided that the quality deteriorates.

  Therefore, an object of the present invention is to provide a speech synthesizer capable of performing segment selection at a high speed and at a constant speed even when a large-scale speech corpus is used.

  Another object of the present invention is to perform speech synthesis capable of performing segment selection at a high speed and a constant speed and improving the quality of synthesized speech even when a large-scale speech corpus is used. Is to provide a device.

  The unit-connected speech synthesizer according to the present invention includes a speech unit database and a table that stores speech units in the speech unit database sorted for each phoneme and using a predetermined feature value as a key. Is a unit-connected speech synthesizer that connects speech units in a speech unit database, and includes a sequence of phoneme labels that are targets of synthesized speech, and a target feature amount associated with each phoneme label, and A table selection means for selecting a table corresponding to the phoneme specified by the phoneme label of the synthesizer command, and a target of the synthesizer command among the tables selected by the table selection means Preliminary selection means for preliminarily selecting a speech element located in a range defined by a predetermined criterion, including a speech element having a predetermined feature value specified by the feature value A selection unit for selecting a speech unit based on a predetermined criterion from speech units preliminarily selected by the preliminary selection unit, and a speech unit selected by the selection unit is connected in accordance with a synthesizer command. Connecting means for outputting a synthesized speech waveform.

  Preferably, the predetermined feature amount is a phoneme length, and the preliminary selection unit includes a range specified by a predetermined criterion including the phoneme length specified by the synthesizer command in the table selected by the table selection unit. Means for selecting a speech unit located at.

  Preferably, the table is provided with permissible number information for specifying the permissible number of speech segments to be preliminarily selected, and the means for selecting is the table selected by the table selecting means, Means for selecting a number of speech units specified by an allowable number centered on a speech unit having a phoneme length specified by the synthesizer command;

  The predetermined feature amount may be a quantized fundamental frequency (F0), and the preliminary selection means includes a fundamental frequency specified by a synthesizer command in a table selected by the table selection means. Means may be included for selecting speech segments located in a range specified by the defined criteria.

  The table is preliminarily provided with permissible number information for specifying the permissible number of speech segments to be preselected, and the means for selecting is a synthesizer command in the table selected by the table selecting means. There may be included means for selecting a number of speech units specified by an allowable number centered on a speech unit having a fundamental frequency specified by.

  Preferably, the preliminary selection means searches for a speech segment having a predetermined feature value specified by the target feature value of the synthesizer command in a table selected by the table selection means by a predetermined search algorithm. And a means for selecting a speech unit located in a predetermined range centered on the speech unit searched for by the search means in the table selected by the table selection means. including.

  Preferably, the search means searches for a speech element having a predetermined feature value specified by the target feature value of the synthesizer command in the table selected by the table selection means, using a binary tree search algorithm. Binary tree search means for doing this.

  Preferably, the selection unit has a predetermined cost calculated from a speech unit candidate preliminarily selected by the preliminary selection unit based on a feature amount of the speech unit and a target feature amount. Means for selecting a speech segment that satisfies the above condition.

[First Embodiment]
−Configuration−
FIG. 1 shows a block diagram of a speech synthesis system 20 according to the first embodiment of the present invention. Referring to FIG. 1, this speech synthesis system 20 classifies speech unit database (DB) 30 similar to the prior art and speech units included in speech unit DB 30 for each phoneme. A phoneme length table creation unit 32 for generating a phoneme length table 34 for each phoneme extracted by extracting information necessary for selecting a segment such as phoneme length, and a synthesizer command obtained as a result of analyzing a text as a synthesis target 36 as an input, the phoneme-specific phoneme length table 34 is used to preselect a substantially constant amount of speech units from the speech unit DB 30, and appropriate speech units are connected from the preselected segment candidates. And a speech synthesizer 38 for outputting a synthesized speech waveform 40.

  FIG. 2 shows the configuration of the phoneme length table creation unit 32 in the form of a block diagram. Referring to FIG. 2, the phoneme length table creation unit 32 extracts the phoneme label, address, phoneme length, and other information necessary for cost calculation of each unit in the speech unit DB 30 from the speech unit DB 30. Then, the phoneme length extraction unit 80 for creating the phoneme length table 82 for each phoneme and the phoneme length table 82 for each phoneme are sorted in ascending order of the phoneme length for each table, so that the sorted phoneme length table 34 sorted. For each phoneme length table 34 for each of the sort processing unit 84 and the phoneme-specific phoneme length table 34, the allowable segment candidate for calculating the allowable range of the phoneme length when the segment candidate is preliminarily selected A number calculator 86. The function of the allowable segment candidate number calculation unit 86 will be described later with reference to FIG.

  Referring to FIG. 1 again, the speech synthesizer 38 receives the synthesizer command 36, is a segment corresponding to the phoneme specified by the synthesizer command 36, and has the phoneme length specified by the synthesizer command 36. A phoneme having a phoneme length of a predetermined width centered on the phoneme is preliminarily selected from the phoneme-specific phoneme length table 34, and receives a synthesizer command 36 and a synthesizer command 36 to receive a segment candidate. A unit selection unit 64 for selecting the unit with the lowest cost from the unit candidates included in the table 62, and speech unit data corresponding to the speech unit selected by the unit selection unit 64 A connection unit 66 for reading out from the speech unit DB 30 and connecting to each other to output the synthesized speech waveform 40 is included.

  Referring to FIG. 3, preliminary selection unit 60 receives synthesizer command 36 and selects phoneme length table 110 corresponding to the phoneme specified by synthesizer command 36 from phoneme-specific phoneme length table 34. A table selection unit 100 is included. The phoneme length table 110 is attached with the allowable segment candidate count 112 calculated by the allowable segment candidate count calculator 86.

  The preliminary selection unit 60 further includes a binary tree search unit 102 for searching for a phoneme length candidate corresponding to the phoneme length given by the synthesizer command 36 by a binary tree search for the phoneme length table 110; In the phoneme length table 110, select a segment candidate with the number of segments specified by the allowable segment candidate number 112 centered on the segment candidate searched by the binary tree search unit 102, and select a segment candidate And a segment candidate selection unit 104 for creating the table 62.

  The number of allowable element candidates when the element candidate selection unit 104 selects an element candidate need not be calculated using a particularly strict criterion, and the number of element candidates can be reduced to a certain number. Anything is acceptable. If it is considered that the segment length distribution follows a normal distribution, the standard deviation σ is calculated, and the number of segments that fall within aσ (a is a predetermined number) may be used.

  In practice, the segment length distribution is often multimodal. In such a case, the allowable segment candidate number calculation unit 86 (see FIG. 2) according to the present embodiment calculates this width based on the criteria shown in FIG.

Referring to FIG. 4, for example, when the distribution is bimodal, a line parallel to the x axis is drawn in contact with the distribution curve at the apex A of the valley. The points where this line intersects the distribution curve are denoted by B and C, respectively. BA = w ₁ and AC = w ₂ . In the present embodiment, w obtained by w = α (w ₁ + w ₂ ) / 2 (α is a predetermined number, preferably 0 <α ≦ 1) is set as an allowable phoneme length width. Even when the distribution curve is multimodal, the same idea may be extended. Of course, the allowable phoneme length width can be determined by various methods other than this.

In the cost calculation by the segment selection unit 64 according to the present embodiment, the cost C ₀ is calculated from the sub cost as follows.

ただし、Ｃ_i1（ｉ1＝１〜Ｎ₁）はターゲットサブコスト、Ｃ_i2（ｉ2＝１〜Ｎ_２）は接続コスト、ｗ_i1（ｉ1＝１〜Ｎ₁）はターゲットサブコスト間に定義された重み、ｗ_i2（ｉ2＝１〜Ｎ_２）は接続サブコスト間に定義された重み、ｐ_１及びｐ_２はそれぞれ、ターゲットコストと接続コスト間に定義された重みである。

Where C _i1 (i1 = 1 to N ₁ ) is defined between the target subcosts, C _i2 (i2 = 1 to N ₂ ) is defined as the connection cost, and w _i1 (i1 = 1 to N ₁ ) is defined between the target subcosts. A weight, w _i2 (i2 = 1 to N ₂ ) is a weight defined between connection sub-costs, and p ₁ and p ₂ are weights defined between a target cost and a connection cost, respectively.

-Operation-
The speech synthesis system 20 operates as follows. The operation of the speech synthesis system 20 is roughly divided into two phases. The first phase is the construction of the phoneme-specific phoneme length table 34. The second phase is speech synthesis by the speech synthesizer 38.

  In the first phase, the following processing is performed. Prior to this processing, it is assumed that the speech segment DB 30 has been created from the speech corpus. The phoneme length extraction unit 80 (see FIG. 2) of the phoneme length table creation unit 32 obtains the phoneme label, address, phoneme length, and other costs from each of the speech unit data included in the speech unit DB 30 as described above. Information necessary for the calculation is extracted and a phoneme-specific phoneme length table 82 is created. The segment data included in each table of the phoneme-specific phoneme length table 82 is not sorted.

  The sort processing unit 84 sorts each phoneme-specific phoneme length table 82 in ascending order by phoneme length. As a result, the phoneme length table 34 sorted by phoneme is created. The allowable element candidate number calculation unit 86 calculates the allowable element candidate number 112 (see FIG. 3) based on the distribution of the phoneme lengths of the speech elements included in each of the phoneme length table 34. Attached to each phoneme length table 34.

  When the above processing is completed, speech synthesis by the speech synthesizer 38 becomes possible. Prior to speech synthesis, the speech synthesizer 38 (configured by a computer) stores the phoneme-specific phoneme length table 34 as an array on the memory so that preliminary selection can be performed at high speed.

  At the time of speech synthesis, if a synthesizer command 36 is obtained by analyzing the text to be synthesized, this synthesizer command 36 is given to the table selection unit 100 of the preliminary selection unit 60 (see FIG. 3). Based on the synthesizer command 36, the table selection unit 100 selects the phoneme length table 110 corresponding to the phoneme specified by the synthesizer command 36 from the phoneme-specific phoneme length table 34.

  The binary tree search unit 102 searches the phoneme length table 110 for a phoneme unit having a phoneme length that matches the phoneme length given by the synthesizer command 36, and determines the phoneme classification of the searched phoneme unit. An address (array index) in the phoneme length table 34 is given to the segment candidate selection unit 104. The segment candidate selection unit 104 calculates the index of speech units in a range (predetermined number before and after the center) specified by the allowable segment candidate number 112 with the speech unit indicated by the given index as the center. All the segment data are read out and stored in the segment candidate table 62.

  Referring to FIG. 1, unit selection unit 64 receives synthesizer command 36 and selects the speech unit included in unit candidate table 62 that has the lowest cost calculated by equation (1). To the connection portion 66. The connection unit 66 reads out the speech waveform data corresponding to the speech unit given from the unit selection unit 64 from the speech unit DB 30, transforms the speech so that the speech is smooth, connects it, and outputs it as a synthesized speech waveform 40. .

  The number of segment candidates determined by the allowable segment candidate number 112 is selected in advance by the preliminary selection unit 60 and stored in the segment candidate table 62, and a segment selection unit 64 selects a speech segment from the segment candidates. The amount of cost calculation for selecting a segment is small and less than a certain amount. It is known that the binary tree search by the binary tree search unit 102 can be performed at high speed, and the calculation amount of the address calculation for the segment candidate extraction by the segment candidate selection unit 104 is small. For this reason, the calculation amount for selecting a segment by the speech synthesizer 38 is small as a whole.

  The element candidate table 62 is made up of a predetermined number of element candidates centered on the element candidate of the specified phoneme length from the phoneme-specific phoneme length table 34 corresponding to the phoneme specified by the synthesizer command 36. Therefore, it contains many speech segments with a small target cost. Therefore, when the speech unit having the lowest cost is selected and connected from among them, the quality deterioration due to the deformation at the time of connection becomes so small that it can be ignored. As a result, the synthesized speech waveform 40 finally obtained shows only a slight deterioration in quality due to the connection of speech units.

  In this embodiment, the preliminary selection unit 60 preliminarily selects a predetermined number of segments from the phoneme-specific phoneme length table 34 based on the phoneme length. However, the present invention is not limited to such an embodiment. For example, the phoneme length table may be sorted on the basis of feature quantities other than phoneme length, such as the fundamental frequency, and used for preliminary selection.

  In the system according to the embodiment of the present invention described above, the number of segment candidates is limited using the allowable number of segment candidates at the time of preliminary selection. However, the present invention is not limited to such an embodiment. For example, if the phoneme length is used as a reference, a segment candidate having a predetermined phoneme length width around the searched segment may be extracted. Alternatively, the second preliminary selection using only the target cost may be performed on the segment candidates extracted as described above.

  In the above-described embodiment, the phoneme-specific phoneme length table 34 is created by sorting the phoneme lengths in ascending order. However, it is obvious that the same effect can be obtained even if the sort is performed in descending order. Furthermore, in the system of the above-described embodiment, the phoneme-specific phoneme length table 34 is used for phoneme selection by storing not only the phoneme length in the phoneme-specific phoneme length table 34 but also other feature quantities. Only the phoneme length, the phoneme label, and the address of the unit in the speech unit DB 30 may be stored in the length table 34, and the phoneme-specific phoneme length table 34 may be used only for preliminary selection of the unit.

  Furthermore, in the system of the above-described embodiment, the number of segment candidates to be preliminarily selected is calculated based on the phoneme length distribution in the phoneme-specific phoneme length table. However, the present invention is not limited to such an embodiment. For example, a fixed number for each phoneme may be determined in advance and used.

  Furthermore, in the above-described embodiment, the segment candidates are sorted based on the phoneme length, and the number of candidates is limited. However, the criteria used for restriction are not limited to phoneme length. For example, a quantized fundamental frequency (F0) may be used. In this case, a segment candidate that matches the fundamental frequency specified by the synthesizer command 36 is searched by a binary tree search, includes the searched segment candidate, and is allowed centering on F0 of the segment candidate. What is necessary is just to extract the segment candidate of the fundamental frequency of a range, or a predetermined number of segment candidates.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is a block diagram of a speech synthesis system 20 according to an embodiment of the present invention. It is a block diagram of the phoneme length table preparation part 32 shown in FIG. FIG. 2 is a block diagram of a preliminary selection unit 60 shown in FIG. 1. It is a figure for demonstrating the calculation method of the number of permissible segment candidates.

Explanation of symbols

  20 speech synthesis system, 30 speech segment DB, 32 phoneme length table creation unit, 34 phoneme length table, 36 synthesizer command, 38 speech synthesizer, 60 preliminary selection unit, 62 segment candidate table, 80 phoneme length extraction , 82 Phoneme-specific phoneme length table (unsorted), 84 Sort processing unit, 86 Allowable segment candidate number calculation unit, 110 Phoneme length table, 112 Allowable segment candidate number

Claims (8)

Using a speech unit database and a table that stores speech units in the speech unit database sorted by phoneme and using predetermined feature values as keys, A unit connection type speech synthesizer for connecting speech units,
The table corresponding to the phoneme specified by the phoneme label of the synthesizer command is received by receiving an input of a synthesizer command defining a sequence of phoneme labels to be a target of synthesized speech and a target feature amount associated with each phoneme label. A table selection means to select;
Within the table selected by the table selection means, the position is within a range determined by a predetermined standard including the speech segment having the predetermined feature value specified by the target feature value of the synthesizer command Pre-selection means for pre-selecting speech segments to be performed;
A selection unit for selecting a speech unit based on a predetermined criterion from the speech units preliminarily selected by the preliminary selection unit;
A unit connection type speech synthesizer comprising: a connection unit configured to connect the speech units selected by the selection unit according to the synthesizer command and output a synthesized speech waveform. The predetermined feature amount is a phoneme length,
The preliminary selection means is for selecting a speech segment located in a range specified by a predetermined criterion, including the phoneme length specified by the synthesizer command, in the table selected by the table selection means. The unit connection type speech synthesizer according to claim 1, comprising: The table is provided with permissible number information for specifying the permissible number of speech segments to be pre-selected in advance,
In the table selected by the table selection means, the means for selecting is a number specified by the allowable number, centered on a speech unit having a phoneme length specified by the synthesizer command. The unit connection type speech synthesizer according to claim 2, comprising means for selecting a speech unit. The predetermined feature amount is a quantized fundamental frequency (F0),
The preliminary selection means is for selecting a speech unit located in a range specified by a predetermined criterion, including a fundamental frequency specified by the synthesizer command, in the table selected by the table selection means. The unit connection type speech synthesizer according to claim 1, comprising: The table is provided with permissible number information for specifying the permissible number of speech segments to be pre-selected in advance,
In the table selected by the table selecting means, the means for selecting is a number specified by the allowable number centered on a speech unit having a fundamental frequency specified by the synthesizer command. 5. The unit connection type speech synthesizer according to claim 4, further comprising means for selecting a speech unit. The preliminary selection means includes
A search for searching for a speech unit having a value of the predetermined feature amount specified by the target feature amount of the synthesizer command in a table selected by the table selection means by a predetermined search algorithm Means,
And means for selecting a speech element located in a predetermined range centered on the speech element searched by the search means from the table selected by the table selection means. The unit connection type speech synthesizer according to any one of claims 1 to 5. The search means is a binary tree search algorithm for a speech element having a value of the predetermined feature value specified by the target feature value of the synthesizer command in the table selected by the table selection means. The unit-connected speech synthesizer according to claim 6, further comprising: a binary tree search means for searching by the above. The selection means has a predetermined cost calculated from a speech element candidate preliminarily selected by the preliminary selection means based on a feature value of the speech element and a target feature value. The unit connection type speech synthesizer according to any one of claims 1 to 7, further comprising means for selecting a speech unit that satisfies the above condition.

Images