JP4424024B2 - Segment-connected speech synthesizer and method - Google Patents

Description

  The present invention relates to a speech synthesizer, and more particularly to a speech synthesizer that performs speech synthesis that matches a synthesizer command by selecting and connecting speech segments based on a predetermined cost function.

  Speech recognition and speech synthesis are important technologies for realizing interfaces between humans and various systems using computers. By using these and artificial intelligence technology together, the user can use various services without being aware that the other party is a computer system.

  Speech synthesis is especially important as an interface for system output to humans. Humans are sensitive to the unnaturalness of synthesized speech. If the user feels that the synthesized speech is unnatural, it will affect the utterance, and as a result, the dialogue between the human and the system may not be successful.

  As a recent speech synthesis technology, a large number of human utterances are collected in advance and speech segments in units of words, syllables, phonemes, etc. are associated with phoneme labels to create a database, and at the time of synthesis, specified words, syllables, There is known one that selects and connects the most appropriate speech segment from speech segments corresponding to phonemes. This is called segment-connected speech synthesis. The phoneme label usually refers to a phoneme symbol describing each phoneme and its start / end time. In addition to this, acoustic feature quantities such as MFCC (Mel-Frequency Cepstrum Coefficient) and fundamental frequency (F0) in the section, and phoneme symbols of the front and rear segments may be included.

  In unit-connected speech synthesis, there is a problem of how to extract an appropriate speech unit from a database based on a given synthesis target.

  The data constituting the synthesis target typically includes phonemes and speech feature quantities such as F0, duration, MFCC, and power. These are hereinafter referred to as “synthesizer commands”.

  In unit-connected speech synthesis, an evaluation function called “cost” is defined to express the difference between the synthesizer command and the F0, duration, MFCC, power, etc. of speech units, and the natural degradation associated with the connection. Then, an optimum speech unit sequence is determined by obtaining a speech unit that minimizes the cost.

  The applicant of the present application decomposes the above-mentioned “cost” into “sub-costs” corresponding to certain voice features, and combines them (for example, linear sum) to define a unit-connected speech synthesis. Has proposed. For example, see Patent Document 1.

  There are sub-costs that are calculated from physical quantities and those that are obtained from rules created in advance from symbolic information. The former is often a non-linear operation for a plurality of sample values, and the amount of calculation is relatively large. The latter is often in the form of a simple table reference, and when it is realized by table reference, the amount of calculation required for sub-cost calculation is very small.

  The above is only an example, but the present invention is not limited to this example, and the calculation amount of each sub-cost often varies greatly depending on the type.

  On the other hand, apart from the above, the sub-costs can be broadly divided into two types, those related to the target cost and those belonging to the connection cost. The target cost represents an error between the synthesis target and the segment candidate. The connection cost represents an error (discontinuity) between adjacent segments in the synthesized speech.

  When such speech synthesis is to be performed in real time, the problem is how to perform segment selection and synthesis at high speed. In order to increase the speed of this process, it is desirable to perform the cost calculation of the segment selection at a high speed and to increase the speed of the process for connecting the selected speech segment.

  When speech units are actually connected, the waveform data of the speech units is required. However, in order to increase the quality of synthesized speech, the amount of individual waveform data is relatively large. It is necessary to increase the number of speech elements stored in the fragment database. As a result, the capacity of the entire speech unit database increases. Therefore, conventionally, only the acoustic feature amount of the segment data is stored in the memory to calculate the cost of segment selection, and the speech segment database is stored in a storage device having a relatively large capacity such as a fixed hard disk, After the segment is selected, the waveform data is read when the segment is connected.

JP 2003-208188 A (paragraphs 0014 to 0047)

  However, the access speed of a storage device with a large capacity is relatively low. For this reason, there is a problem that after selecting a segment, it takes a long time to acquire waveform data of the speech segment, and it takes a long time to synthesize speech.

  Therefore, an object of the present invention is to provide a unit connection type speech synthesis apparatus and method capable of performing speech synthesis at high speed while maintaining a large number of usable speech units.

  The segment-connected speech synthesizer according to the first aspect of the present invention calculates a cost including a plurality of sub-costs between a synthesized speech target and a speech segment candidate, and based on the cost, A speech synthesizer that synthesizes speech by selecting and connecting speech units from a speech unit database including speech unit candidates, wherein each speech unit candidate is an acoustic feature of each speech unit A first storage device for storing a speech unit database, and acoustic feature data of a plurality of speech unit candidates stored in the speech unit database, including volume data and waveform data of each speech unit Can be accessed at a higher speed than the first storage device for storing the waveform data of a speech unit candidate selected according to a predetermined standard among a plurality of speech unit candidates stored in the speech unit database Including a second storage device The plurality of sub-costs includes an access speed cost related to the access speed to the storage device in which the waveform data of each speech unit candidate is stored, and the acoustic feature value data of each of the plurality of speech unit candidates is The speech synthesizer further includes a plurality of sub-costs with the target of the synthesized speech. The first flag indicates whether the waveform data of the single candidate is stored in the first storage device or the second storage device. Selecting means for selecting one speech unit candidate for which the cost calculated in step 1 satisfies a predetermined condition from a plurality of speech unit candidates based on the acoustic feature quantity stored in the second storage device; Based on the first flag corresponding to the selected speech waveform, the speech unit candidate speech waveform selected by the selection unit is read from either the first storage device or the second storage device and synthesized. voice Connect according to the target, and a connection means for outputting a synthesized speech waveform.

  Preferably, the speech unit database stored in the first storage device is a speech to store corresponding waveform data in the second storage device among a plurality of speech unit candidates included in the speech unit database. Selection criterion information serving as a criterion for selecting a segment candidate is attached, and the segment-connected speech synthesizer further includes a plurality of speech elements included in the speech segment database stored in the first storage device. Load means for loading waveform data of a speech element candidate selected by the selection criterion information among the candidate pieces into the second storage device is included.

  More preferably, it further includes selection criterion information generating means for generating selection criterion information of the speech segment database based on the test data.

  The selection criterion information generating means includes: speech synthesis simulating means for simulating speech synthesis using speech segment candidates included in the speech segment database based on the test data; and a plurality of selection criteria information generating means included in the speech segment database. For each of the speech segment candidates, a frequency recording unit for recording the frequency selected at the time of speech synthesis by the speech synthesis simulation unit may be included. The selection criterion information may be frequency information of each of a plurality of speech unit candidates included in the speech unit database recorded by the frequency recording unit.

  Preferably, the loading unit selects a predetermined number of high-frequency ones selected by the speech synthesis by the speech synthesis simulation unit from the speech units included in the speech unit database based on the frequency information, and selects them. Means for loading the waveform data of the recorded speech segment into the second storage device.

  More preferably, the speech synthesis simulation means calculates an access speed cost among a plurality of sub-costs between a target of synthesized speech obtained from the test data and a plurality of speech unit candidates included in the speech unit database. One speech element for which the cost calculated including the access cost to be excluded and the selection frequency cost calculated based on the frequency with which each speech segment candidate recorded by the frequency recording means is selected satisfies a predetermined condition Means are included for selecting the piece candidates from the speech unit database.

  The unit connection type speech synthesizer is provided corresponding to a cache memory, a cache memory management mechanism provided for the cache memory, and a plurality of speech unit candidates. Means for storing a second flag for recording whether or not it has been selected, means for initializing the value of the second flag to a predetermined first value, and a plurality of speech segment candidates Flag updating means for updating the second flag corresponding to the selected speech segment candidate to a predetermined second value different from the first value each time any of the above is selected by the selection means; Further, it may be included. The access speed cost is calculated on the basis of the first and second flags, and the selection means determines that the cost calculated including a plurality of sub-costs including the access speed cost is a predetermined condition with the target of the synthesized speech. A means for selecting one speech unit candidate satisfying the above from a plurality of speech unit candidates based on the acoustic feature quantity stored in the second storage device may be included.

  The unit connection type speech synthesis method according to the second aspect of the present invention calculates a cost including a plurality of sub-costs between a target of synthesized speech and a speech unit candidate, and based on the cost, A unit-connected speech synthesis method for performing speech synthesis by selecting and connecting speech units from a speech unit database including speech unit candidates, wherein each speech unit candidate is an acoustic feature of each speech unit A step of storing the speech unit database in the first storage device, and the acoustic feature data of the plurality of speech unit candidates stored in the speech unit database. And the second speech unit waveform data selected on the basis of a predetermined criterion among the plurality of speech unit candidates stored in the speech unit database, which can be accessed at a higher speed than the first storage device. The memory to be stored in the storage device The plurality of sub-costs includes an access speed cost related to an access speed to the storage device in which the waveform data of each speech unit candidate is stored, and each acoustic feature amount data of the plurality of speech unit candidates Includes a first flag indicating whether the waveform data of the speech segment candidate is stored in the first storage device or the second storage device, and the speech synthesis method further includes a target for the synthesized speech, Selection that selects one speech unit candidate whose cost calculated including a plurality of sub-costs satisfies a predetermined condition from the plurality of speech unit candidates based on the acoustic feature quantity stored in the second storage device The speech waveform of the speech unit candidate selected in the step and the selection step is selected from either the first storage device or the second storage device based on the first flag corresponding to the selected speech waveform. Out the connections per synthesizer command and a connection step of outputting synthetic speech waveform.

  Preferably, the speech unit database stored in the first storage device is a speech to store corresponding waveform data in the second storage device among a plurality of speech unit candidates included in the speech unit database. The unit-connected speech synthesis method further includes selection criterion information serving as a criterion for selecting a segment candidate, and further includes a plurality of speech unit candidates included in the speech unit database stored in the first storage device. And a loading step of loading waveform data of the speech segment candidate selected by the selection criterion information into the second storage device.

  More preferably, the unit connection type speech synthesis method further includes a selection criterion information generation step of generating selection criterion information of the speech unit database based on the test data.

  The selection criterion information generation step includes a speech synthesis simulation step of simulating speech synthesis using speech unit candidates included in the speech unit database based on the test data, and a plurality of speeches included in the speech unit database. A frequency recording step for recording the frequency selected at the time of speech synthesis by the speech synthesis simulation step for each of the segment candidates may be included. The selection criterion information may be frequency information of each of a plurality of speech unit candidates included in the speech unit database recorded in the frequency recording step.

  Preferably, the loading step selects a predetermined number of high-frequency ones selected in the speech synthesis by the speech synthesis simulation step from among the speech units contained in the speech unit database based on the frequency information, and selects them. Loading waveform data of the generated speech segment into the second storage device.

  More preferably, the speech synthesis simulation step calculates an access speed cost among a plurality of sub-costs between a target of synthesized speech obtained from test data and a plurality of speech unit candidates included in the speech unit database. One speech element for which the cost calculated including the access cost to be excluded and the selection frequency cost calculated based on the frequency at which each speech element candidate recorded in the frequency recording step is selected satisfies a predetermined condition Selecting a segment candidate from the speech segment database.

  The unit connection type speech synthesis method may be a method executed on a computer having a cache memory and a cache memory management mechanism provided for the cache memory. The unit connection type speech synthesis method is further provided with a second flag provided corresponding to a plurality of speech unit candidates and recording whether or not the corresponding speech unit candidate is selected in the selection step. Each of the speech unit candidates, the step of initializing the value of the second flag to a predetermined first value, and the selection of each of the plurality of speech unit candidates in the selection step. The method may further include a step of updating the second flag corresponding to the single candidate to a predetermined second value different from the first value. The access speed cost is calculated based on the first and second flags. The selection step stores, in the second storage device, one speech unit candidate whose cost calculated by including a plurality of sub-costs including the access speed cost satisfies a predetermined condition with the target of synthesized speech The method may include a step of selecting from a plurality of speech segment candidates based on the acoustic feature value thus determined.

  In the unit selection type speech synthesis, the actually selected speech unit is biased. Therefore, by storing speech units that are easily selected in a storage device (for example, a memory) that can be accessed at a higher speed than the speech unit database, the overall speech synthesis speed can be increased. Furthermore, when selecting a segment, the access speed of the storage device storing the waveform data of each speech segment is added to the cost. The cost corresponding to the access speed of the storage device is referred to as “access speed cost” in this specification. The calculation formula is designed in advance so that the access speed cost decreases as the access speed of the storage device increases (close to 0) and increases as the access speed decreases. A speech unit having waveform data stored in a high-speed storage device is more likely to be actually selected for speech synthesis, and the speech synthesis speed can be increased as a whole.

  FIG. 1 shows a block diagram of a speech synthesis system 20 according to an embodiment of the present invention. Referring to FIG. 1, this speech synthesis system 20 uses speech unit DB 30 similar to the prior art and speech data using speech unit DB 30 using test data 42 consisting of a number of texts for speech synthesis. Frequency information generation for simulating synthesis, calculating the frequency used in speech synthesis for each speech unit included in speech unit DB 30, and generating speech unit DB 34 with frequency information having frequency information 44 Device 32.

  The speech synthesis system 20 further receives a synthesizer command 36 obtained as a result of analyzing the target text as an input, and selects and connects an appropriate speech unit from speech units included in the speech unit DB 34 with frequency information. A speech synthesizer 38 for outputting a synthesized speech waveform 40. The speech synthesizer 38 selects the acoustic information of each speech unit in the speech unit DB 34 with frequency information and selects the frequency information 44 out of the speech units in the speech unit DB 34 with frequency information. A memory 48 for storing in advance what is represented with a high appearance frequency is included.

  The speech synthesis system 20 further refers to the acoustic information of each speech unit in the speech unit DB 34 with frequency information and the frequency information 44 when the speech synthesizer 38 is started up, so that the speech unit DB 34 with the frequency information is stored. A speech unit data loading device 46 for loading waveform data of a predetermined number of speech units having a higher appearance frequency selected from among them into the memory 48;

  Referring to FIG. 2, the frequency information generation device 32 uses the speech unit DB 30 to perform the same process as speech synthesis on each input included in the test data 42 and Has a function to calculate the usage frequency. The frequency information generation device 32 receives each text sentence of the test data 42 and generates a synthesizer command creating unit 60 for creating a synthesizer command 62 as a synthesis target, and the synthesizer command 62 and the speech unit DB 30. A target cost calculation unit 68 for calculating a target cost with the acoustic feature amount of each speech unit, and a connection cost between the synthesizer command 62 and the acoustic feature amount of each speech unit in the speech unit DB 30 are calculated. And a selection frequency cost calculation unit 72 for calculating a selection frequency cost that reflects the frequency at which each speech unit in the speech unit DB 30 is selected by speech synthesis.

The selection frequency cost is a cost introduced in order to intentionally bias the speech unit selected by the frequency information generation device 32. The selection frequency cost is calculated by a calculation formula that decreases as the frequency at which a speech segment is selected increases and increases as the frequency decreases. In the present embodiment, the selection frequency cost Cs _i of the i-th speech element is calculated by the following equation.

ただしｎ_iはｉ番目の音声素片の選択回数、Ｎは全ての素片の出現回数の和、ａ、Ｎ₀、及びｒは適当な定数である（ただしｒ＞０、ａ＞０、及びＮ≧０）。この式によれば、ｉ番目の音声素片の選択頻度コストＣｓ_iの値は、選択頻度ｎ_iが０であればａであり、選択頻度ｎ_iが多くなるにつれて０に近づいて行く。選択頻度コスト算出部７２はこのため、頻度情報４４に記憶されている各音声素片の頻度情報を使用する。

Where n _i is the number of times the i-th speech unit is selected, N is the sum of the number of occurrences of all the units, a, N ₀ , and r are appropriate constants (where r> 0, a> 0, and N ≧ 0). According to this equation, the value of the selected frequency cost Cs _i of i-th speech unit is a If 0 is selected frequency n _i, approaches zero as the selection frequency n _i increases. For this reason, the selection frequency cost calculation unit 72 uses the frequency information of each speech unit stored in the frequency information 44.

  Further, the frequency information generation device 32 is based on the target cost calculated by the target cost calculation unit 68, the connection cost calculated by the connection cost calculation unit 70, and the selection frequency cost calculated by the selection frequency cost calculation unit 72. The total cost is calculated, and the speech unit having the smallest total cost is selected, so that the unit selection unit 64 for simulating the unit selection at the time of actual speech synthesis and the unit selection unit 64 select A frequency information update unit 66 for updating the frequency information 44 is included with respect to the speech unit.

  With reference to FIG. 3, the data loaded into the memory 48 by the speech segment data loading device 46 shown in FIG. 1 will be described. As shown in FIG. 3, the memory 48 includes an acoustic feature amount storage area 120 for storing acoustic feature amounts 130,... Of all speech units in the speech unit DB 34 with frequency information, and speech units with frequency information. A waveform data storage area 122 for storing waveform data having an appearance frequency recorded in the frequency information 44 having a frequency equal to or higher than a predetermined value among speech elements in the DB 34 is included. These are all loaded into the memory 48 by the speech unit data loading device 46 when the speech synthesizer 38 is activated.

As described above, each acoustic feature 130 includes a phoneme label, a fundamental frequency (F0), an MFCC, a power (not shown), and a duration (not shown). The first flag (F ₁ ) 140 indicating whether the waveform data is stored in the speech unit DB 34 with frequency information or in the waveform data storage area 122 of the memory 48, and the waveform data of the speech unit are And a second flag (F ₂ ) 142 indicating whether or not the data has been read. The flag 140 is used when the speech element data loading device 46 loads the acoustic feature quantity into the acoustic feature quantity storage area 120 and then loads the waveform data of the high-frequency speech element into the waveform data storage area 122. Set by the single data load device 46.

  In the present embodiment, when the first flag 140 is 0, the waveform data is stored in the speech unit DB 34 with frequency information, and when the flag is 1, the waveform data is stored in the memory 48 waveform. It indicates that the data is stored in the data storage area 122. Therefore, it is possible to determine where to read the waveform data by looking at the value of the first flag 140.

  The flag 142 is initially initialized to 0 and updated to “1” when the waveform data is read while actually performing speech synthesis. This is because the data read from the hard disk or the memory may be stored in a cache that can be accessed at a higher speed than the memory when the device is realized by a computer. The value of the second flag 142 is reflected in the access speed cost.

  Referring to FIG. 4, speech synthesizer 38 receives synthesizer commands 36 for determining synthesis targets in addition to memory 48 described above, is a speech unit stored in memory 48, and is a synthesizer command. A target cost calculation unit 82 for calculating a target cost between the acoustic feature quantity of the speech unit having the phoneme label specified by 36 and the synthesizer command 36, and the speech unit and synthesizer in the memory 48. The access cost cost is calculated based on the connection cost calculation unit 84 for calculating the connection cost between the command 36 and the first and second flags 140 and 142 corresponding to the speech segments stored in the memory 48. And an access speed cost calculation unit 86 for calculation.

  The speech synthesizer 38 further determines the total cost based on the target cost calculated by the target cost calculation unit 82, the connection cost calculated by the connection cost calculation unit 84, and the access speed cost calculated by the access speed cost calculation unit 86. And the synthesized speech waveform 40 is output by connecting the segment selection unit 80 for selecting the speech unit having the minimum total cost and the waveform data of the speech unit selected by the unit selection unit 80. Connecting portion 88 for the purpose.

  The connection unit 88 outputs the following signals in order to read the speech unit designated by the unit selection unit 80 from the speech unit DB 34 with frequency information or the memory 48. That is, the connection unit 88 takes a level corresponding to the first flag 140 of the speech unit specified by the unit selection unit 80, and reads the waveform data from either the memory 48 or the speech unit DB 34 with frequency information. It has a function of outputting a selection signal 100 for designating the address and an address signal 102 for designating the address for reading the waveform data. The selection signal 100 takes the H level when the flag of the designated speech unit is 1, and takes the L level otherwise.

  The speech synthesizer 38 has first and second inputs as functional blocks for reading out waveform data by the connection unit 88, and the first and second input signals according to the level of the selection signal 100. When receiving a selection circuit 90 for selecting and outputting one, an inversion circuit 92 for inverting the level of the selection signal 100 and outputting an inversion selection signal 104, and an inversion selection signal 104 of H level, And an access unit 94 for reading out waveform data at an address designated by the signal 102 from the speech unit DB 34 with frequency information and supplying the waveform data to the first input of the selection circuit 90. On the other hand, when the memory 48 receives the selection signal 100 of H level, the memory 48 gives the waveform data of the address specified by the address signal 102 to the second input of the selection circuit 90. The selection circuit 90 selects and outputs the second input signal when the selection signal 100 is at the H level, and selects the first input signal when the selection signal 100 is at the L level.

The speech synthesizer 38 further updates the second flag 142 (F ₂ ) of the corresponding speech unit in the memory 48 with respect to the waveform data instructed to be read out by the connection unit 88, to “1”. An update unit 96 is included. In any case, data read from the speech element DB 34 with frequency information or the memory 48 is usually stored in a cache memory of a computer. The cache memory can be accessed even faster than the memory 48. Since the waveform data read even once is highly likely to be stored in the cache memory, the second flag is updated in this way and reflected in the cost calculation in the next access speed calculation unit, making it easier to select. To do.

  Since the capacity of the cache memory is limited, data stored in the cache is selected and deleted by some algorithm, and new data is stored there. Originally, it is only necessary to know which waveform data is stored in the cache memory, but the cache memory is managed by computer hardware, and the contents of the cache cannot be known by software. Therefore, in this embodiment, the design assumes that the data that has been read even once is stored in the cache memory, apart from what data is actually stored in the cache memory. . Of course, if it is possible to easily know which speech element corresponds to the waveform data stored in the cache memory, it is preferable to reflect this in the calculation of the access speed cost.

  The sub-costs used in the present embodiment correspond to the fundamental frequency (F0) error, duration error, MFCC error, F0 discontinuous error, MFCC discontinuous error, and phoneme environment error in addition to the access speed cost described above. Includes 6 types of sub-costs. Among these, the former three belong to the target cost, and the latter three belong to the connection cost.

In the cost calculation by the segment selection unit 64 according to the present embodiment, the cost C ₀ is calculated from the sub cost as follows.

ただし、Ｃ_i1（ｉ1＝１〜３）はターゲットサブコスト、Ｃ_i2（ｉ2＝１〜３）は接続コスト、ｗ_i1（ｉ1＝１〜３）はターゲットサブコスト間に定義された重み、ｗ_i2（ｉ2＝１〜３）は接続サブコスト間に定義された重み、ｐ_１及びｐ_２はそれぞれ、ターゲットコストと接続コスト間に定義された重み、ｗ_sはアクセス速度コストの重み、Ｆ₁及びＦ₂はそれぞれ第１及び第２のフラグの値である。また重みｗsの大きさは、他のサブコストの値の範囲を考慮して適当に決定し、品質に影響を及ぼす極端な偏りが生じないようにする。

Where C _i1 (i1 = 1 to 3) is a target sub cost, C _i2 (i2 = 1 to 3) is a connection cost, w _i1 (i1 = 1 to 3) is a weight defined between the target sub costs, w _{i2 (i2} = _1~3) is the weight that is defined between the connection sub-cost, respectively p ₁ and p ₂ are the weight defined between the target costs and the concatenation cost, w _s is the weight of the access speed cost, F ₁ and F ₂ is the value of the first and second flags, respectively. The size of the weight ws is appropriately determined in consideration of the range of other sub-cost values so as not to cause an extreme bias affecting quality.

  The speech synthesis system 20 according to the first embodiment described above operates as follows. Broadly speaking, there are two aspects to the operation of the speech synthesis system 20. The first aspect is the creation of the speech unit DB34 with frequency information, and the second aspect is the speech synthesis using the speech unit DB34 with frequency information created in the first aspect. Hereinafter, it demonstrates in order.

  First, in the first aspect, the frequency information generating device 32 creates a speech element DB 34 with frequency information as follows. With reference to FIG. 2, first, a speech segment DB 30 is prepared. The frequency information is not attached to the speech unit DB 30. Further, an area for storing the frequency information 44 is secured on the memory. Further, the frequency of selection of each speech unit in the frequency information 44 is initialized to 0.

  When the first text of the test data 42 is given to the frequency information generating device 32, the synthesizer command creation unit 60 creates a synthesizer command 62 for each synthesis target phoneme based on the text, and sends it to the segment selection unit 64. give. The segment selection unit 64 gives the acoustic feature amount specified by the synthesizer command 62 to the target cost calculation unit 68 and the connection cost calculation unit 70. The target cost calculation unit 68 and the connection cost calculation unit 70 calculate a target cost and a connection cost with each speech unit included in the speech unit DB 30 based on the given acoustic feature amount, and a unit selection unit 64. To give. The selection frequency cost calculation unit 72 calculates the selection frequency cost of each speech unit candidate according to the equation (1) and gives it to the unit selection unit 64. In the first process, since the selection frequency is all 0, the selection frequency cost is “a” from the equation (1).

  The element selection unit 64 uses the target cost given from the target cost calculation unit 68, the connection cost given from the connection cost calculation unit 70, and the selection frequency cost given from the selection frequency cost calculation unit 72 to formula (2). To calculate the total cost. The unit selection unit 64 selects the speech unit having the minimum total cost, and gives information indicating the selected speech unit to the frequency information update unit 66.

  The frequency information update unit 66 adds 1 to the frequency of the phoneme selected by the segment selection unit 64 among the frequency information in the frequency information 44. This completes the process for the first phoneme.

  Similar processing is repeated for each phoneme of the first text. By repeating this, the frequency information 44 is gradually updated. In the selection frequency cost calculation by the selection frequency cost calculation unit 72, the contents of the frequency information 44 are reflected. In other words, the selection frequency cost decreases as the number of selections increases. Therefore, it is highly likely that a segment candidate that has been selected will be selected in subsequent segment selection. As a result, when such a process is repeated, speech unit having similar acoustic feature considerations, which have been selected, are easier to select, and candidates that have not been selected are Furthermore, it becomes difficult to select.

  By repeating the above-described processing for all the texts of the test data 42, the speech unit DB 34 with frequency information is completed. When the speech unit DB 34 with frequency information is completed, speech synthesis by the speech synthesizer 38 becomes possible.

  By intentionally biasing the appearance frequency of speech elements, only some of the elements are often selected in regions where the density of the elements is high in the feature space, and the frequency of appearance of other elements Go down. Accordingly, the frequency ranking of the other segments is relatively increased in the portion where the density of the segments is low. As a result, the distribution of speech units storing waveform data in a storage device that can be accessed at high speed during actual synthesis spreads, and speech units corresponding to waveform data stored in a high-speed storage device are selected more frequently. Become so.

  Actual speech synthesis is performed as follows. Referring to FIG. 1, first, an acoustic feature quantity of a speech unit in speech unit DB 34 with frequency information is stored in memory 48 by speech unit data loading device 46. The values of the first and second flags associated with the acoustic feature quantity are initialized with 0. Next, the waveform data of a predetermined number of upper speech units is stored in the memory 48 with the frequency information 44 as a reference. For the speech element having the waveform data loaded into the memory 48, the value of the first flag 140 (see FIG. 3) associated with the acoustic feature quantity stored in the memory 48 is set to “1”.

  When the voice unit data loading device 46 finishes loading the data into the memory 48, the actual voice synthesis is started. Referring to FIG. 4, when the synthesizer command 36 is given, the segment selection unit 80 transmits the sound designated by the synthesizer command 62 to the target cost calculation unit 82, the connection cost calculation unit 84 and the access speed cost calculation unit 86. Give a feature value. The target cost calculation unit 82 and the connection cost calculation unit 84 use the given acoustic feature amount, and among the speech unit candidates stored in the memory 48, the acoustic feature amount of the speech unit candidate of the specified phoneme label. The target cost and the connection cost are calculated between them and given to the segment selection unit 80. The access speed cost calculation unit 86 calculates an access speed cost based on the values of the first flag 140 and the second flag 142 of each speech unit candidate, and gives the access speed cost to the unit selection unit 80.

  The segment selection unit 80 calculates the total cost according to the equation (2) based on the target cost, the connection cost, and the access cost given from the target cost calculation unit 82, the connection cost calculation unit 84, and the access speed cost calculation unit 86. calculate. The segment selection unit 80 further selects a speech unit candidate having the smallest total cost calculated as described above, and gives information indicating the speech unit candidate and an acoustic feature amount to the connection unit 88. The connection unit 88 outputs a waveform data address to the address signal 102 and a first flag 140 to the selection signal 100 in the given acoustic feature amount.

  For example, when the waveform data of the selected speech unit is stored in the memory 48, the value of the first flag of the speech unit is 1, and the selection signal 100 is H level. The address signal 102 is the address of the waveform data of the selected speech unit in the memory 48. Address signal 102 is provided to memory 48. Since the H level selection signal 100 is applied to the memory 48, the memory 48 reads the waveform data at the address specified by the address signal 102 and supplies it to the second input of the selection circuit 90. Since the inversion selection signal 104 is at the L level, the access unit 94 does nothing.

  Since the selection signal 100 is at the H level, the selection circuit 90 selects the second input. In other words, the selection circuit 90 gives the output from the memory 48 to the connection unit 88. The connection unit 88 performs waveform connection using this waveform data.

  When the waveform data of the selected speech unit is stored in the speech unit DB 34 with frequency information, the value of the first flag of the speech unit is 0, and the selection signal 100 is L level. . The address signal 102 is the address of the waveform data of the selected speech unit in the speech unit DB 34 with frequency information. Address signal 102 is applied to access unit 94. Since the H level inversion selection signal 104 is supplied to the access unit 94, the access unit 94 reads the waveform data at the address specified by the address signal 102 from the speech unit DB 34 with frequency information, and the first input of the selection circuit 90. To give. Since the selection signal 100 is at L level, nothing is output from the memory 48.

  Since the selection signal 100 is at the L level, the selection circuit 90 selects the signal of the first input and supplies it to the connection unit 88. That is, in this case, the waveform data read from the speech element DB 34 with frequency information is given to the connection unit 88 and used for waveform connection.

  Regardless of the waveform data in the memory 48 or the waveform data in the speech element DB 34 with frequency information, the data read by the connection unit 88 is likely to be stored in a cache memory (not shown). Accordingly, the flag update unit 96 updates the value of the second flag 142 associated with the acoustic feature amount of the speech unit from which the waveform data is read by the connection unit 88 in the memory 48 to “1”. As a result, when the same speech unit is selected next, the access speed cost becomes smaller, and the possibility that the same speech unit is selected increases. It is highly possible that the waveform data corresponding to the speech segment is stored in the cache and can be accessed at high speed. Therefore, there is a high possibility that the reading of waveform data as a whole is speeded up.

  As described above, in the speech synthesis system 20 according to the present embodiment, the speech synthesis experiment using the test data 42 is performed using the speech segment DB 30 to check the frequency with which each speech frequency is selected. During the speech synthesis experiment, a sub-cost called a selection frequency cost is introduced so that the selected unit is more easily selected and the other units are more difficult to select according to the number of times the speech unit is selected. , Make sure that the selection of speech segments is artificially biased.

  At the time of speech synthesis, waveform data of a predetermined number of speech segment candidates having higher frequencies is stored in the memory. Furthermore, as a sub-cost, an access speed cost reflecting the access speed of the storage device storing the waveform data is defined, and the speech unit candidate storing the waveform data in the storage device capable of high-speed access is easily selected. . As a result, a speech unit stored in a storage medium such as a memory that can be accessed at high speed can be easily selected, and the speech synthesis process as a whole is accelerated.

  Further, in consideration of the use of a cache memory or the like, the access speed cost is designed so that the access speed cost is calculated to be low for the recently selected speech waveform. Thereby, it is possible to improve the processing speed using the cache control by the computer.

  In the apparatus of the above-described embodiment, it is assumed that the frequency information 44 is not updated after the frequency information 44 is once created. However, the present invention is not limited to such an embodiment. For example, at the time of speech synthesis, the frequency information 44 is transferred to the memory 48, and the frequency information 44 is updated each time a segment candidate is selected. When the entire process is completed, or a predetermined number of segment candidates is selected. Each time it is performed, the original frequency information 44 may be written back. By doing so, it is possible to determine the storage location of the waveform data based on the selection frequency in the speech synthesis based on the actual data as well as the experiment.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

1 is a block diagram of a speech synthesis system 20 according to an embodiment of the present invention. It is a block diagram of the frequency information generation device 32 shown in FIG. It is a figure which shows typically the structure of the one part storage area of the memory 48 shown in FIG. It is a block diagram of the speech synthesizer 38 shown in FIG.

Explanation of symbols

  20 speech synthesis system, 30 speech segment DB, 32 frequency information generating device, 34 speech segment DB with frequency information, 36,62 synthesizer command, 38 speech synthesizer, 40 synthesized speech waveform, 42 test data, 44 frequency information 46, speech unit data load device, 48 memory, 60 synthesizer command creation unit, 64, 80 unit selection unit, 66 frequency information update unit, 68, 82 target cost calculation unit, 70, 84 connection cost calculation unit, 72 Selection frequency cost calculation unit, 86 access speed cost calculation unit 88 connection unit, 90 selection circuit, 100 selection signal, 102 address signal, 104 inversion selection signal

Claims (14)

Calculate the cost including multiple sub-costs between the target of synthesized speech and the speech element candidate, and select and connect the speech element from the speech element database including the multiple speech element candidates based on the cost A speech synthesizer that performs speech synthesis, and each speech unit candidate includes acoustic feature data of each speech unit and waveform data of each speech unit;
A first storage device for storing the speech segment database;
Acoustic feature quantity data of a plurality of speech unit candidates stored in the speech unit database and a speech unit selected based on a predetermined criterion among the plurality of speech unit candidates stored in the speech unit database A second storage device for storing one candidate waveform data, which is accessible at a higher speed than the first storage device,
The plurality of sub-costs includes an access speed cost related to an access speed to a storage device in which waveform data of each speech unit candidate is stored,
Each acoustic feature amount data of the plurality of speech unit candidates includes a first flag indicating which of the first and second storage devices the waveform data of the speech unit candidate is stored.
The speech synthesizer further includes:
One speech segment candidate whose cost calculated including the plurality of sub-costs satisfies a predetermined condition with the target of the synthesized speech is stored as an acoustic feature amount stored in the second storage device. A selection means for selecting from the plurality of speech segment candidates based on;
Based on the first flag corresponding to the selected speech waveform, the speech unit candidate speech waveform selected by the selection means is selected from either the first storage device or the second storage device. Connection unit for reading and connecting according to the target of the synthesized speech and outputting a synthesized speech waveform. The speech unit database stored in the first storage device is a speech to store corresponding waveform data in the second storage device among a plurality of speech unit candidates included in the speech unit database. Selection criteria information that serves as a criterion for selecting segment candidates is attached.
The unit-connected speech synthesizer further includes a speech unit selected according to the selection criterion information from the plurality of speech unit candidates included in the speech unit database stored in the first storage device. The unit connection type speech synthesizer according to claim 1, further comprising loading means for loading candidate waveform data into the second storage device. The unit connection type speech synthesizer according to claim 2, further comprising selection criterion information generation means for generating the selection criterion information of the speech segment database based on test data. The selection criterion information generating means includes:
Speech synthesis simulation means for simulating speech synthesis using speech segment candidates included in the speech segment database based on the test data;
Frequency recording means for recording the frequency selected during speech synthesis by the speech synthesis simulation means for each of the plurality of speech unit candidates included in the speech unit database;
The unit connection type speech synthesis according to claim 3, wherein the selection criterion information is frequency information of each of the plurality of speech unit candidates included in the speech unit database recorded by the frequency recording unit. apparatus. Based on the frequency information, the loading unit selects a predetermined number of high-frequency ones selected by speech synthesis by the speech synthesis simulation unit from among speech units included in the speech unit database, 5. The unit-connected speech synthesizer according to claim 4, further comprising means for loading waveform data of a selected speech unit into the second storage device. The speech synthesis simulating means includes the access speed of the plurality of sub-costs between a target of synthesized speech obtained from the test data and the plurality of speech unit candidates included in the speech unit database. The cost calculated including the access cost excluding the cost and the selection frequency cost calculated based on the frequency with which each speech segment candidate recorded by the frequency recording means is selected satisfies a predetermined condition. The unit-connected speech synthesizer according to claim 4 or 5, comprising means for selecting one speech unit candidate from the speech unit database. The unit connection type speech synthesizer comprises:
Cache memory,
A cache memory management mechanism provided for the cache memory;
Means for storing a second flag provided corresponding to the plurality of speech unit candidates and for recording whether or not the corresponding speech unit candidate has been selected by the selection unit;
Means for initializing the value of the second flag to a predetermined first value;
Each time one of the plurality of speech unit candidates is selected by the selection unit, the second flag corresponding to the selected speech unit candidate is set to a predetermined second value different from the first value. A flag updating means for updating,
The access speed cost is calculated based on the first and second flags,
The selection means selects one speech unit candidate for which the cost calculated by including the plurality of sub-costs including the access speed cost satisfies a predetermined condition with respect to the target of the synthesized speech. The unit connection type speech synthesizer according to any one of claims 1 to 6, further comprising means for selecting from the plurality of speech unit candidates based on an acoustic feature amount stored in the storage device. Calculate the cost including multiple sub-costs between the target of synthesized speech and the speech element candidate, and select and connect the speech element from the speech element database including the multiple speech element candidates based on the cost A speech synthesis method for performing speech synthesis by performing speech synthesis, wherein each speech unit candidate includes acoustic feature data of each speech unit and waveform data of each speech unit;
Storing the speech segment database in a first storage step;
Acoustic feature quantity data of a plurality of speech unit candidates stored in the speech unit database and a speech unit selected based on a predetermined criterion among the plurality of speech unit candidates stored in the speech unit database Storing waveform data of one candidate in a second storage device accessible at a higher speed than the first storage device,
The plurality of sub-costs includes an access speed cost related to an access speed to a storage device in which waveform data of each speech unit candidate is stored,
Each acoustic feature amount data of the plurality of speech unit candidates includes a first flag indicating which of the first and second storage devices the waveform data of the speech unit candidate is stored.
The speech synthesis method further includes:
One speech segment candidate whose cost calculated including the plurality of sub-costs satisfies a predetermined condition with the target of the synthesized speech is stored as an acoustic feature amount stored in the second storage device. A selection step of selecting from the plurality of speech segment candidates based on;
Based on the first flag corresponding to the selected speech waveform, the speech unit candidate speech waveform selected in the selection step is selected from either the first storage device or the second storage device. A unit connection type speech synthesis method including a connection step of reading out and connecting in accordance with the synthesizer command and outputting a synthesized speech waveform. The speech unit database stored in the first storage device is a speech to store corresponding waveform data in the second storage device among a plurality of speech unit candidates included in the speech unit database. Selection criteria information that serves as a criterion for selecting segment candidates is attached.
The unit-connected speech synthesis method further includes a speech unit selected by the selection criterion information among the plurality of speech unit candidates included in the speech unit database stored in the first storage device. The segment connection type speech synthesis method according to claim 8, further comprising a load step of loading candidate waveform data into the second storage device. The unit connection type speech synthesis method according to claim 9, further comprising a selection criterion information generation step of generating the selection criterion information of the speech segment database based on test data. The selection criterion information generation step includes:
A speech synthesis simulation step of simulating speech synthesis using speech unit candidates included in the speech unit database based on the test data;
A frequency recording step for recording the frequency selected at the time of speech synthesis by the speech synthesis simulation step for each of the plurality of speech unit candidates included in the speech unit database;
The unit connection type speech synthesis according to claim 10, wherein the selection criterion information is frequency information of each of the plurality of speech unit candidates included in the speech unit database recorded in the frequency recording step. Method. Based on the frequency information, the loading step selects a predetermined number of high-frequency ones selected in speech synthesis by the speech synthesis simulation step from among speech units included in the speech unit database, The unit-connected speech synthesis method according to claim 11, comprising loading waveform data of the selected speech unit into the second storage device. In the speech synthesis simulation step, the access speed of the plurality of sub-costs between the target of synthesized speech obtained from the test data and the plurality of speech unit candidates included in the speech unit database. The cost calculated including the access cost excluding the cost and the selection frequency cost calculated based on the frequency at which each speech segment candidate recorded in the frequency recording step is selected satisfies a predetermined condition. The unit-connected speech synthesis method according to claim 11 or 12, comprising a step of selecting two speech unit candidates from the speech unit database. The unit connection type speech synthesis method is executed on a computer having a cache memory and a cache memory management mechanism provided for the cache memory, and
Storing in a predetermined storage device a second flag provided corresponding to the plurality of speech unit candidates and recording whether or not the corresponding speech unit candidate is selected in the selection step;
Initializing the value of the second flag to a predetermined first value;
Each time one of the plurality of speech unit candidates is selected in the selection step, the second flag corresponding to the selected speech unit candidate is set to a predetermined second value different from the first value. And further updating
The access speed cost is calculated based on the first and second flags,
In the selection step, one speech unit candidate whose cost calculated by including the plurality of sub-costs including the access speed cost satisfies a predetermined condition with respect to the target of the synthesized speech is selected as the second speech unit candidate. The unit connection type speech synthesis method according to claim 8, further comprising a step of selecting from the plurality of speech unit candidates based on an acoustic feature amount stored in the storage device.

Images