JP4622356B2 - Script generator for speech synthesis and script generation program for speech synthesis - Google Patents

Description

The present invention relates to a script generating program for speech synthesis script generating apparatus and speech synthesis to generate a script for speech synthesis.

A speech synthesizer is known in which a speech synthesis script including a character string corresponding to a speech to be synthesized and a symbol representing a prosody such as pitch, length, and strength is synthesized and speech synthesis is performed according to a rule. Yes.
For example, in Patent Document 1, reference data composed of a character string and height, strength, and length data is created from a musical score, and actual singing voice segmentation is performed using the reference data. Is a device for creating voice synthesis data of singing voice, in which height data, which is one of reference data, is used as a pitch of representative points of each mora.
JP-A-3-7995

  As described above, in order to perform natural speech synthesis, there has been a conventional technique in which actual speech data is analyzed to create speech synthesis data. However, in the synthesis based on the generated speech synthesis data, the pitch designation of each mora is specified. Since there was no degree of freedom, the naturalness of synthesized speech was insufficient.

For example, if the pitch is extracted based on the actual singing voice, even if it is sung at a constant pitch, a change in pitch from the pitch of the previous sound can be seen at the rise of the sound.
FIG. 10 is a diagram illustrating an example of a pitch change in a singing voice.
When such a song is sung by voice synthesis, if the pitch (scale) of this segment is determined with reference to the pitch S1 at the timing shown in FIG. Even if the change in pitch is described using high and low accent symbols that indicate pitch increase / decrease, if the pitch change width due to the pitch increase / decrease symbol does not match the scale when the pitch is almost constant, it will be heard. End up.
When singing a song, the pitch at the pitch where the pitch is constant or vibrato, etc. in the first place in the pitch change in the character units (syllables, mora, etc.) described in the text such as letters. And the difference in pitch from that is expressed by a symbol that raises or lowers the pitch, the above-described unnaturalness can be eliminated. In the case of FIG. 10, a good result can be obtained by using the scale of the pitch (S2) per timing (2) as the reference of this segment.

The present invention has as object to provide a small amount of data in a script generating program for generating apparatus and the speech synthesis script for speech synthesis which can generate a script for speech synthesis that enables high-quality voice synthesis Yes.

  In order to achieve the above object, the speech synthesis script generation device and the speech synthesis script generation program according to the present invention can detect a section or vibrato where the pitch is as constant as possible by looking at the pitch transition information of the speech to be synthesized. Main reference is to detect a reference pitch section that is a certain section, determine a reference pitch such as an average pitch in the reference pitch section, and generate a speech synthesis script that expresses the pitch transition information as a relative change with respect to the reference pitch. Features.

According to the speech synthesis script generation apparatus or the speech synthesis script generation program of the present invention, it is possible to generate a speech synthesis script that enables high-quality speech synthesis with a small amount of data.
In particular, since it represents the pitch transition information relative change from the reference pitch of the reference pitch intervals, the singing synthesis pitch change has a significant useful.

FIG. 1 is a functional block diagram showing the configuration of an embodiment of a script generator for speech synthesis according to the present invention.
In this figure, 1 is input data, 2 is a constant pitch interval detecting unit for detecting a constant pitch interval where the pitch of the input data continues to take a value within a certain threshold, and 3 is a vibrato interval for detecting a vibrato interval in which the pitch vibrates. A detecting unit 4 outputs a scale information (pitch name) nearest to a reference pitch of input data based on outputs from the constant pitch interval detecting unit 2 and the vibrato interval detecting unit 3, and 5 is the input It is a synthesis script creation unit that creates a speech synthesis script based on data and an output from the scale extraction unit 4.
Here, in the speech synthesis script according to the present invention, for example, the utterance length can be changed according to the number of the long sound symbols “−”, for example. The pitch change by the high and low accent symbols is performed, and the change level by the high and low accent symbols can be realized by describing a numerical value immediately after the high and low accent symbols.
In addition, although each said component can also be implement | achieved as each separate process part, it can also implement | achieve by the program processing in a computer.

The input data 1 includes a character string to be synthesized (mora information), segment information (speech length information, utterance of a phoneme section) when each mora is uttered, by analyzing an actual utterance or directly inputting the utterance Start time and utterance end time), pitch transition information (pitch sequence), and volume transition information (volume sequence). Note that the volume transition information may not be included.
FIG. 2 is a diagram showing an example of a configuration for obtaining such input data 1 by voice analysis. This configuration can also be realized by a computer program.
In this figure, reference numeral 11 is an input phoneme symbol string (kana, alphabet, etc.) that transcribes the utterance content and a speech waveform, is segmented, and a section (segment position) in which each mora is uttered in the speech waveform. Is a segmentation unit for obtaining a character string (kana, alphabet, etc.) and outputting the segment position. When a speech recognition unit is used instead of a segmentation unit, a character string and a segment position can be obtained without inputting the input phoneme symbol string.
Reference numeral 12 denotes a pitch detection unit that obtains a pitch from the input voice waveform by using a method such as zero cross section measurement and outputs a pitch sequence representing a time-series pitch change.
Reference numeral 13 denotes a volume detection unit that outputs a volume sequence that represents a time-series change in volume by a method such as obtaining an average energy of a certain section of the input voice waveform.
Thus, the input data 1 can be obtained by analyzing the actual speech waveform.

FIG. 3 is a diagram showing an example of the input data 1.
Here, (b) shows an example of input data obtained when “asa” is pronounced with the pitch change and utterance length as shown in (a).
As shown in this figure, segment information (speech start position, utterance end position) for each character (mora) and a pitch change sequence per unit time of each character (mora) (for example, a sequence of pitch values every 10 msec) Is a set.

  Such input data 1 is input to a scale extraction processing unit including the constant pitch interval detection unit 2, the vibrato interval detection unit 3, and the scale extraction unit 4. In this scale extraction processing unit, a pitch constant section that continues to take a value within a certain threshold or a vibrato section that crosses the same pitch many times is detected from pitch information of each syllable (mora) unit, and a constant pitch section or vibrato The longer one of the sections is set as a reference pitch section, and the nearest pitch name of the average pitch in the reference pitch section is output as scale information. When neither a constant pitch section nor a vibrato section is detected, the nearest pitch name of the average pitch information is output as scale information.

The fixed pitch interval and the vibrato interval will be described with reference to FIG.
FIG. 4A is a diagram showing a constant pitch section. As shown in the figure, a section in which the fluctuation of the pitch is within a predetermined threshold value (a constant section determination threshold value) is a constant pitch section.
Further, as shown in FIG. 4B, a section where the same pitch is crossed many times (section where the pitch vibrates) is a vibrato section.
From the pitch sequence in the input data, the constant pitch interval detector 2 detects the constant pitch interval shown in FIG. 4A, and the vibrato interval detector 3 detects the vibrato interval shown in FIG. 4B. To detect.

FIG. 5 is a flowchart showing a flow of constant pitch interval detection processing executed in the constant pitch interval detector 2.
First, the maximum constant section length (MaxLeglen) and the maximum constant section start position (MaxLegStartPos) are set to the initial value “0” (step S1). Note that the fixed section discrimination threshold (Δ) and the minimum length (MinimumLegLen) that is recognized as a fixed section are set to predetermined values, and the section from the utterance start position to the utterance end position for each mora. The segment length (SegmentLen), which is the time length of, is obtained from the input data 1.
Next, the search start point (StartPos) is set to “0” (step S2).
Then, it is determined whether or not the value obtained by adding the maximum constant section length to the search start point (StartPos + MaxLeglen) is smaller than the segment section length (SegmentLen) (step S3). As a point +1 (StartPos + 1) (step S4), it is determined whether or not the difference between the pitch (Pitch [i]) at the search point i and the pitch (Pitch [StartPos]) at the search start point is greater than Δ (step S4). Step S5). When the result is larger than Δ, it is determined whether or not the distance from the search start point to i (i-StartPos) is larger than the maximum constant section length (MaxLeglen) (step S6). StartPos) is the new maximum constant section length (MaxLeglen), its search start point (StartPos) is the maximum constant section start position (MaxLegStartPos) (step S7), and (i-StartPos) is the maximum constant section length (MaxLeglen) In the following cases, the inspection start point (StartPos) is updated to (StartPos + 1) as it is (step S8). Then, the process returns to step S3.

On the other hand, when the difference between the pitch (Pitch [i]) at the search point i and the pitch (Pitch [StartPos]) at the search start point (StartPos) is equal to or less than Δ in step S5, i is set to (i + 1). Update and advance the search point (step S9). If i is equal to or less than the segment section length (SegmentLen) (step S10 is NO), the process returns to step S5.
If i exceeds the segment section length (SegmentLen) (YES in step S10), whether or not the distance (i-StartPos) between i and the search start point is greater than the maximum constant section length (MaxLeglen). Is determined (step S11). As a result, if it is larger than the maximum fixed section length, (i-StartPos) is set as the new maximum fixed section length (MaxLeglen), and the search start point (StartPos) is set as the maximum fixed section start position (MaxLegStartPos) (step S12). If not, the process proceeds to step S13, and it is determined whether or not the maximum constant section length (MaxLeglen) exceeds a minimum length (MinimumLegLen) that is recognized as a constant section. When the maximum constant section length (MaxLeglen) exceeds the minimum length (MinimumLegLen), the section from the maximum constant section start position (MaxLegStartPos) to (MaxLegStartPos + MaxLeglen) is determined as a constant section (step S14). When the length is not more than the minimum length (MinimumLegLen), “no fixed interval” is set (step S15).
In this way, the constant pitch interval detection unit 2 determines the longest interval among the intervals in which the pitch variation is equal to or less than the predetermined threshold value (Δ) as the constant pitch interval.

FIG. 6 is a flowchart showing a flow of vibrato section detection processing executed in the vibrato section detecting unit 3.
First, both the maximum vibrato section length (MaxVibLen) and the maximum vibrato section start position (MaxVibStartPos) are set to the initial value “0” (step S21), and the search start point (StartPos) is set to “0” (step S22). . The minimum length (MinimumVibLen) that is recognized as a vibrato section is set to a predetermined value in advance, and the segment section length (SegmentLen) is obtained from the input data 1.
Next, it is determined whether or not the value obtained by adding the maximum vibrato section length to the search start point (StartPos + MaxViblen) is smaller than the segment section length (SegmentLen) (step S23). As the search start point + 1 (StartPos + 1) (step S24), the process proceeds to the cross point determination in step S25.

In step S25, the pitch of the search point i (Pitch [i]) is equal to or greater than the pitch of the search start point (Pitch [StartPos]) and the pitch of the next search point (i + 1) (Pitch [i + 1]). Is less than or equal to the pitch of the search start point (Pitch [StartPos]), or conversely, the pitch of the search point i (Pitch [i]) is less than or equal to the pitch of the search start point (Pitch [StartPos]), and When the pitch (Pitch [i + 1]) of the next search point (i + 1) is equal to or greater than the pitch (Pitch [StartPos]) of the search start point, the search point i is determined to be a cross point.
If the search point i is a cross point, it is determined whether the length from the search start point (StartPos) to the search point i (i-StartPos) is greater than the maximum vibrato section length (MaxViblen) (step S26) When the value is larger, the length (i-StartPos) from the search start point to the search point i is set as a new maximum vibrato section length (MaxViblen), and the search start point (StartPos) is set as the maximum vibrato section start position ( MaxVibStartPos) (step S27). Then, the search point is advanced by setting i to i + 1 (step S28).
If it is determined in step S25 that it is not a cross point and if it is determined in step S26 that it is equal to or shorter than the maximum vibrato section length, the search point is advanced as it is (step S28).

Until the position i of the search point exceeds the segment section length (SegmentLen), the processing after step S25 (determination of the cross point) is repeated (step S29). When the search section position i exceeds the segment section length, the search start point (StartPos) Is advanced by 1 (step S30), the process returns to step S23, and the processing from step S24 onward is repeated while the sum of the search start point and the maximum vibrato section length (StartPos + MaxViblen) is smaller than the segment section length.
When the sum of the search start point and the maximum vibrato section length is equal to or greater than the segment section length (YES in step S23), the process proceeds to step S31, and the obtained maximum vibrato section length (MaxViblen) is a vibrato section. It is determined whether or not it is larger than the minimum length (MinimumVibLen) that is allowed. It is determined that it is a vibrato section (step S32). Otherwise, it is determined that there is no vibrato section (step S33).
In this way, the vibrato section detection unit 3 detects a vibrato section.

As described above, the scale extraction unit 4 uses the longer interval of the constant pitch interval detected by the constant pitch interval detection unit 2 and the vibrato interval detected by the vibrato interval detection unit 3 as a reference pitch interval. And the pitch name having the frequency closest to the reference pitch is output as the scale information with the average pitch in the reference pitch section as the reference pitch. When neither the constant pitch interval nor the vibrato interval is detected, a pitch name having a frequency closest to the average pitch in the segment interval is output.
The reference pitch is not limited to the average pitch in the reference pitch section, and the lowest pitch or the highest pitch in the reference pitch section may be used as the reference pitch.

Based on the character (mora) string, segment position, pitch string, and volume string data inputted as the input data 1 and the scale information extracted by the scale extracting part 4 Create a speech synthesis script suitable for the corresponding speech synthesis system.
As described above, the speech synthesis script according to the present invention includes scale information representing the pitch of a sound, a symbol representing a speech length, and a high / low accent symbol representing a pitch increase / decrease.
The symbol representing the utterance length is, for example, “-” (long sound symbol), and this symbol represents the utterance length in a predetermined unit.
FIG. 7 is a diagram illustrating an example of high and low accent symbols. In this figure, (a) is a symbol "'" (apostrophe) representing a pitch increase, (b) is a symbol "_" (underscore) representing a pitch decrease, and (c) is a symbol representing a pitch shift down. “$” (Dollar sign).
In the example shown in FIG. 7, “′” indicates that the pitch increases by a predetermined value (P0) in a predetermined time unit T0 (for example, the same time unit as the utterance length unit by “−”), and “_” It represents that the pitch is lowered by a predetermined value (P0), and “$” can represent that the pitch is shifted downward by a predetermined value (P0). The change level (P0) is a change amount with respect to the reference pitch.

  By storing a high / low accent symbol / pitch conversion table that indicates the correspondence between such high / low accent symbols and the relative change in pitch corresponding to the high / low accent symbols, by referring to this table, the mode of relative pitch change by high / low accent symbols can be changed. You can also choose. For example, the pitch change level (P0) in the predetermined time unit T0 in FIGS. 7 (a) to 7 (c) is different, or a straight line as shown in FIGS. 7 (a) and 7 (b). In this case, a plurality of data corresponding to various pitch changes, such as those that are changed in a curved line, are stored and can be selected and used.

FIG. 8 is a diagram showing an example of a speech synthesis script created by the synthesis script creation unit 5. In this figure, (a) shows an example of a pitch sequence, and (b) shows a speech synthesis script corresponding to the pitch sequence.
When a character “a” having a pitch sequence as shown by the solid line in FIG. 8A is input, the latter half of the pitch is regarded as a vibrato section, and the scale is calculated from the average pitch of the section. The information is determined as “D # 4”.
When the high / low accent symbol / pitch conversion table to be used in the synthesizing script creation unit 5 is determined, the utterance length (T0) for one long sound symbol and the pitch change level (P0) for one accent symbol are determined. Is done.
Therefore, first, the pitch change level between the first pitch value (at t0) and the scale information (D # 4 in this case) in this sound generation unit indicated by the solid line in FIG. Difference (in this case, −2 × P0) is detected, and the accent symbol “$ 2” that shifts the pitch downward by 2 × P0 from the scale information is determined. Next, the process of determining a value in units of the change amount (P0) of the pitch closest to the input pitch after the utterance long time (T0) has elapsed, and determining the corresponding high and low accent symbols is repeated. In FIG. 8A, since the pitch is increased by 2 × P0 between times t1 and t2, the accent symbol “′ 2” and the long sound symbol “−” representing this are determined. Next, when the amount of change in pitch from time t2 to t3 is detected, it is an increase in P0, so "'-" is determined. From time t3 to t4, it decreases by 2 × P0. "Is determined. Similarly, the speech synthesis script shown in FIG. 8B is created. Note that the scale information (D # 4) is described at the beginning of the script.
In this way, the change in pitch indicated by the solid line in FIG. 8A can be expressed as a broken line.
When a volume column (volume transition information) is also input, the pitch column described above is also used for the volume column using symbols (for example, “>”, “<”, etc.) that indicate an increase or decrease in volume. As in the case of, the volume transition can be represented by a symbol.

Next, a speech synthesizer that synthesizes speech using the speech synthesis script created in this way will be described with reference to FIG. This sound source synthesizer can also be realized by program processing in a computer.
In FIG. 9, the input speech synthesis script is analyzed by the script analysis unit 21 and divided into mora (character) information, pitch information (scale, pitch accent symbol), and volume information.
The phoneme data creation unit 22 extracts phoneme data (for example, form information and waveform data of “A”) from the phoneme database 23 from the mora information.
The pitch string generator 24 uses the same pitch accent / pitch conversion table 25 as that used in the speech synthesis script generator to sequentially change the pitch according to the pitch accent to the pitch of the scale. Apply to generate a pitch train (pitch transition information). Similarly, for the volume information, the volume sequence generator 26 generates a volume sequence (volume transition information).
The speech synthesizer 27 outputs the speech with the pitch and volume added to the phoneme data extracted from the phoneme database 23 based on the pitch sequence and the volume sequence.
Here, the synthesis method may be any method as long as pitch and volume can be added, such as general formant synthesis and waveform synthesis.

Note that the high and low accent symbols in the above are not limited to those described above, and any symbol may be used as long as it is defined as a script grammar.
In the above description, the reference pitch is represented by the pitch name (scale information). However, the present invention is not limited to this and may be represented by other methods such as the pitch frequency itself.

It is a functional block diagram which shows the structure of one Embodiment of the speech synthesis script production | generation apparatus of this invention. It is a figure which shows an example of the structure for obtaining input data by audio | voice analysis. It is a figure which shows an example of input data. It is a figure for demonstrating a pitch constant area and a vibrato area, (a) shows a pitch constant area, (b) is a figure which shows a vibrato area. It is a flowchart which shows the flow of a fixed pitch area detection process. It is a flowchart which shows the flow of a vibrato area detection process. It is a figure which shows an example of a high / low accent symbol, (a) is a symbol which shows the raise of a pitch, (b) is a symbol which shows the fall of a pitch, (c) is a figure which shows the example of the symbol which shows the shift of a pitch. . It is a figure which shows an example of the script for speech synthesis, (a) is a pitch sequence, (b) is a diagram which shows the script for speech synthesis corresponding to this pitch sequence. It is a block diagram which shows the structure of the speech synthesizer which synthesize | combines speech with the created script for speech synthesis. It is a figure which shows an example of the pitch in a singing voice.

Explanation of symbols

  1: input data, 2: pitch constant interval detection unit, 3: vibrato interval detection unit, 4: scale extraction unit, 5: synthesis script creation unit, 11: segmentation unit, 12: pitch detection unit, 13: volume detection unit , 21: script analysis unit, 22: phoneme data creation unit, 23: phoneme database, 24: pitch sequence generation unit, 25: pitch accent / pitch conversion table, 26: volume sequence generation unit

Claims (5)

A speech synthesis script generation device that generates a speech synthesis script to be supplied to a speech synthesis device,
Means for inputting mora information of speech to be synthesized, segment information indicating the timing at which each mora is uttered, and pitch transition information indicating a pitch change of each mora;
A constant pitch interval detecting means for detecting a constant pitch interval in which a variation in pitch is within a predetermined threshold from the pitch transition information;
Vibrato section detecting means for detecting a vibrato section in which the pitch is oscillating from the pitch transition information;
The longer one of the constant pitch section detected by the constant pitch section detection means and the vibrato section detected by the vibrato section detection means is determined as a reference pitch section, and a reference pitch in the reference pitch section is detected. , Scale extraction means for outputting the nearest scale information of the reference pitch,
Based on the mora information, the segment information, the pitch transition information, and the scale information, the mora information, the scale information, and a voice including a high and low accent symbol that represents a pitch change amount with respect to a reference pitch in a predetermined time unit. A speech synthesis script generation device comprising: synthesis script creation means for creating a synthesis script.   The speech synthesis script generation apparatus according to claim 1, further comprising means for extracting the mora information, the segment information, and the pitch transition information from speech waveform information.   3. The voice synthesis according to claim 1, wherein the reference pitch is an average pitch in the reference pitch section, a lowest pitch in the reference pitch section, or a highest pitch in the reference pitch section. Script generator. A table representing correspondence between the height accent symbols and the pitch change modes corresponding to the height accent symbols;
The speech synthesis script generation apparatus according to claim 1, wherein the synthesis script creation unit creates the speech synthesis script with reference to the table. On the computer,
Inputting the mora information of the speech to be synthesized, the segment information indicating the timing when each mora is uttered, and the pitch transition information indicating the pitch change of each mora;
A constant pitch interval detecting step for detecting a constant pitch interval in which a variation in pitch is within a predetermined threshold from the pitch transition information;
A vibrato section detecting step for detecting a vibrato section in which the pitch is oscillating from the pitch transition information;
The longer one of the constant pitch interval detected by the constant pitch interval detection step and the vibrato interval detected by the vibrato interval detection step is verified as a reference pitch interval, and a reference pitch in the reference pitch interval is detected. A scale extraction step for outputting the nearest scale information of the reference pitch;
Based on the mora information, the segment information, the pitch transition information, and the scale information, the mora information, the scale information, and a voice including a high and low accent symbol that represents a pitch change amount with respect to a reference pitch in a predetermined time unit. A speech synthesis script generation program that executes a synthesis script creation step for creating a synthesis script.

Images