JP5712818B2 - Speech synthesis apparatus, sound quality correction method and program - Google Patents

Description

  The present invention relates to a speech synthesizer and a sound quality correction method.

  There is speech synthesis technology that analyzes text input by a user, converts it into synthesized speech, and reads it out. For example, the input text is analyzed to create the target prosody, and the synthesized speech is generated by selecting and connecting the appropriate waveform according to the target prosody from the waveform dictionary that describes the information for assembling the output speech Speech synthesis technology is known. When there is no desired waveform that can be selected, it aims to output a natural sound by performing modification to match the purpose of use or by generating a new waveform.

  In order to improve the quality of synthesized speech, there is known a method for predicting the quality of synthesized speech, and presenting to the user phonemes whose quality is determined to be out of the set value range by the predicting means. ing. Further, a primary selection unit that selects a plurality of synthesis unit candidates using a predetermined evaluation criterion, and a secondary selection unit that selects a segment from the selected plurality of candidates so that the distortion of the segment connection is reduced. There is also an example in which single selection is performed. In this example, if it is determined that a certain quality cannot be obtained at the time of synthesis, an alternative segment process is performed.

  In addition, whether or not the prosody of the synthesized speech is natural is determined for each of a plurality of parameters such as speech speed and intonation, and signal processing or speech unit sequence search may be performed again according to the determination result.

  On the other hand, a method is also known in which a user listens to synthesized speech, inputs a portion with poor quality, and corrects the sound quality by re-synthesizing by giving a penalty to a phoneme piece corresponding to the portion with poor quality.

JP-A-1-284898 JP 2006-313176 A JP-A-8-263095 Japanese Patent No. 3423276 JP 2008-139931 A

  The current quality of synthesized speech as described above is not sufficient compared to human speech. For example, since there is a quality degradation of synthesized speech in some places, there are many requests that users want to easily correct the quality degradation.

  However, in the example in which the prediction unit presents a phoneme whose quality is out of the set range value, the presented sound quality correction position may deviate from the user's sense, and the sound quality is corrected according to the user's sense. There are things that cannot be done. In the example of reselecting phonemes to improve the quality of synthesized speech, simply re-synthesize the selected phonemes etc., and the newly selected phoneme or phoneme string will be the sound quality desired by the user. The sound quality may not be corrected.

  On the other hand, in an example where the user points out a poor quality part, it is difficult for the user to specify the sound quality correction part accurately, and when it is specified vaguely, the sound quality correction of the part that the user does not want is performed. There is a case.

  In view of the above problems, the present invention provides a speech synthesizer, a sound quality correction method, and a program capable of correcting synthesized speech so as to match the user's subjectivity.

  A speech synthesizer according to one aspect includes a synthesized speech acquisition unit, a range acquisition unit, a degradation cost calculation unit, a degradation type determination unit, a degradation type acquisition unit, a correction information generation unit, and a speech synthesis unit. Yes. The synthesized voice acquisition unit acquires the synthesized voice. The range acquisition unit acquires a first range for correcting the voice. The deterioration cost calculation unit calculates at least one deterioration cost which is information corresponding to sound quality deterioration in the first range. The deterioration type determination unit determines at least one correction candidate deterioration type corresponding to the nature of the sound quality deterioration, which is a candidate for correction in the first range, based on the at least one deterioration cost. The deterioration type acquisition unit acquires a correction deterioration type selected for correction from the determined correction candidate deterioration types. The correction information generation unit generates correction information for correcting the sound based on the correction deterioration type. The speech synthesizer re-synthesizes the speech based on the correction information.

  According to another aspect of the present invention, there is provided a speech correction method that acquires synthesized speech, acquires a first range for performing correction on the speech, and at least one deterioration that is information corresponding to sound quality deterioration in the first range. Calculate the cost. Further, in the sound correction method, at least one correction candidate deterioration type corresponding to the property of the sound quality deterioration that is a candidate for correction in the first range is determined based on at least one of the deterioration costs. Further, the voice correction method acquires a correction deterioration type selected for correction from the determined correction candidate deterioration types, and generates correction information for correcting the sound based on the correction deterioration type The speech is re-synthesized based on the correction information.

  In addition, even if it is a program for causing a computer to perform the method according to the present invention described above, since the program is executed by the computer, the same operations and effects as the method according to the present invention described above are achieved. The aforementioned problems are solved.

  According to the speech synthesizer, the speech correction method, and the program according to the aspect described above, it is possible to correct the synthesized speech so as to match the user's subjectivity.

It is a block diagram which shows the structure of the speech synthesizer by 1st Embodiment. It is a block diagram which shows the function of the speech synthesizer by 1st Embodiment. It is a figure which shows the segment selection penalty by 1st Embodiment. It is a figure which shows the segment connection penalty by 1st Embodiment. It is a figure explaining the "phoneme environment" by 1st Embodiment. It is a figure which shows the degradation cost function and degradation type by 1st Embodiment. It is a flowchart which shows the operation | movement of the degradation classification display in the speech synthesizer by 1st Embodiment. It is a figure which shows the example of a display of the phonetic text and continuous phoneme string by 1st Embodiment. It is a flowchart which shows the operation | movement of degradation type determination by 1st Embodiment. It is a figure which shows the example of the degradation position by 1st Embodiment. It is a flowchart which shows the operation | movement of deterioration classification selection by 1st Embodiment. It is a flowchart which shows the operation | movement of deterioration classification determination by the modification of 1st Embodiment. It is a block diagram which shows the function of the speech synthesizer by 2nd Embodiment. It is a flowchart which shows the operation | movement of designation | designated of the degradation range by 2nd Embodiment. It is a flowchart which shows the operation | movement of a deterioration classification designation | designated by 2nd Embodiment. It is a block diagram which shows an example of the hardware constitutions of a standard computer.

(First embodiment)
The speech synthesizer according to the first embodiment will be described below with reference to the drawings. In this specification, the phoneme is a wording corresponding to each of vowels, repellent sounds, prompt sounds, and consonants. A mora is a word corresponding to one sound of a Japanese utterance, such as a vowel, repelling sound, prompting sound, or “consonant + vowel”.

  First, the configuration of the speech synthesizer 1 according to the first embodiment will be described with reference to FIGS. 1 and 2. FIG. 1 is a block diagram showing a configuration of a speech synthesizer according to the first embodiment, and FIG. 2 is a block diagram showing functions of the speech synthesizer according to the first embodiment.

  As shown in FIG. 1, the speech synthesizer 1 includes an input unit 3, a storage unit 5, a speech synthesizer 7, a sound quality correction determination unit 9, and an output unit 11, which are connected to each other via a bus 13. . When the input unit 3 is operated by a user of the speech synthesizer 1, the input unit 3 acquires input of various information from the user associated with the operation content, and the acquired input information is input to the speech synthesizer 7 or the sound quality. Devices that are sent to the correction determination unit 9 are a keyboard device, a mouse device, a touch panel device, and the like.

  The storage unit 5 is, for example, a readable semiconductor storage device, a semiconductor storage device capable of writing / reading as needed, and a portable storage medium capable of writing / reading as needed. The storage unit 5 stores a program for controlling the basic operation of the speech synthesizer 1, information necessary for executing speech synthesis described later, and is used as a work area as needed in speech synthesis processing. This is a storage device.

  For example, the speech synthesizer 7 generates a target prosody for speech synthesis according to the phonetic text input by the user via the input unit 3, selects an appropriate speech unit according to the target prosody, and selects the selected speech Selection connection information is generated by connecting the pieces to each other. Furthermore, the speech synthesizer 7 performs waveform processing based on the generated selected connection information to generate synthesized speech.

  The sound quality correction determination unit 9 acquires the synthesized speech generated by the speech synthesizer 7, acquires a sound quality deterioration range that feels the sound quality deterioration input by the user, and the sound quality deterioration range is degraded according to the nature of the sound quality deterioration. Present the type to the user. Examples of the degradation type include “sound (sound quality) is bad”, “inflation is bad”, “smooth tongue is bad”, and the like. When the user selects a correction deterioration type to be corrected from the presented deterioration types, the sound quality correction determination unit 9 acquires the selected correction deterioration type and provides correction information for performing correction according to the correction deterioration type. Output.

  The output unit 11 is a speaker that outputs the voice synthesized by the voice synthesis unit 7 and the voice recombined based on the correction information output from the sound quality correction determination unit 9, and the deterioration determined by the sound quality correction determination unit 9. It is a display device or the like that displays a type or the like. The bus 13 is a communication path for connecting the above devices and the like to exchange data.

  Next, the configuration of the speech synthesizer 1 will be described in more detail with reference to FIG. As shown in FIG. 2, the speech synthesis unit 7 has functions of a language processing unit 23, prosody generation unit 25, segment selection connection unit 27, and waveform processing unit 29. The storage unit 5 stores a language dictionary 31, a prosody dictionary 33, and a waveform dictionary 35. The sound quality correction determination unit 9 has functions of a range acquisition unit 41, a deterioration cost calculation unit 43, a deterioration type determination unit 45, a deterioration position acquisition unit 47, and a correction means determination unit 49. The sound quality correction user interface (UI) unit 21 represents the interface function of the input unit 3 and the output unit 11 as a representative.

  Note that the speech synthesizer 7 and the sound quality correction determination unit 9 store programs for realizing the respective functions in advance in a nonvolatile storage device, for example, and read and execute these programs by an arithmetic device (not shown). It can be set as the function implement | achieved by doing.

  The sound quality correction UI unit 21 is an interface device for inputting / outputting information between the user and the speech synthesizer 1, and receives an input text for speech synthesis from the user and a phonetic sound. The text and synthesized speech waveform are presented on a screen of a display device (not shown). Also, the sound quality modification UI unit 21 outputs the synthesized speech as speech, and when the user feels sound quality degradation in the synthesized speech, the sound quality degradation felt to be specified as specified in the speech synthesis waveform or phonetic text. Get the range. Furthermore, the sound quality degradation type determined within the area designated by the user is presented to the user via a pop-up menu or the like, and the user can select the degradation type that matches the sound quality degradation content felt by the user. Configure as follows.

  The language processing unit 23 of the speech synthesizing unit 7 analyzes the part of speech of the input text input via the sound quality correcting UI unit 21 with reference to the language dictionary 31 stored in the storage unit 5, Generate text. The input text is, for example, a sentence to be subjected to speech synthesis, such as “Today's weather is sunny”. The phonetic text is a text string in which the input text is expressed in katakana and the accent position is expressed in symbols, and is a character string such as “Kyo'no / Tenkiha / Hare'Death.”. Here, “′” indicates an accent position, “/” indicates an accent phrase delimiter, and “.” Indicates a sentence end (punctuation point). An accent phrase means a segment boundary unit constituting one accent or a finer unit. The above sentence example is an example in which three accent phrases are formed, and each accent phrase matches a phrase boundary unit. The language dictionary 31 is information that records word readings, parts of speech, grammar, and accent information.

  The prosody generation unit 25 further converts the generated phonetic text into phonemes, and generates a target prosody as a target of speech synthesis while referring to the prosody dictionary 33. Here, the prosody is information representing, for example, the length, height, magnitude, and the like of a sound by changes in frequency and amplitude with respect to the time of the sound. In the prosody dictionary 33, prosody information obtained by statistically processing probabilistic information extracted from voluminous voice data such as narrators and announcers is extracted from variations of rhythms composed of various dynamics, lengths, and shorts. The target prosody is a target prosody when generating synthesized speech from the input text. Based on the above-mentioned phonetic text, an appropriate sound strength generated from the prosody information stored in the prosody dictionary 33, It is a long, short, high and low rhythm.

  The segment selection / connection unit 27 selects an appropriate speech segment from the waveform dictionary 35 according to the generated target prosody and generates selection connection information for connection to each other. The waveform dictionary 35 stores recorded narrator speech waveforms and various phoneme information (phoneme names corresponding to speech forms, phoneme lengths, pitch frequencies, etc.). Hereinafter, the term “segment” refers to a phoneme string (one or more continuous phonemes) selected by segment selection or a group of mora strings (one or more consecutive mora). Note that the segment can be either one phoneme or N phoneme, depending on the selection.

  The segment selection connection unit 27 selects a segment as close as possible to the target prosody generated by the prosody generation unit 25 from the waveform dictionary 35 (segment selection). Select a segment whose connection boundary is a natural prosody (segment connection). When selecting an element, an element selection penalty and an element connection penalty, which will be described later, are calculated, and an element selection penalty evaluation value PP and an element connection penalty evaluation value PC calculated using them are set to the smallest possible values. Make a selection.

  The segment selection / connection unit 27 outputs the selected connection information, the calculated segment selection penalty, and the segment connection penalty to the range acquisition unit 41 as intermediate information. The segment selection / connection unit 27 receives correction information from the correction unit determination unit 49 (to be described later), performs segment selection again for speech resynthesis, and regenerates the segment connection information.

  The waveform processing unit 29 reads out the segment waveforms selected from the waveform dictionary 35 based on the selection connection information generated by the segment selection connection unit 27 and connects them by waveform processing to generate synthesized speech.

  The range acquisition unit 41 of the sound quality correction determination unit 9 uses the sound quality deterioration range specified by the user and the intermediate information generated by the language processing unit 23, the prosody generation unit 25, and the segment selection connection unit 27 of the speech synthesis unit 7. Based on this, a range for sound quality correction is acquired. That is, the range acquisition unit 41 acquires a synthesized speech range (hereinafter referred to as a target phoneme range) corresponding to the sound quality degradation range designated by the user via the sound quality correction UI unit 21. At this time, the range acquisition unit 41 determines any segment of the phoneme unit, the phoneme sequence unit, the mora unit, and the mora sequence unit, which completely includes the sound quality degradation range, as a resynthesis range in which speech synthesis is performed. You may comprise as follows.

  The deterioration cost calculation unit 43 calculates each deterioration cost in the target phoneme range corresponding to the sound quality deterioration range. The degradation cost and the degradation cost function are defined by at least one combination including a segment selection penalty, which is one piece of intermediate information of the speech synthesizer 7, or a segment connection penalty. The deterioration cost calculation unit 43 calculates the deterioration cost from the deterioration cost function. The deterioration cost is preferably calculated for each phoneme or mora in the above-described sound quality deterioration range, but may be for each synthesized phoneme unit (phoneme string unit, mora string unit). Hereinafter, the phoneme unit, the mora unit, the phoneme string unit, and the mora string unit are collectively referred to as a segment unit. Details of the deterioration cost will be described later.

  Note that the segment selection penalty is the difference between the target prosody (pitch, phoneme length, etc.) and acoustic feature (tone, etc.) from the prosody or acoustic feature of the speech unit selected from the waveform dictionary 35. Is quantified and defined as a penalty. The segment connection penalty is defined as a penalty by quantifying the difference in prosody and acoustic feature values before and after the boundary when connecting selected segments. The greater the difference between the segment selection penalty and the segment connection penalty, the farther the target prosody or acoustic feature value is, the more the sound quality deteriorates. By defining each penalty combination as a component of the deterioration cost function, the deterioration cost is suitable for determining the deterioration type, and the determination accuracy of the deterioration type can be improved. Details of the segment selection penalty and the segment connection penalty will be described later.

  The deterioration type determination unit 45 determines the sound quality deterioration type based on the calculated deterioration cost. The deterioration type determination unit 45 classifies the deterioration type into sound quality deterioration characteristics such as “sound (sound quality), intonation, smooth tongue” and the like. Each deterioration type is classified based on at least one deterioration cost. Among the deterioration costs, the deterioration type determination unit 45 selects a deterioration type corresponding to a deterioration cost that is higher than a predetermined order from the highest value or a deterioration type that exceeds a predetermined threshold value. The degradation type is determined. Hereinafter, the deterioration type of the sound quality correction candidate is referred to as a correction candidate deterioration type. Hereinafter, a degradation cost having a degradation cost value of a predetermined order from the highest value is referred to as a higher degradation cost. The deterioration type determination unit 45 presents the determined correction candidate deterioration type to the user via the sound quality correction UI unit 21.

  As described above, the deterioration type determination unit 45 is configured to determine a deterioration type including at least one higher deterioration cost among the deterioration costs as the deterioration type of the sound quality correction candidate. This is because a high deterioration cost is likely to cause sound quality deterioration, and a deterioration type including a higher deterioration cost is likely to be a cause of sound quality deterioration.

  Further, the deterioration type determination unit 45 may present all deterioration types to the user via the sound quality correction UI unit 21 regardless of the presence or absence of the sound quality correction candidate. This is to prevent the user from losing the means for correcting the sound quality when the deterioration type of the sound quality deterioration felt by the user is not an option. When a deterioration type that is not a sound quality correction candidate is selected by the user, for example, it is preferable that a manual correction UI (not shown) that is optimal for the correction of the deterioration type is activated.

  The deterioration position acquisition unit 47 acquires and holds the deterioration cost value of the deterioration type and its deterioration position for the correction candidate deterioration type determined as the sound quality correction candidate. On the sound quality correction UI unit 21, the user selects a deterioration type that suits his / her feeling from the presented correction candidate deterioration types. The deterioration type selected by the user and acquired by the speech synthesizer 1 is hereinafter referred to as a corrected deterioration type.

  However, the deterioration position acquisition unit 47 may be configured to acquire and hold deterioration costs and deterioration positions of all deterioration types regardless of whether or not the deterioration type is a sound quality correction candidate. Even if it cannot be determined that the sound quality correction candidate is a deterioration type, when the user selects a deterioration type that feels sound quality deterioration, the deterioration type and the corresponding deterioration cost are set as sound quality deterioration correction candidates. Can be modified.

  The correction means determination unit 49 acquires the correction deterioration type selected by the user via the sound quality correction UI unit 21, acquires the deterioration cost and the deterioration position corresponding to the deterioration type from the deterioration position acquisition unit 47, and the range acquisition unit From 41, a re-synthesis range for re-synthesizing speech is acquired. In order to improve sound quality deterioration based on the re-synthesis range and each deterioration cost corresponding to the correction deterioration type, the correction means determination unit 49 improves the element selection penalty and the element connection penalty corresponding to the deterioration cost. Is generated and output to the segment selection / connection unit 27. The correction information includes a segment selection penalty evaluation value PP and a segment connection penalty evaluation value PC, which will be described later.

  With the above configuration, the speech synthesizer 1 determines the sound quality deterioration position and the sound quality deterioration type from the sound quality deterioration range designated by the user, and presents the sound quality deterioration type to the user. The user selects a degradation type that suits his / her sense from the presented degradation types, and the speech synthesizer 1 performs sound quality correction according to the degradation type.

  Hereinafter, the element selection penalty, the element selection penalty evaluation value PP, the element connection penalty, and the element connection penalty evaluation value PC will be described with reference to FIGS. FIG. 3 is a diagram showing a segment selection penalty. As shown in FIG. 3, the segment selection penalty is a quantification of the difference between the target prosody of the synthesized speech (pitch frequency, phoneme length, acoustic feature amount, etc.) and the prosody of the segment selected from the waveform dictionary 35. By giving it as a penalty, it is a variable for selecting a segment closer to the target prosody and suppressing the sound quality degradation of the synthesized speech. The penalty means a reference value corresponding to the degree of voice deterioration.

  As shown in Table 75 of FIG. 3, examples of the segment selection penalty include pitch penalties PPaa and PPab, phoneme length penalties PPba and PPbb, and timbre penalties PPca. Pitch penalties PPaa and PPab are penalties regarding the pitch frequency. The pitch penalty PPaa is equal to or higher than the pitch frequency of the segment selected from the waveform dictionary 35 (hereinafter referred to as the waveform dictionary segment pitch frequency, hereinafter the same) in the target prosody. Is the case. The phoneme length penalty PPbb is a case where the target pitch frequency is less than the waveform dictionary segment pitch frequency.

  Phoneme length penalties PPba and PPbb are penalties for phoneme length. The phoneme length penalty PPba is the case where the target phoneme length is greater than or equal to the waveform dictionary segment phoneme length, and the phoneme length penalty PPbb is the case where the target phoneme length is less than the waveform dictionary segment phoneme length. The timbre penalty PPca is a penalty related to the timbre (acoustic feature amount). The timbre penalty PPca is when the target timbre is significantly different from the waveform dictionary fragment timbre.

An example of the definition of the segment selection penalty is shown below.
Pitch penalty PPaa = Σ (target pitch frequency / waveform dictionary segment pitch frequency) (formula A1-1)
In Expression A1-1, Σ represents the sum of pitch ratios when the target phoneme range is a single phoneme.
Pitch penalty PPaa = Σ (Σ (target pitch frequency / waveform dictionary segment pitch frequency)) (formula B1-1)
In Expression B1-1, the first Σ represents the sum of all phonemes in the target phoneme range, and the second Σ represents the sum of pitch ratios of the phonemes in the target phoneme range. In addition, in Formula A1-1 and Formula B1-1, it adds together about target pitch frequency> waveform dictionary fragment pitch frequency.

Pitch penalty PPab = Σ (waveform dictionary segment pitch frequency / target pitch frequency) (Formula A1-2)
Pitch penalty PPab = Σ (Σ (waveform dictionary segment pitch frequency / target pitch frequency)) (Formula B1-2)
Here, regarding Σ, Formula A1-2 is the same as Formula A1-1, and Formula B1-2 is the same as Formula B1-1. In addition, in Formula A1-2 and Formula B1-2, it adds together about target pitch frequency <waveform dictionary fragment | piece pitch frequency.

  A high pitch penalty PPaa indicates a strong tendency of “target pitch frequency ≧ waveform dictionary segment pitch frequency” and a strong tendency that the synthesized voice is lower than the target voice level. Moreover, if the pitch penalty PPab is high, it indicates that the tendency “target pitch frequency <waveform dictionary segment pitch frequency” is strong, and the tendency that the synthesized voice is higher than the target voice is strong. Thus, for both pitch penalties PPaa and PPab, when the value is high, there is a strong tendency to produce synthesized speech with unnatural or accented sounds.

  As described above, (Equation A1-1) and (Equation A1-2) are equations for accumulating the pitch penalty in units of phonemes and calculating the pitch penalty in units of phonemes. (Equation B1-1) and (Equation B1-2) are equations for accumulating the pitch penalties of a plurality of phonemes and calculating a pitch penalty for each phoneme string. Both of these formulas are penalties that reflect the inflection difference between the target and the waveform dictionary segment, and are clues to know the unnaturalness of the inflection. It is preferable that the pitch penalties PPaa and PPab in the calculated phonemes or phoneme strings are all stored in, for example, the storage unit 5 and can be read out as necessary.

  Further, the pitch penalties PPaa and PPab may be calculated in units of mora instead of being calculated in units of phonemes, and may be calculated in units of mora sequences instead of being calculated in units of phoneme sequences.

The “phoneme length penalty” is obtained by quantifying the difference between the target phoneme length and the phoneme length of the waveform dictionary as a penalty, and is defined by the following equation, for example.
Phoneme length penalty PPba = target phoneme length / waveform dictionary phoneme length (formula A2-1)
Phoneme length penalty PPba = Σ (target phoneme length / waveform dictionary phoneme length) (formula B2-1)
Σ indicates the sum of phoneme length ratios of each phoneme in the target phoneme range. Here, the sum is calculated for the case where target phoneme length ≧ waveform dictionary phoneme length.
Phoneme length penalty PPbb = Waveform dictionary phoneme length / target phoneme length (formula A2-2)
Phoneme length penalty PPbb = Σ (waveform dictionary phoneme length / target phoneme length) (formula B2-2)
Σ indicates the sum of phoneme length ratios of each phoneme in the target phoneme range. Here, the sum is calculated for the case where target phoneme length ≦ waveform dictionary phoneme length.

  Here, if the phoneme length penalty PPba is high, it indicates that the tendency “target phoneme length ≧ waveform dictionary phoneme length” is strong, and the tendency that the synthesized speech is earlier (unsatisfactory) than the target is strong. A high phoneme length penalty PPbb indicates a strong tendency of “target phoneme length <waveform dictionary phoneme length”, and a strong tendency to be synthesized speech that is slower than the target. In either case, the smooth tongue tends to be an unnatural synthetic voice.

  Note that (Equation A2-1) and (Equation A2-2) integrate phoneme length penalties in units of phonemes and calculate phoneme length penalties in units of phonemes. (Equation B2-2) (Equation B2-2) integrates phoneme length penalties of a plurality of phonemes, and calculates a phoneme length penalty for each phoneme string. Both of these formulas are penalties that reflect the smooth tongue of the target and the waveform dictionary fragment, and are clues to know the unnaturalness of the smooth tongue. It is preferable that all the phoneme length penalties PPba and PPbb in the calculated phonemes or phoneme strings are held in, for example, the storage unit 5 and can be read out as necessary.

  Further, the phoneme length penalties PPba and PPbb may be calculated in units of mora instead of being calculated in units of phonemes, or may be calculated in units of mora sequences instead of being calculated in units of phoneme sequences.

The “tone color penalty” is obtained by quantifying the difference between the target tone color and the tone color of the waveform dictionary phoneme as a penalty, and may be defined by the following equation, for example.
Tone penalty PPca = sqrt (Σ (target MFCC (n) −waveform dictionary MFCC (n)) ² ) (Formula A3-1)
In Expression A3-1, Σ represents the mean square sum of phoneme timbres when the target phoneme range is a single phoneme.

Tone penalty PPca = Σ (sqrt (Σ (target MFCC (n) −waveform dictionary MFCC (n)) ² )) (formula B3-1)
In Formula B3-1, the first Σ represents the sum of all phonemes in the target phoneme range, and the second Σ is the mean square sum of each phoneme tone color when the target phoneme range is a single phoneme. Indicates.

  In the timbre penalty PPca, a Mel Frequency Cepstrum Coefficient (MFCC) is used as an acoustic feature amount expressing a timbre. Expressions A3-1 and B3-1 are examples in which N-dimensional MFCC is adopted as a timbre. For the target MFCC, an average MFCC may be calculated in advance for each purpose and for each phoneme, and an N-order square sum of the target MFCC and the target phoneme MFCC may be defined as a timbre penalty. Note that “by purpose” means, for example, normal speech synthesis and speech synthesis with different purposes such as emotional speech synthesis expressing emotions (joy, anger, sorrow, comfort, etc.). For example, normal voice synthesis and emotional voice synthesis have different tone colors even for the same phoneme, and it is preferable to calculate an average MFCC for each phoneme for each purpose.

  Here, if the timbre penalty PPca is a high value, the waveform dictionary phoneme and the target timbre tend to be different from each other, and thus there is a strong tendency to be unnatural sound (sound quality). In Formula A3-1, the timbre penalties are integrated in units of phonemes, and the timbre penalty in units of one phoneme is calculated. Formula B3-1 integrates the timbre penalties of a plurality of phonemes and calculates the timbre penalty for each phoneme string. Both of these formulas are penalties that reflect the timbre of the target and the waveform dictionary fragment, and are clues to know the unnaturalness of the sound quality. It is preferable that all the timbre penalties PPca in the calculated phonemes or phoneme strings are held in, for example, the storage unit 5 and can be read out as necessary. The timbre penalty PPca may be calculated in units of mora instead of being calculated in units of phonemes, and may be calculated in units of mora sequences instead of being calculated in units of phoneme sequences.

  Next, the unit connection penalty will be described with reference to FIG. FIG. 4 is a diagram showing a unit connection penalty. The unit connection penalty is the connection boundary by quantifying the difference in acoustic characteristics at the connection boundary when connecting the unit sequences (phoneme sequences) selected by the unit selection penalty. This is a variable for selecting a connection that matches the acoustic feature amount and suppressing deterioration in the quality of the synthesized speech.

  The “pitch change penalty” is obtained by quantifying the difference in pitch frequency before and after the connection boundary as a penalty when connecting segment strings (phoneme strings) selected from the waveform dictionary. .

Pitch change penalty PCaa = Pitch frequency before boundary / Pitch frequency after boundary (formula A4-1)
However, the cases where the pitch frequency before boundary ≧ the pitch frequency after boundary are added up.

Pitch change penalty PCab = post-boundary pitch frequency / pre-boundary pitch frequency (formula A4-2)
However, the case where the pitch frequency before boundary ≦ the pitch frequency after boundary is added up.

  Here, the higher the value of the pitch change penalty PCaa, the stronger the tendency of “pre-boundary pitch frequency ≧ post-boundary pitch frequency”, and the synthesized voice in which the voice pitch suddenly decreases from before the boundary to after the boundary. Become. The higher the value of the pitch change penalty PCab, the stronger the tendency of “pre-boundary pitch frequency ≦ post-boundary pitch frequency”, and the synthesized speech becomes higher in voice from before the boundary to after the boundary. In either case, there is a strong tendency for the synthesized speech to have unnatural or accented sounds.

The “phoneme environment penalty” is obtained by quantifying the difference in phoneme environment before and after the connection boundary when connecting segment strings (phoneme strings) selected from the waveform dictionary, and is defined by the following equation, for example.
Phoneme environment penalty PCba = Pre-boundary phoneme environment score (formula A5-1)
Phoneme environment penalty PCbb = phoneme environment score after boundary (formula A5-2)

  Hereinafter, the phoneme environment penalty PCba, PCbb will be described with reference to FIG. FIG. 5 is a diagram for explaining the “phoneme environment”. As shown in FIG. 5, for example, the pre-boundary phoneme string 350 before the connection boundary 362 is “庵”, the phonetic text 352 is “Iori”, the synthesized phoneme string 354 is “house”, and the phonetic text 356 is It is assumed that “yes”, the post-boundary phoneme string 358 is “up”, and the phonetic text 362 is “we”.

  In the above example, the first phoneme “I” of the phoneme sequence “庵” and the end phoneme “e” of the phoneme sequence “up” are connected together to synthesize “house”. The sound quality tends to deteriorate. That is, the utterance “yes” is usually divided into an utterance portion “i”, an utterance portion “e”, and an utterance portion called “crossover” that changes from “i” to “e”. However, “庵” is “I” that changes from “I to O”, and “Up” is “E” that changes from “U to D”. ”Does not match at all, and there is a strong tendency that the timbre of the transition part becomes unnatural and the sound quality deteriorates.

  Therefore, in boundary connection, at least two phonemes before and after the boundary are important (phoneme environment match), and if they do not match, the sound quality is likely to deteriorate. Therefore, if they do not match, the phoneme environment score is adopted as a penalty. Therefore, the phoneme environment score is defined as score 0 when the synthesized phoneme and the phoneme before or after the boundary match, and in the case of mismatch, a positive score is added according to the type of the synthesized phoneme and the waveform dictionary phoneme. Configure as follows. Therefore, as described above, the higher the value of the phoneme environment penalty PCba, PCbb, the stronger the tendency to become unnatural sound quality synthesized speech at the connection boundary.

The “timbre change penalty” is obtained by quantifying a difference in timbre before and after the connection boundary as a penalty when connecting segment strings (phoneme strings) selected from the waveform dictionary 35, and is defined by the following equation, for example.
Tone change penalty PCca = sqrt (Σ (pre-boundary phoneme MFCC (n) −post-boundary phoneme MFCC (n)) ² ) (Formula A6-1)
The mel frequency cepstrum coefficient (MFCC) described in the above-mentioned “timbre penalty” is used as the acoustic feature amount representing the timbre, and the timbre difference between the pre-boundary phoneme and the post-boundary phoneme is represented as a timbre change penalty.

  For example, when synthesizing the long vowel `` A '' by connecting it at the phoneme boundary between `` A '' and `` A '', there are various timbres `` A ''. The sound will be unnatural. That is, the higher the value of the timbre change penalty PCca is, the more the synthesized speech becomes unnatural sound (sound quality) at the connection boundary.

  In addition, as in the case where the synthesized speech is “Yes”, when using segments with different front and rear phonemes, the transition part in the latter half of “I” and the transition part in the first half of “D” If the timbre change penalty PCca according to Formula A6-1 is used for the mel frequency cepstrum, a penalty can be given according to the timbre difference in the transition portion even between different phonemes.

The formula A6-1 is an example in which N-dimensional MFCC is adopted as a timbre. For the target MFCC, an average MFCC may be calculated in advance for each purpose and for each phoneme, and an N-order square sum of the target MFCC and the target phoneme MFCC may be defined as a timbre penalty.
It is preferable that each calculated value of the unit connection penalty is stored in, for example, the storage unit 5 and can be read out as necessary.

The element selection / connection unit 27 calculates the following element selection penalty evaluation value PP and element connection penalty evaluation value PC, which are calculated based on the calculated element selection penalty and the element connection penalty, so that the following element selection penalty evaluation value PC is minimized. Select and connect pieces.
Segment selection penalty evaluation value PP = α1 × PPaa + α2 × PPab + α3 × PPba + α4 × PPbb + α5 × PPca (Formula C1)
Element connection penalty evaluation value PC = β1 × PCaa + β2 × PCab + β3 × PCba + β4 × PCbb + β5 × PCca (Formula C2)
Here, αl, βl (l = 1, 2,...) Are weighting coefficients corresponding to the degree of influence of the calculated segment selection penalty or segment connection penalty on speech synthesis.

  Next, referring to FIG. 6, explanations of “correspondence between sound quality deterioration type and deterioration cost” and “deterioration cost function” will be given below. FIG. 6 is a diagram illustrating a deterioration cost function and a deterioration type. Note that the coefficients w10 to w67 in FIG. 6 are normalization coefficients for comparing the degradation costs constituted by the degradation cost function in the same range, and W00 to W27 are the same for each degradation type constituted by the degradation costs. This is a normalization coefficient for comparison by range.

  First, the deterioration type “sound (sound quality)” (hereinafter simply referred to as “sound”) will be described. General users who are not familiar with speech synthesis often cannot classify sound quality degradation, and inferior unnaturalness, smooth tongue unnaturalness, false accents, etc. are all regarded as sound quality degradation that cannot be distinguished. There are many cases. For this reason, in the deterioration type “sound”, it is preferable to define a degradation cost including all of the segment selection penalty and the segment connection penalty, and its degradation cost function.

For example, as shown in FIG. 6, when “sound” is a deterioration type of sound quality deterioration, the deterioration costs C00 to C05 are calculated by defining them as follows.
C00 = W00 × (w10 × PPaa + w11 × PPab) (1)
In this case, the identified cause of deterioration is “pitch difference”.
C01 = W01 × (w20 × PPba + w21 × PPbb) Equation (2-1)
In this case, the identified cause of deterioration is “phoneme length difference”.
C02 = W02 × (w30 × PPca) (formula (3-1))
In this case, the identified deterioration cause is “phoneme tone color difference”.
C03 = W03 × (w40 × PCaa + w41 × PCab) Formula (4-1)
In this case, the identified cause of deterioration is “boundary pitch difference”.
C04 = W04 × (w50 × PCba + w51 × PCbb) Equation (5-1)
In this case, the identified cause of deterioration is “phoneme environment difference”.
C05 = W05 × (w60 × PCca) Expression (6-1)
In this case, the identified cause of deterioration is “boundary tone color difference”.

  Here, as mentioned above, general users who are not familiar with speech synthesis often cannot classify sound quality degradation, and are unable to distinguish between unnatural inflection, unnatural smooth tongue, and false accents. Often perceived as possible sound quality degradation. For this reason, when the user selects the deterioration type “sound”, the deterioration type is automatically set to “inflection” or “smooth tongue” other than “sound” according to various cost values from the deterioration cost C00 to C05. It is preferable to be configured to correct. For this reason, a correction method for the degradation type may be designated in advance according to various cost values from the degradation costs C00 to C05.

  Furthermore, false accents exist as one of unnatural sounds. Since it is difficult to automatically correct an erroneous accent, it is preferable that the user is forced to transit to a manual accent correction UI when “false accent” is selected as the deterioration type detail by the user.

Next, “intonation” in the degradation type column 82 will be described. The sound quality degradation of intonation is mainly due to the following two types.
Intonation example 1) When the voice of the synthesized speech is too high or too low compared to the target, or when they are mixed Intonation example 2) When the timbre of the connection boundary suddenly changes unnaturally

  Intonation example 1) is expressed as a case where at least one of the pitch penalties PPaa and PPab in the segment selection penalty is large. In other words, since the synthesized speech is at a pitch higher than the target, the intonation is unnatural. Further, it is also expressed as a case where the pitch change penalty PCaa of the unit connection penalty is large. In other words, since the pitch changes greatly before and after the connection boundary, the intonation becomes unnatural.

  Intonation example 2) appears when the timbre change penalty PCca of the segment connection penalty is large. That is, since the timbre before and after the boundary changes unnaturally, it tends to feel as if the inflection has changed abruptly on hearing.

  For the above reason, using the “pitch penalty PPaa, PPab” of the segment selection penalty and the “pitch change penalty PCaa” and “timbre change penalty PCca” of the segment connection penalty, as shown in FIG. Define degradation cost and degradation cost function.

First, as shown in the type detail column 84 of FIG. 6, for example, when the deterioration type details of the sound quality deterioration are felt as “high intonation”, it is a case where the voice is synthesized with a voice higher than the target and is related to the pitch penalty PPab. To do. Therefore, the degradation cost C10 and its degradation function are defined as follows as a function of the pitch penalty PPab as shown in the equation (1-2) in FIG.
C10 = W10 × (w12 × PPab) (1-2)
The cause of deterioration determined at this time is “pitch difference”.

Furthermore, if the voice pitch changes unnaturally before and after the connection boundary, even if the voice changes higher or lower than the target, it tends to feel that the inflection has changed relatively. is there. Therefore, as shown in the equation (4-2) in FIG. 6, the degradation cost C11 and its degradation function are defined as functions of the pitch change penalties PCaa and PCab.
C11 = W11 × (w42 × PCaa + w43 × PCab) Formula (4-2)
The cause of deterioration determined at this time is “boundary pitch difference”.

In addition, when the tone color difference before and after the connection boundary is large, the inflection changes abruptly, so that the intonation tends to feel large in terms of hearing. Therefore, as shown in the equation (6-2) in FIG. 6, the degradation cost C12 and its degradation function are defined as a function of the timbre change penalty PCca.
C12 = W12 × (w61 × PCca) Expression (6-2)
The cause of deterioration determined at this time is “boundary tone color difference”.

Next, when the sound quality deterioration type details indicate that “the intonation is small”, the voice is synthesized with a voice lower than the target in the pitch penalty. Therefore, as shown in the equation (1-3) in FIG. 5, the degradation cost C13 and its degradation function are defined as a function of the pitch penalty PPaa.
C13 = W13 × (w13 × PPaa) Formula (1-3)
The cause of deterioration determined at this time is “pitch difference”.

  Furthermore, false accents exist as one of the unnatural inflections. Since it is difficult to automatically correct an erroneous accent, it is preferable that the user is forced to transit to a manual accent correction UI when “false accent” is selected as the deterioration type detail by the user.

Next, the deterioration type “smooth tongue” will be described. The sound quality deterioration of the tongue is mainly due to the following two types.
Tongue 1) When the voice of the synthesized voice is too jealous, traced, or mixed, compared to the target, Tongue 2) When the tone at the connection boundary changes unnaturally

  Smooth tongue 1) is a case where at least one of phoneme length penalties PPba and PPbb in the segment selection penalty is large. In such a case, the synthesized speech becomes dull or distorted, and the smooth tongue becomes unnatural.

  The smooth tongue 2) is a case where at least one of the phoneme environment penalties PCba and PCbb of the segment connection penalty is large. In such a case, the transition of the sound at the connection boundary between the segments is unnatural, the smoothness of the timbre changes is lost, and the smooth tongue is unnatural. Furthermore, the timbre change penalty PCca may be large. In such a case, since the timbre before and after the boundary changes unnaturally, the change in timbre is not smooth, and the smooth tongue becomes unnatural.

  Therefore, using the unit selection penalty “phoneme length penalty PPba, PPbb” and the unit connection penalty “phoneme environment penalty PCba, PCbb” “timbre change penalty PCca” as shown in FIG. Define degradation cost and degradation cost function.

First, when it is felt that “smooth tongue is ugly” as a detail type of sound quality degradation, it is a case where the phoneme length is shorter than the target. Therefore, as shown in the equation (2-2) in FIG. 6, the degradation cost C20 and its degradation function are defined using the phoneme length penalty PPba.
C20 = W20 × (w22 × PPba) Formula (2-2)
The cause of deterioration determined at this time is “phoneme length difference”.

Furthermore, when the phoneme environment is different before and after the connection boundary, the smoothness of the sound connection before and after the boundary is lost, and it tends to feel ugly or violent. Therefore, as shown in the equation (5-2) in FIG. 6, the degradation cost C21 and its degradation function are defined using the phoneme environment penalty PCba, PCbb.
C21 = W21 × (w52 × PCba + w53 × PCbb) Formula (5-2)
Note that the cause of deterioration determined at this time is “phoneme environment difference”.

In addition, even when the timbre difference before and after the connection boundary is large, the smoothness of the sound connection before and after the boundary is lost, and there is a tendency that it feels ugly or traced. Therefore, as shown in the equation (6-3) in FIG. 6, the degradation cost C22 and its degradation function are defined using the timbre change penalty PCca.
C22 = W22 × (w62 × PCca) (formula 6-3)
The cause of deterioration determined at this time is “boundary tone color difference”.

Next, when it is felt that “the smooth tongue is rugged” in the deterioration type details of the sound quality deterioration, it is a case where the synthesis is performed with a phoneme length longer than the target. Therefore, as shown in the equation (2-3) in FIG. 6, the degradation cost C23 and its degradation function are defined using the phoneme length penalty PPbb.
C23 = W23 × (w23 × PPbb) (2-3)
The cause of deterioration determined at this time is “phoneme length difference”.

Furthermore, when the phoneme environment is different before and after the connection boundary, the smoothness of the connection of sounds before and after the boundary is lost, and there is a tendency to feel irritated or ugly. Therefore, as shown in the equation (5-3) in FIG. 6, the degradation cost C24 and its degradation function are defined using the phoneme environment penalty PCba, PCbb.
C24 = W24 × (w54 × PCba + w55 × PCbb) Formula (5-3)
Note that the cause of deterioration determined at this time is “phoneme environment difference”.

In addition, even when the timbre difference before and after the connection boundary is large, the smoothness of the sound connection before and after the boundary is lost, and there is a tendency to feel confused or ugly. Therefore, the degradation cost C25 and its degradation function are defined using the timbre penalty PPca as shown in the equation (6-4) in FIG.
C25 = W25 × (w63 × PCca) Formula (6-4)
The cause of deterioration determined at this time is “boundary tone color difference”.

  When “false accent” is selected as the degradation type details by the user, it is preferable to force the transition to the manual accent correction UI as in the case of “inflection”.

  With the above configuration, using the fact that the segment selection penalty and segment connection penalty included in the intermediate information at the time of speech synthesis are information that quantitatively represents sound quality degradation, the combination of each penalty is a degradation cost and a degradation cost function. Define as Thereby, the deterioration cost becomes an index suitable for correctly determining the deterioration type, and the determination accuracy of the deterioration type can be improved. Furthermore, since the degradation type does not correspond to the degradation cost on a one-to-one basis and is a combination of a plurality of degradation costs, it is possible to improve the determination accuracy of the degradation type.

  The operation of the speech synthesizer 1 according to the first embodiment will be described below with reference to FIGS. FIG. 7 is a flowchart showing the operation of displaying the degradation type in the speech synthesizer 1. FIG. 8 is a diagram showing a display example of the phonetic text and the continuous phoneme string.

  As shown in FIG. 7, first, in the speech synthesizer 1, the user inputs Japanese text as input text via the sound quality correction UI unit 21. The language processing unit 23 of the speech synthesizing unit 7 acquires the phonetic text by converting the input text (S101). The phonetic text is Japanese text reading in katakana, and there may be additional information such as accent information necessary for speech synthesis.

  In addition, the sound quality correction UI unit 21 acquires the voice synthesized by the voice synthesis unit 7, and the user reproduces the synthesized voice and designates the range where the voice feels degraded as the sound quality degradation range. Is displayed (S102).

  In the speech synthesis unit 7, the prosody generation unit 25 generates a target prosody according to the read text 54, and the segment selection connection unit 27 relates to the connection between the speech units selected from the waveform dictionary 35 according to the target prosody and the segments. Select connection information is generated. The waveform processing unit 29 performs waveform processing on the speech unit selected from the waveform dictionary 35 based on the selected connection information, and generates and reproduces, for example, the synthesized speech waveform 70 of FIG.

  At this time, as shown in FIG. 8, the sound quality correction UI unit 21 presents the input text 52, the reading text 54, the synthesized speech waveform 70, and the like on a display unit (not shown) as in the display example 50. When the input text 52 is “Today's weather is sunny” as in the display example 50, the reading text 54 is expressed as “Kyono / Tenkiwa / Haledes.”. The synthesized speech waveform 70 is represented as time on the horizontal axis and amplitude on the vertical axis.

  Here, the display example 50 is configured to display a reproduction button 62 for reproducing the synthesized voice, a stop button 64 for stopping, and a save button 66 for saving the synthesized voice.

  When the user feels sound quality degradation in the synthesized speech output, the user designates the sound quality degradation range 56 from the reading text 54 or the synthesized speech waveform 70 presented on the display unit. FIG. 8 shows an example in which the sound quality degradation range 56 is designated on the reading text 54. The sound quality degradation range 56 is designated as “Kiha / Hare”.

  That is, returning to FIG. 7, the range acquisition unit 41 acquires the sound quality degradation range 56 designated by the user in the reading text 54 (S103). At this time, for example, in the speech synthesizer 1, when the user selects a “kiwa / halle” portion using a mouse device (not shown) or the like, the range acquisition unit 41 receives the sound quality degradation range via the sound quality correction UI unit 21. 56 is acquired.

  In addition, the range acquisition unit 41 determines the target phoneme range of the synthesized speech that matches the sound quality deterioration range 56 specified by the user based on the sound quality deterioration range 56 specified by the user and the intermediate information from the speech synthesis unit 7. get.

  The deterioration cost calculation unit 43 acquires intermediate information from the segment selection connection unit 27 (S104), and calculates a deterioration cost in the target phoneme range corresponding to the sound quality deterioration range 56 based on the acquired intermediate information and the sound quality deterioration range 56. (S105). The deterioration cost is calculated based on the segment selection penalty and the segment connection penalty which are one of the intermediate information of the speech synthesizer 7 as described above. At this time, the calculated deterioration cost is stored in the storage unit 5, for example.

  By the way, regarding the degradation cost and the degradation type, the correspondence between the degradation type and the degradation cost and the definition method of the degradation cost function are different depending on the speech synthesis method. Briefly described.

  The first speech synthesis method is a waveform conversion method for generating synthesized speech by matching a segment selected from a waveform dictionary with a target prosody by waveform processing such as pitch conversion or phoneme length conversion. Since the waveform conversion process is performed on the segment selected from the waveform dictionary, the waveform dictionary size is relatively small, and there is an advantage that synthesized speech according to the target prosody can be generated. However, the sound quality may be deteriorated by the waveform processing.

  The second speech synthesis method is a waveform non-conversion method in which a segment close to the target prosody is selected from the waveform dictionary without being subjected to waveform conversion processing and connected as it is. Since waveform conversion processing is not performed, there is an advantage that a synthesized voice with a high real voice can be generated, but there is a disadvantage that the prosody breaks down without necessarily following the target prosody. Moreover, the prosody breakdown is suppressed by increasing the waveform dictionary, but it is not perfect.

  The third speech synthesis method is a mixed method of a waveform conversion method and a waveform non-conversion method. Although it is based on the no-waveform conversion method, the use of the waveform conversion method only in places where the prosody breaks down, the advantage of being able to generate synthesized speech with stable prosody while maintaining the real voice feeling with the minimum necessary waveform conversion However, there are cases where a portion with a high sense of real voice and a portion with a loss of real voice are mixed, resulting in an uncomfortable feeling.

  In the first embodiment, in any of the first to third speech synthesis methods, the degradation type is adjusted by adjusting the association between the degradation type and the degradation cost and the definition method of the degradation cost function. Sound quality can be modified accordingly. However, in the present specification, for convenience of explanation, the “variable penalty” shown in FIGS. 3 to 6 and the “degradation type” shown in FIG. "Association of deterioration cost" and "Definition of deterioration cost function" were explained.

  Returning to FIG. 7, when the deterioration cost calculation unit 43 calculates each deterioration cost in the target phoneme range corresponding to the sound quality deterioration range, the deterioration type determination unit 45 determines the sound quality deterioration in the target phoneme range based on the deterioration cost. The deterioration type of “sound, intonation, smooth tongue” is determined (S106).

  Here, the operation of determining the deterioration type will be described with reference to FIG. FIG. 9 is a flowchart showing the operation of determining the degradation type in the speech synthesizer 1. As shown in FIG. 9, the deterioration type determination unit 45 initializes a predetermined number M of deterioration cost upper HC (m) (m = 0 to M−1) (S121). The integer M is the number of deterioration costs Cn that is referred to in order to determine a correction candidate deterioration type when selecting a deterioration type. The deterioration cost Cn is a generic name of the deterioration costs C00 to C25 shown in FIG. Here, n = 00-25.

  The degradation type determination unit 45 acquires the degradation cost Cn calculated by the degradation cost calculation unit 43 (S122). The deterioration type determination unit 45 determines whether or not deterioration cost Cn> deterioration cost upper HC (m) (S123). When the deterioration cost Cn exceeds the deterioration cost upper HC (m) (S123: YES), the deterioration type determination unit 45 proceeds to S124, and sets the deterioration cost upper HC (m) = deterioration cost Cn. If the degradation cost Cn ≦ deterioration cost upper HC (S123: NO), the degradation type determination unit 45 advances the process to S125. The degradation type determination unit 45 sorts the degradation costs Cn by repeatedly executing S123 and S124 for m = 0 to M−1 and registers them in the storage unit 5, for example.

  The degradation type determination unit 45 determines whether or not the next degradation cost Cn exists (S125). While the next degradation cost Cn exists (S125: YES), the degradation cost Cn has a high value of the degradation cost Cn. Repeat sorting in order. As a result, the degradation type determination unit 45 obtains M sorted degradation cost high rank HC (m).

  When there is no unprocessed degradation cost Cn (S125: NO), the degradation type determination unit 45 selects a degradation type from the degradation costs Cn registered in the M or less degradation cost upper HC (m). That is, the degradation type determination unit 45 sets m = 0 (S126), and if m <M (S127: YES), determines whether 0 ≦ n ≦ 05 is satisfied (S128). If 0 ≦ n ≦ 05 (S128: YES), the deterioration type determination unit 45 sets the deterioration type to “sound”, does not display the type details (S129), and updates m = m + 1 (S130). And return to S127.

  When 0 ≦ n ≦ 05 is not satisfied (S128: NO), the process proceeds to S131, and the deterioration type determination unit 45 determines whether 10 ≦ n ≦ 12. When 10 ≦ n ≦ 12 is satisfied (S131: YES), the deterioration type determination unit 45 sets the deterioration type to “intonation”, sets the type details to “large” (S132), and updates m = m + 1 (S130). Then, the process returns to S127.

  When 10 ≦ n ≦ 12 is not satisfied (S131: NO), the process proceeds to S133, and the deterioration type determination unit 45 determines whether n = 13. When n = 13 (S133: YES), the deterioration type determination unit 45 sets the deterioration type to “intonation”, sets the type details to “small” (S134), and updates m = m + 1 (S130) to S127. Return.

  When it is not n = 13 (S133: NO), the process proceeds to S135, and the deterioration type determination unit 45 determines whether 20 ≦ n ≦ 22. When 20 ≦ n ≦ 22 is satisfied (S135: YES), the deterioration type determination unit 45 sets the deterioration type as “smooth tongue”, sets the type details as “sad” (S136), and sets m = m + 1 (S130). Update and return to S127.

   If 20 ≦ n ≦ 22 is not satisfied (S135: NO), the process proceeds to S137, and the deterioration type determination unit 45 determines whether 23 ≦ n ≦ 25. When 23 ≦ n ≦ 25 is satisfied (S137: YES), the deterioration type determination unit 45 sets the deterioration type as “smooth tongue”, sets the type details as “traceable” (S138), and sets m = m + 1 (S130). Update and return to S127.

  When 23 ≦ n ≦ 25 is not satisfied, the degradation type determination unit 45 outputs an error. If m> M, the degradation type determination unit 45 returns to S106 in FIG.

  As described above, each degradation type is classified based on at least one degradation cost. The degradation type discriminating unit 45 determines whether or not the degradation cost value included in the classified degradation type is higher than other degradation costs. In this embodiment, the degradation cost upper HC (m) is determined. It is determined by extracting. When it is determined that the deterioration cost Cn is a high value, that is, the deterioration cost Cn corresponding to the deterioration cost upper HC (m) is determined as the deterioration type of the sound quality correction candidate.

  Returning to the processing in FIG. 7, the deterioration position acquisition unit 47 acquires and holds the deterioration cost value of the deterioration type and its deterioration position for the deterioration type determined as the sound quality correction candidate (S107). FIG. 10 is a diagram illustrating an example of the deterioration position. As shown in FIG. 10, the phoneme string 160 is an example of a re-synthesis range corresponding to the sound quality deterioration range 56 specified in the reading text 54. “Kiwa /” in the sound quality degradation range 56 corresponds to “K | I | W | A” of the phoneme string 162, and “Hare” corresponds to “H | A | R | E” of the phoneme string 163. doing. Here, “|” indicates a phoneme boundary.

  For example, it is assumed that the deterioration cost C20 is determined as the higher deterioration cost HC (m) in the deterioration type determination process of FIG. At this time, the degradation position acquisition unit 47 determines that the degradation cost C20 is a function of the phoneme length penalty PPba, and further refers to the calculated phoneme length penalty PPba in the storage unit 5 to determine a location where the value is high. Identified as a degraded position. The degradation position preferably corresponds to the selected segment. In FIG. 10, the phoneme string “WA” is acquired as the deterioration position 165 corresponding to the deterioration cost C20.

  As another example, it is assumed that the degradation cost C05 is determined as the higher degradation cost HC (m). At this time, the degradation position acquisition unit 47 determines that the degradation cost C05 is a function of the timbre change penalty PCca, and further refers to the calculated timbre change penalty PCca in the storage unit 5 to determine a location where the value is high. Identified as a degraded position. In this example, the phoneme string “WA-HA” in FIG. 10 is acquired as the deterioration position 167 corresponding to the deterioration cost C05.

  The voice synthesizer 1 presents the deterioration type of the correction candidate to the user via the sound quality correction UI unit 21. FIG. 8 is an example in which four types of “sound”, “intonation”, “smooth tongue”, and “false accent” are presented as the degradation types 58 of the correction candidates. In FIG. 8, “dear” and “trace” are further displayed as options as the degradation type details 60. This is the end of the degradation type display process.

  Next, the degradation type selection operation will be described with reference to FIG. FIG. 11 is a flowchart showing the operation of selecting the degradation type. The user selects a degradation type that suits his / her feeling from the presented degradation type 58 and its type details 60 through the sound quality correction UI unit 21. As shown in FIG. 11, the correction means determination unit 49 acquires the deterioration type selected by the user via the sound quality correction UI unit 21, and the deterioration cost corresponding to the deterioration type selected from the deterioration position acquisition unit 47. A degradation position is acquired (S151). In addition, the correction means determination unit 49 receives the re-synthesis range from the range acquisition unit 41, and selects the segment corresponding to the degradation cost in order to improve the sound quality degradation based on the range to be re-synthesized and each degradation cost of the degradation type. Correction information for improving the penalty and segment connection penalty is generated and output to the segment selection / connection unit 27 (S152).

  The correction information is a coefficient αl (l = 1, 2,...) In the expression C1 of each element selection penalty included in the deterioration function indicating the deterioration cost corresponding to the selected deterioration type or deterioration type details, Alternatively, the coefficient βl (l = 1, 2,...) In the equation C2 of each unit connection penalty.

  Returning to FIG. 8, the user selects “sound quality is unnatural” or “smooth tongue is unnatural” as the deterioration type 58 in the sound quality deterioration range 56. Suppose you select “Sad”. It is assumed that the correction means determination unit 49 determines that the deterioration cost is the deterioration cost C20 corresponding to the selected deterioration type “smooth tongue (smooth)” and the deterioration cost corresponding to “sound” is the deterioration cost C05. . At this time, the correction means determination unit 49 determines that the phoneme length penalty PPba corresponds to the degradation cost C20. Further, it is determined that the deterioration cost C05 corresponds to the timbre change penalty PCca.

  Then, in the example of FIG. 10, the correction means determination unit 49 weights the weight of the coefficient α3 corresponding to the phoneme length penalty PPba in the phoneme string 162, and the coefficient β5 corresponding to the timbre change penalty PCca in the phoneme string 163. Correction information weighted with weights is generated.

  When the segment selection / connection unit 27 receives the correction information, the segment selection / connection of the recombination range is performed again, and the selected connection information is generated again. Here, the resynthesis range may be set as a resynthesis range 169 including at least the sound quality deterioration range 56 as shown in FIG. That is, the range acquisition unit 41 determines any segmentation of a phoneme unit, a phoneme sequence unit, a mora unit, or a mora sequence unit that includes the entire sound quality degradation range 56 specified by the user as a resynthesis range. Configure.

  In the example of FIG. 10, for the reading text 54 “Kyono / Tenkiwa / Haledes.”, The user selects “Kiwa / Hare” as the sound quality deterioration range 56. At this time, the designated range of the user is “K | I | W | A | H | A | R | E”, which is the sum of the phoneme sequence 162 and the phoneme sequence 163, as the resynthesis range 168, and the speech synthesis unit 7 Synthesis may be performed. The phoneme string unit “K | I | W | A” and the phoneme string unit “H | A | R | E | D | E | S | U” including the user's designated range may be used as the re-synthesis range 169. . The re-synthesis range 169 is an example in which the two phoneme string units each match the two unit units of the synthesized speech. By dividing the re-synthesis range into units, the sound quality falls outside the range specified by the user. Even when there is deterioration, it is possible to obtain a sound quality improvement effect.

  Returning to FIG. 11, the waveform processing unit 29 re-synthesizes the voice based on the regenerated selection connection information. That is, only the re-synthesis range 168, only the re-synthesis range 169, or all the speech corresponding to the input text 52 including the re-synthesis range 168 or the re-synthesis range 169 is re-synthesized to regenerate the speech waveform of the synthesized speech ( S153). As described above, the speech synthesizer 1 generates correction information that weights the weight corresponding to each segment selection penalty or segment connection penalty corresponding to the degradation type selected by the user. Then, for the above-mentioned re-synthesis range, the speech is selected so that the unit selection penalty PP and the unit connection penalty PC weighted to the target unit selection penalty and the unit connection penalty causing the sound quality degradation are minimized. The sound at the time of re-synthesis is corrected by re-synthesis.

  As described above in detail, the speech synthesizer 1 determines the sound quality deterioration position and the sound quality deterioration type by calculating the deterioration cost in the sound quality deterioration range 56 designated by the user, and determines the sound quality deterioration type as the user. To present. The user selects a degradation type that suits his / her feeling from the presented degradation types. The speech synthesizer 1 acquires the selected deterioration type, generates correction information in which the weight coefficient corresponding to the segment selection penalty or the unit connection penalty corresponding to the acquired deterioration type is updated, and the voice is generated based on the correction information. The speech is corrected by recombining.

  With the above configuration, the speech synthesizer 1 determines the sound quality deterioration position and the sound quality deterioration type from the sound quality deterioration range designated by the user, and presents the sound quality deterioration type to the user. Therefore, the user can select a degradation type that suits his / her sense from the presented degradation types, and sound quality correction according to the degradation type is possible.

  In the speech synthesis technology described above, when selecting a segment from the waveform dictionary and when connecting the selected segment, the selection of a segment that does not match the target prosody has a large penalty, and the segment connection is likely to deteriorate the sound quality. Also give a big penalty. Then, the sound quality of the synthesized speech is improved by performing segment selection and segment connection that minimize the segment selection penalty evaluation value PP and segment connection penalty evaluation value PC. Therefore, as described above, a segment that becomes speech closer to the target prosody is obtained by re-synthesizing by weighting the unit selection penalty corresponding to the degradation type selected by the user or the penalty weight for the segment connection penalty. Selection and segment connection are performed. Thereby, sound quality degradation corresponding to the degradation type designated by the user can be improved.

  A segment selection penalty and a segment connection penalty, which are one of the intermediate information at the time of speech synthesis, are penalties related to sound quality degradation, and the degradation cost is defined using each penalty. Thereby, the deterioration cost is suitable for determining the deterioration type, and the determination accuracy of the deterioration type can be improved. These deterioration types do not correspond to each deterioration cost on a one-to-one basis, and the deterioration type is correctly determined according to a combination of a plurality of deterioration costs. Therefore, it is possible to improve the determination accuracy of the deterioration type.

  As the correction candidate deterioration type, it is possible to improve the determination accuracy of the deterioration type by selecting the deterioration type from the higher deterioration cost that is likely to cause the sound quality deterioration. In other words, those with a high deterioration cost are likely to be the cause of sound quality deterioration, and deterioration types including such a high deterioration cost are also likely to be a cause of sound quality deterioration, so the determination accuracy of the deterioration type is improved. It becomes possible.

  When the sound quality deterioration range is designated, the user may designate the sound quality within an approximate range including the range in which the sound quality is felt, and it is not necessary to designate the range limited to the location where the sound quality deterioration occurs. Further, the range acquisition unit 41 may acquire a sound quality deterioration range that completely includes the sound quality deterioration range 56 designated by the user and that is divided into phoneme units or mora units. At this time, as an alternative to the phoneme unit or the mora unit, a phoneme string unit or a mora string unit that matches the unit of the synthesized speech may be used. When acquiring the sound quality degradation range, the range acquisition unit 41 collates the sound quality degradation range 56 designated by the user with the read text 54 obtained by the speech synthesis unit 7 or the intermediate information of the speech synthesis unit 7. It is preferable.

  In addition, the speech synthesis is not necessarily performed in units of one phoneme or one mora, but is synthesized by mixing a plurality of continuous phonemes (phoneme strings) or a plurality of mora (mora strings). Therefore, the sound quality degradation range acquired above does not necessarily coincide with the boundary of the unit unit of the synthesized speech. Therefore, by setting the segment unit segment that completely includes the sound quality degradation range as the resynthesis range, while maintaining the sound quality of the synthesized speech part where sound quality degradation has not occurred, there was sound quality degradation outside the specified range Even in this case, the sound quality can be improved by re-synthesis in the minimum necessary range.

  The method for acquiring the degradation type is not limited to the above. For example, the degradation type may be presented to the user in descending order of degradation cost, and furthermore, the degradation type with a low possibility of sound quality degradation is not presented, or it is presented inactive that is not reflected at the time of resynthesis even if presented. You may comprise.

(Modification of the first embodiment)
Hereinafter, the speech synthesizer 1 according to a modification of the first embodiment will be described with reference to FIG. This modification is a modification related to the deterioration type determination in the speech synthesizer 1 according to the first embodiment. Therefore, in this modification, detailed description of the same parts as the configuration and operation of the speech synthesizer 1 according to the first embodiment will be omitted, and only the deterioration type determination operation will be described.

  FIG. 12 is a flowchart showing the operation of determining the deterioration type in the present modification. In this modification, determination is made based on whether or not the deterioration type of the correction candidate exceeds a predetermined threshold. As illustrated in FIG. 12, the deterioration type determination unit 45 acquires the deterioration cost Cn calculated by the deterioration cost calculation unit 43 (S181). Subsequently, the degradation type determination unit 45 first sets n = 0 (S182). The degradation type determination unit 45 determines whether or not n <26. If n <26, whether each of C00 to C25 is higher than the preset thresholds S00 to S25. It is determined whether or not (S183). Then, in S183, the deterioration cost Cn determined to be higher than the threshold value is determined in S184 and the subsequent steps.

  In S184, the deterioration type determination unit 45 determines whether or not the deterioration cost Cn <threshold Sn, and if Cn <threshold Sn is not satisfied (S184: NO), n = n + 1 (S185), and the process returns to S183. If Cn <threshold Sn (S184: YES), the process proceeds to S186.

  The degradation type determination unit 45 determines whether or not 0 ≦ n ≦ 05 (S186). When 0 ≦ n ≦ 05 (S186: YES), the deterioration type determination unit 45 sets the deterioration type to “sound”, does not display the type details (S187), and updates to n = n + 1 (S185). And return to S127.

  When 0 ≦ n ≦ 05 is not satisfied (S186: NO), the process proceeds to S188, and the degradation type determination unit 45 determines whether 10 ≦ n ≦ 12. When 10 ≦ n ≦ 12 is satisfied (S188: YES), the deterioration type determination unit 45 sets the deterioration type as “intonation”, sets the type details as “large” (S189), and updates n = n + 1 (S185). Then, the process returns to S183.

  When 10 ≦ n ≦ 12 is not satisfied (S188: NO), the process proceeds to S190, and the deterioration type determination unit 45 determines whether n = 13. When n = 13 (S190: YES), the deterioration type determination unit 45 sets the deterioration type to “intonation”, sets the type details to “small” (S191), and updates n = n + 1 (S185) to S183. Return.

  When n is not 13 (S190: NO), the process proceeds to S192, and the deterioration type determination unit 45 determines whether 20 ≦ n ≦ 22. When 20 ≦ n ≦ 22 is satisfied (S192: YES), the deterioration type determination unit 45 sets the deterioration type as “smooth tongue”, sets the type details as “noticeable” (S193), and sets n = n + 1 (S185). Update and return to S183.

   When 20 ≦ n ≦ 22 is not satisfied (S192: NO), the process proceeds to S194, in which the deterioration type determination unit 45 sets the deterioration type to “smooth tongue”, sets the type details as “tracing”, and sets n = n + 1 (S185). Update and return to S183. If n <26 is not satisfied in S183, the degradation type determination unit 45 returns the process to S106 of FIG.

  For example, in the example in which the deterioration costs C20 and C05 calculated by the deterioration cost calculation unit 43 exceed a predetermined threshold, the deterioration type 58 is “sound quality is unnatural” and “smooth tongue is unnatural”. It can be displayed so that it can be selected, and “dear” can be displayed in the deterioration type details 60 so as to be selectable.

  As described above, according to this modification, the degradation type determination unit 45 uses the predetermined threshold Sn (n: 00 to 25) in which the degradation cost Cn calculated by the degradation cost calculation unit 43 is set in advance for each degradation cost. ), The deterioration type of the deterioration cost Cn is selected as the correction candidate deterioration type.

  Therefore, according to the present modification, by setting a predetermined threshold value, it is possible to avoid the risk of erroneously presenting to the user by determining the deterioration type even though the sound quality deterioration has not occurred. .

  Furthermore, when each deterioration cost does not exceed a predetermined threshold value in this modification, the operation is switched to the operation shown in FIG. 9 according to the first embodiment, and a predetermined number of deterioration cost upper ranks are selected from each deterioration cost. You may make it comprise so that the degradation type of degradation cost high rank may be selected. Such a configuration avoids the problem that “no degradation type” occurs when all degradation costs do not exceed the respective predetermined thresholds, and the sound quality degradation of the user cannot be corrected, and includes higher degradation costs. By presenting the degradation type to the user, the sound quality degradation can be improved.

  If each deterioration cost calculated by the deterioration calculation unit 43 does not exceed a predetermined threshold value, all the deterioration types are presented to the user, and the correction means determination unit 49 passes the sound quality correction UI unit 21 to the user. The manual correction UI corresponding to the degradation type selected by can be activated. For example, the accent manual correction UI may be activated even in the case of a degradation type that is difficult to automatically correct, such as an erroneous accent.

  Normally, it is preferable to present all the sound quality deterioration types because the determined sound quality deterioration type may not match the deterioration type felt by the user, but in this case, the sound quality of the deterioration type selected by the user It is difficult to automatically correct the deterioration. However, by starting the manual correction UI corresponding to the degradation type selected by the user, the optimum manual correction UI is automatically started without selecting the manual correction UI to be started by the user himself. Modifications can be made possible.

(Second Embodiment)
The speech synthesizer according to the second embodiment will be described below with reference to FIGS. In the second embodiment, redundant description of the same configuration and operation as those of the speech synthesizer 1 according to the first embodiment will be omitted.

  FIG. 13 is a block diagram showing the functions of the speech synthesizer according to the second embodiment, FIG. 14 is a flowchart showing the operation of designating a degradation range according to the second embodiment, and FIG. 15 is a diagram showing the second embodiment. It is a flowchart which shows the operation | movement of the deterioration classification designation | designated by a form. The speech synthesizer according to the second embodiment is configured to include a speech correction determination unit 230 instead of the speech correction determination unit 9 in the speech synthesizer 1 according to the first embodiment. Hereinafter, the speech synthesizer according to the second embodiment is referred to as a speech synthesizer 200.

  As shown in FIG. 13, the speech correction determination unit 230 of the speech synthesizer 200 has a correction history database 232 in addition to the configuration of the speech correction determination unit 9. The correction history database 232 is a storage device that stores the degradation type selected by the user.

  As shown in FIG. 14, first, the range acquisition unit 41 acquires the sound quality degradation range 56 designated by the user (S201). Based on the acquired sound quality degradation range 56, the range acquisition unit 41 determines the resynthesis range as, for example, the resynthesis range 169 in FIG. 10 as described above, and initializes the resynthesis number i (S202). The deterioration cost calculation unit 43 calculates the deterioration cost for the sound quality deterioration range 56 by the method described above (S203).

  The deterioration type determination unit 45 determines the deterioration type based on the calculated deterioration cost, for example, by the method described in the first embodiment or its modification. Further, the deterioration position acquisition unit 47 acquires a deterioration position corresponding to the determined deterioration type (S204). Further, the degradation type determination unit 45 presents the degradation type via the sound quality correction UI unit 21 (S205).

  As shown in FIG. 15, the correction means determination unit 49 determines whether or not the deterioration type is acquired as “false accent” via the sound quality correction UI unit 21 (S211). When it is acquired that it is “false accent” (S211: YES), the manual correction UI is activated (S216).

  If it is not acquired as “false accent” (S211: NO), the correction means determination unit 49 generates correction information for weighting the segment selection penalty or segment connection penalty corresponding to the selected degradation type. (S212). The speech synthesizer 7 re-synthesizes the speech based on the generated correction information. At this time, the correction means determination unit 49 increments the re-synthesis number i = i + 1, and sets the target degradation type and the re-synthesis number i. It is stored in the correction history database 232 (S213).

  The deterioration type determination unit 45 determines whether or not the total deterioration cost Cn is less than a predetermined threshold value (S214). If there is at least one degradation cost Cn greater than or equal to the threshold (S214: NO), the correction means determination unit 49 determines whether or not recombination is N times or less for the degradation type for which the degradation cost Cn is not less than the threshold. (S215). If the re-synthesis is N times or less (S215: YES), the process returns to S203. When the re-synthesis exceeds N times (S215: NO), the manual correction UI is activated. The manual correction UI generates correction information stored in association with the degradation type that has been recombined over N times, and corrects the voice based on the generated correction information. At this time, the correction information is preferably stored in, for example, the storage unit 5 in association with the corresponding deterioration type.

  If all the degradation costs Cn are less than the threshold value in S214 (S214: YES), the correction means determination unit 49 presents a correction end confirmation screen to the user via the sound quality correction UI unit 21 (S217), and the sound quality is determined. It is determined whether or not the correction is completed (S218). When the end of the sound quality correction is not confirmed (S218: NO), the manual correction UI is activated, and when the end of the sound quality correction is confirmed (S218: YES), the speech synthesis process is ended.

  As described above, according to the speech synthesizer 200 according to the second embodiment, the recombination count i regarding the same deterioration type is stored in the correction history database 232. When the user selects the same deterioration type a predetermined number of times, the correction means determination unit 49 determines that it is difficult to automatically correct the sound quality desired by the user, and the deterioration selected by the user via the sound quality correction UI unit 21. A manual correction UI corresponding to the type is activated.

  As described above, even when the sound quality deterioration of the deterioration type selected by the user is automatically corrected, if there are consecutive cases in which the sound quality does not become the user's desired sound quality, the user can use the optimum manual correction UI to repeat automatic correction unnecessarily. The sound quality can be corrected by manual operation.

  In the second embodiment, when the number of times selected as the correction deterioration type does not exceed the predetermined N times, the correction stored in association with the most frequently selected deterioration type You may make it correct an audio | voice with information.

(Modification according to the second embodiment)
Hereinafter, a speech synthesizer 200 according to a modification of the second embodiment will be described with reference to FIG. This modification is a modification regarding the resynthesis execution determination process in the speech synthesizer 200 according to the second embodiment. Therefore, in this modification, detailed description of the same parts as the configuration and operation of the speech synthesis apparatus 200 according to the second embodiment will be omitted, and only the resynthesis execution determination will be described.

  In this modification, the correction means determination unit 49 integrates the selection frequency of the deterioration type when the user performs sound quality correction, and, for example, the deterioration type whose integrated value exceeds a predetermined threshold is stored in the correction history database 232. Configure to register. Furthermore, the correction means determination unit 49 generates correction information as follows when performing normal speech synthesis processing. That is, the correction means determination unit 49 corrects the penalty weight with respect to various unit selection penalties or various unit connection penalties constituting the deterioration cost function of the deterioration cost corresponding to the deterioration type of the correction history database 232. Generate information. The segment selection connection unit 27 of the speech synthesizer 7 generates a synthesized speech based on this correction information.

  As described above, according to the present modification, by referring to the correction history stored in the correction history database 232, the sound quality deterioration type selected by the user is frequently used. It is judged that Then, during speech synthesis, in order to suppress deterioration of sound quality of the target degradation type in advance, speech synthesis processing that improves the segment selection penalty and segment connection penalty, which are components of the degradation cost function for calculating the degradation cost of the degradation type I do. As a result, it is possible to reduce the burden of correcting the sound quality deterioration of the user.

  In general, there is a strong individual difference in how sound quality is perceived, so it is easy for users to understand the tendency of individual users to correct sound quality, and to generate synthesized speech that has undergone user preference sound quality correction when generating synthesized speech. It is possible to reduce the burden of correcting sound quality degradation. Furthermore, even if the sound quality deterioration of the deterioration type selected by the user is automatically corrected, if there are consecutive cases in which the sound quality does not become the user desired sound quality, not only automatic sound quality correction according to the deterioration type but also an optimal manual correction UI When activated, the user can correct the sound quality by manual operation.

  In the first embodiment, the second embodiment, and the respective modifications, the correction means determination unit 49 is an example of a correction information generation unit and an acquisition frequency coefficient unit, and a unit selection penalty and unit connection A penalty is an example of deterioration information. In particular, the segment selection penalty is an example of first degradation information, and the segment connection penalty is an example of second degradation information.

  Furthermore, the pitch penalties PPaa and PPab are examples of first difference information, the phoneme length penalty PPba and PPbb are examples of second difference information, and the timbre penalty PPca is an example of third difference information. is there. Further, the pitch change penalty PCaa, PCab is an example of the fourth difference information, the phoneme environment penalty PCba, PCbb is an example of the fifth difference information, and the timbre change penalty PCca is the sixth difference information. It is an example. In the first range, the sound quality degradation range 56 is an example of the first range, and the resynthesis ranges 168 and 169 are examples of the second range.

  Here, an example of a computer that is commonly applied to cause the computer to perform the operation of the speech synthesizer according to the first or second embodiment and the respective modifications will be described. FIG. 16 is a block diagram illustrating an example of a hardware configuration of a standard computer. As shown in FIG. 16, a computer 300 includes a central processing unit (CPU) 302, a memory 304, an input device 306, an output device 308, an external storage device 312, a medium driving device 314, a network connection device, and the like via a bus 310. It is connected.

  The CPU 302 is an arithmetic processing unit that controls the operation of the entire computer 300. The memory 304 is a storage unit for storing in advance a program for controlling the operation of the computer 300 or using it as a work area when necessary when executing the program. The memory 304 is, for example, a random access memory (RAM), a read only memory (ROM), or the like. The input device 306 is a device that, when operated by a computer user, acquires various information input from the user associated with the operation content and sends the acquired input information to the CPU 302. Keyboard device, mouse device, etc. The output device 308 is a device that outputs a processing result by the computer 300, and includes a display device and the like. For example, the display device displays text and images according to display data sent by the CPU 302.

  The external storage device 312 is a storage device such as a hard disk, and stores various control programs executed by the CPU 302, acquired data, and the like. The medium driving device 314 is a device for writing to and reading from the portable recording medium 316. The CPU 302 can read out and execute a predetermined control program recorded on the portable recording medium 316 via the recording medium driving device 314 to perform various control processes. The portable recording medium 316 is, for example, a Compact Disc (CD) -ROM, a Digital Versatile Disc (DVD), a Universal Serial Bus (USB) memory, or the like. The network connection device 318 is an interface device that manages transmission / reception of various data performed between the outside by wired or wireless. A bus 310 is a communication path for connecting the above devices and the like to exchange data.

  A program that causes a computer to execute the speech synthesizer according to the first or second embodiment and the respective modifications is stored in, for example, the external storage device 312. The CPU 302 reads the program from the external storage device 312 and causes the computer 300 to perform a speech synthesis operation. At this time, first, a control program for causing the CPU 302 to perform speech synthesis processing is created and stored in the external storage device 312. Then, a predetermined instruction is given from the input device 306 to the CPU 302 so that the control program is read from the external storage device 312 and executed. The program may be stored in the portable recording medium 316.

  The present invention is not limited to the above-described embodiments and modifications, and various configurations or embodiments can be adopted without departing from the gist of the present invention. For example, the deterioration position acquisition unit 47 acquires the deterioration position corresponding to the correction candidate deterioration type, but it is sufficient to acquire at least the deterioration position corresponding to the correction deterioration type. In addition, the correction information is generated based on the deterioration type and the deterioration position. However, for example, in the case where the sound quality deterioration range is set to be very short, the correction information may not be generated based on the deterioration position. .

DESCRIPTION OF SYMBOLS 1 Speech synthesizer 3 Input part 5 Memory | storage part 7 Speech synthesizer 9 Voice correction determination part 11 Output part 21 Sound quality correction UI part 23 Language processing part 25 Prosody generation part 27 Segment selection connection part 29 Waveform processing part 31 Language dictionary 33 Prosody Dictionary 35 Waveform dictionary 41 Range acquisition unit 43 Deterioration cost calculation unit 45 Degradation type determination unit 47 Deterioration position acquisition unit 49 Correction means determination unit

Claims (15)

A synthesized voice acquisition unit for acquiring synthesized voice;
A range acquisition unit for acquiring a first range for performing correction in the voice;
A degradation cost calculation unit that calculates at least one degradation cost that is information according to sound quality degradation in the first range;
A deterioration type determining unit that determines and presents at least one correction candidate deterioration type according to the nature of the sound quality deterioration that is a candidate for correction in the first range based on the at least one deterioration cost;
A deterioration type acquiring unit that acquires a correction deterioration type selected by the user to perform correction from the correction candidate deterioration types determined and presented ;
A correction information generating unit for generating correction information for correcting the sound based on the correction deterioration type;
A speech synthesizer that re-synthesizes the speech based on the correction information;
A speech synthesizer characterized by comprising: The degradation cost calculation unit
The deterioration cost is calculated based on deterioration information corresponding to the sound quality deterioration calculated for each of a phoneme, a phoneme string, a mora, and a mora string of speech in the first range. The speech synthesizer described. A deterioration position acquisition unit that acquires a deterioration position corresponding to the acquired correction deterioration type;
Further comprising
The correction information generation unit
The speech synthesis apparatus according to claim 2, wherein correction information for correcting the voice is generated based on the correction deterioration type and the deterioration position. The degradation position acquisition unit
The speech synthesis apparatus according to claim 3, wherein the deterioration position is acquired based on the deterioration information. The speech synthesizer
A prosody generation unit that generates a target prosody to be a target when the speech is synthesized;
Selecting a unit for synthesizing the speech based on the target prosody, and a unit selection connecting unit for connecting the selected unit to each other;
Have
The deterioration information is
First degradation information according to a difference between the selected one segment and the target prosody corresponding to the selected segment, and the target prosody corresponding to the segment in a connection portion when connecting the segments The speech synthesizer according to any one of claims 2 to 4, further comprising at least one of second deterioration information corresponding to the difference. The first deterioration information is
First ratio information according to a pitch frequency ratio between the target prosody and the selected segment; second ratio information according to a phoneme length ratio between the target prosody and the selected segment; and 6. The speech synthesizer according to claim 5, comprising at least one of first difference information corresponding to a tone color difference between the target prosody and the selected segment. The second deterioration information is
Third ratio information according to the pitch frequency ratio before and after the connection portion, second difference information according to the phoneme environment difference between the segments before and after the connection portion, and first information according to the timbre difference before and after the connection portion. 7. The speech synthesizer according to claim 5 or 6, wherein at least one of the three pieces of difference information is included. The deterioration type determination unit
2. The deterioration type according to at least one of the at least one deterioration cost, which is equal to or higher than a predetermined order in descending order, is determined as the correction candidate deterioration type. Item 8. The speech synthesizer according to any one of Items 7. The deterioration type determination unit
8. The deterioration type according to the deterioration cost that exceeds a predetermined first threshold according to the deterioration cost is determined as the correction candidate deterioration type. The speech synthesizer described. The speech synthesizer
Wherein said first range, described for the second range the Ru first range of the range of Der claim 1, characterized in that to perform the re-synthesis of the speech to one of claims 9 Speech synthesizer. If the degradation class acquisition unit, at least one of the degradation cost corresponding to the acquired modified degraded type was determined not to exceed a second predetermined threshold value in response to the degradation cost,
The correction information generation unit
The correction information stored in association with the correction deterioration type corresponding to the deterioration cost not exceeding the second threshold is generated as the correction information for correcting the sound. speech synthesis apparatus according to claim 10. An acquisition number counting unit that counts the number of times each modified deterioration type acquired by the deterioration type acquisition unit is acquired,
Further comprising
The correction information generation unit
When any of the number of times exceeds a predetermined number of times, the correction information stored in association with the correction deterioration type acquired exceeding the predetermined number of times is generated as correction information for correcting the sound. The speech synthesizer according to any one of claims 1 to 10. An acquisition number counting unit that counts the number of times each modified deterioration type acquired by the deterioration type acquisition unit is acquired;
Further comprising
The correction information generation unit
Analyzing the number of times, and generating correction information stored in association with a correction deterioration type whose number of times is larger than other correction deterioration types as correction information for correcting the sound,
The speech synthesizer according to any one of claims 1 to 10, wherein: Get the synthesized speech,
Obtaining a first range for correction in the voice;
Calculating at least one deterioration cost which is information corresponding to sound quality deterioration in the first range;
Determining and presenting at least one modification candidate degradation type according to the nature of the sound quality degradation that is a candidate for modification in the first range based on the at least one degradation cost;
The correction deterioration type selected by the user to perform correction from the correction candidate deterioration types determined and presented is acquired,
Generating correction information for correcting the voice based on the correction deterioration type;
Re-synthesize the speech based on the correction information;
An audio correction method characterized by the above. Get the synthesized speech,
Obtaining a first range for correction in the voice;
Calculating at least one deterioration cost which is information corresponding to sound quality deterioration in the first range;
Determining and presenting at least one modification candidate degradation type according to the nature of the sound quality degradation that is a candidate for modification in the first range based on the at least one degradation cost;
The correction deterioration type selected by the user to perform correction from the correction candidate deterioration types determined and presented is acquired,
Generating correction information for correcting the voice based on the correction deterioration type;
Re-synthesize the speech based on the correction information;
A program that causes a computer to execute processing.