JP2010026223A - Target parameter determination device, synthesis voice correction device and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily determine a target parameter which is used for processing in which prosody of synthesis voice is corrected to target prosody. <P>SOLUTION: A target parameter determination device is provided with a voice data storage section; a prosody selection section in which model voice data are obtained including time variation information of a basic frequency of voice, and time information of a phoneme, tone model data, pitch model data and time model data are selected according to a category of the model voice data, and time variation information of each basic frequency of the selected pitch model data and the tone model data, and time information of each phoneme of the selected tone model data and the time model data are obtained; and a target parameter determination section in which the time variation information of the basic frequency included in the tone model data is changed according to the time variation information of the basic frequency included in the pitch model data, and further the time variation information of the basic frequency which is the target parameter is determined, according to time information of the phoneme included in the time model data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a target parameter determination device that determines a prosody of a synthetic speech, a synthetic speech correction device that corrects a synthetic speech according to the determined prosody, and a computer program thereof.

The prosody of a synthesized speech is often predicted and generated based on linguistic information such as the content of the utterance and the context before and after. When the predicted and generated prosody is unnatural, it is necessary to correct the prosody of the synthesized speech to a natural prosody (hereinafter referred to as “target prosody”).
Patent Document 1 discloses a technique related to a user interface that assists a corrector who corrects a prosody. In the technique disclosed in Patent Document 1, a corrector first determines a value of a parameter (hereinafter, referred to as “target parameter”) used for processing for correcting a prosody of a synthesized speech to a target prosody. Then, the user operates the slider displayed on the screen based on the determined target parameter value to correct the prosody of each syllable.

Patent Document 2 discloses a technique for recording spoken voice and extracting feature parameters from the recorded voice. Patent Document 3 discloses a technique for performing prosody conversion. In the technique disclosed in Patent Document 3, speech conversion including prosody is performed on speech segments. Non-Patent Document 1 also discloses a technique for performing prosody conversion.
JP 2003-36100 A JP 2007-140002 A Japanese Patent No. 3913770 Toru Tsuzuki and Tetsuo Umeda, “Voice quality conversion method for correcting distortion in the spectral domain and psychological evaluation of its quality”, Science theory (A), vol. J73-A, no. 3, pp. 387-396, March 1990

  However, since it takes technical knowledge and experience to determine the specific value of the target parameter, the image of the prosody after correction of the synthesized speech to be corrected (hereinafter referred to as “correction target speech”) is corrected. However, it is difficult for the corrector to determine a specific value of the target parameter. Furthermore, even a corrector having such specialized knowledge and experience has a problem that it takes a lot of time to determine a target parameter.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a target parameter determination device, a synthesized speech correction device, and a computer program that can easily determine a target parameter. Is.

[1] In order to solve the above-described problem, a target parameter determination apparatus according to an aspect of the present invention includes a voice, speech content of the voice, time change information of the fundamental frequency of the voice, and phonemes included in the voice. Audio data storage unit for storing the time information of the phonemes representing the timing of the sound, the time variation information of the basic frequency of the sound, and the time information of the phonemes representing the timing of the phonemes included in the sound In accordance with the type of the sample audio data, the tone sample data that is a sample of tone, the pitch sample data that is a sample of pitch, and the time sample data that is a sample of phoneme timing are selected and selected. Further, time change information of the fundamental frequency of each of the pitch sample data and the tone sample data, and the phoneme of each of the selected tone sample data and the time sample data Prosody selection unit for acquiring interval information, time change information of the fundamental frequency included in the tone sample data is changed according to time change information of the fundamental frequency included in the pitch sample data, and further the time sample The time change information of the fundamental frequency as a target parameter is determined by matching with the time information of the phoneme included in the data, and the time information of the phoneme included in the tone sample data is matched with the time information of the phoneme included in the time sample data. And a target parameter determination unit that determines time information of phonemes as target parameters.
Here, the time change information of the fundamental frequency is data in which the values of the fundamental frequency are arranged for each elapsed time. This fundamental frequency value corresponds to the absolute pitch of the sound. Therefore, the time change information of the fundamental frequency described above includes the pitch of the sound.
According to this configuration, each of the tone sample data, the pitch sample data, and the time information sample data is selected according to the type of the sample audio data. Then, according to the time change information of the fundamental frequency of the pitch sample, the time change information of the basic frequency of the tone sample is changed, and further changed to match the time information of the phoneme of the time sample data, so that the target The time change information of the fundamental frequency is obtained. Furthermore, the time change information of the target phoneme is determined by matching the time information of the phonemes of the tone sample data with the time information of the phonemes of the time sample data.

[2] Further, according to one aspect of the present invention, in the target parameter determination device, a voice input unit that receives voice input, an utterance content acquisition unit that acquires utterance content corresponding to the voice, and the voice input unit A speech analysis unit that calculates time change information of the fundamental frequency of the speech and time information of the phoneme of the speech based on the speech and the utterance content received by the voice, and the prosody selection unit, The voice having the time change information of the fundamental frequency and the time information of the phoneme calculated by the voice analysis unit is acquired as the sample voice data.
According to this configuration, the input of the voice and the utterance content of the voice is accepted, and the time change information of the fundamental frequency of the voice and the time information of the phoneme are obtained based on the received voice and the utterance content, and the sample is obtained. Audio data was obtained. This makes it possible to use the input voice as voice sample voice data.

[3] Further, according to an aspect of the present invention, in the target parameter determination device, the sample voice designation unit that receives an input of an instruction to select sample voice data, and the instruction received by the sample voice designation unit A sample voice search unit that obtains the sample voice data by searching a voice data storage unit is further provided, and the prosody selection unit acquires the sample voice data obtained by the sample voice search unit. .
According to this configuration, audio data corresponding to an instruction for selecting a sample from the audio data stored in the audio data storage unit can be used as the sample audio data.

[4] Further, according to one aspect of the present invention, in the target parameter determination device, a correction target voice designation unit that receives an input of an instruction to select a correction target voice to be corrected, and the voice data storage unit are searched. A sample voice search unit that obtains a sample voice having the same notation as the correction target voice, and the prosody selection unit acquires the sample voice data obtained by the sample voice search unit. And
According to this configuration, sample voice data having the utterance content corresponding to the voice is obtained in accordance with the designation of the voice to be corrected.

  [5] Further, according to one aspect of the present invention, in the target parameter determination apparatus, a correction target voice storage unit that stores a voice that is a correction target and a notation of the voice that is the correction target from the voice data storage unit A sample voice search unit that obtains sample voice data having a different notation and the same phoneme number or mora number, and the prosody selection unit includes the sample voice data obtained by the sample voice search unit It is characterized by acquiring.

[6] Moreover, one aspect of the present invention is a synthesized speech correction device, the target parameter determination device described above, a correction target speech storage unit (synthesized speech storage unit) that stores speech to be corrected, A correction unit that reads the correction target voice and corrects the correction target voice based on the time change information of the fundamental frequency and the time information of the phoneme determined by the target parameter determination device.
According to this configuration, the target parameter of the synthesized speech to be corrected by the target parameter determination device is determined only by specifying the sample voice data. Then, based on the determined target parameter, the prosody of the audio data to be corrected is corrected.

  [7] In addition, the computer program according to one aspect of the present invention provides a phoneme, utterance content of the speech, time change information of the fundamental frequency of the speech, and time information of phonemes indicating timing of phonemes included in the speech. A computer having a voice data storage unit that stores and associates with each other, obtains sample voice data having time change information of the fundamental frequency of voice and time information of phonemes representing the timing of phonemes included in the voice, According to the type of sample audio data, tone sample data that is a sample of tone, pitch sample data that is a sample of pitch, and time sample data that is a sample of phoneme timing are selected, and the selected pitch Time variation information of the fundamental frequency of each of the sample data and the tone sample data, and the phonemes of the selected tone sample data and the time sample data, respectively. Prosody selection means for acquiring time information, time change information of the fundamental frequency of the tone sample data is changed according to time change information of the fundamental frequency of the pitch sample data, and the time sample data The time change information of the fundamental frequency serving as a target parameter is determined by matching with the time information of the phoneme possessed by the phoneme, and the time information of the phoneme included in the tone sample data is matched with the time information of the phoneme included in the time sample data Is a computer program for functioning as target parameter determining means for determining time information of phonemes as target parameters.

  According to the present invention, it is possible to easily determine a target parameter without requiring the corrector to specifically examine the value of the target parameter to be used for processing for correcting the prosody of the synthesized speech to the target prosody.

[First Embodiment]
Hereinafter, a plurality of embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a functional configuration of a target parameter determination device according to the first embodiment. As shown in the figure, the target parameter determination device 1 includes a speech database 11, a correction target speech designation unit 12, a synthesized speech storage unit 13, a speech input unit 14, and a speech text input unit 15 (utterance content acquisition unit). The voice analysis unit 16, the sample voice designation unit 17, the sample voice search unit 18, the prosody selection unit 19, and the target parameter determination unit 20 are configured.

The speech database 11 stores speech data including speech signal data that is time-series data of speech signal amplitude, speech prosody information, and speaker identification information. The prosody information includes time change information of the fundamental frequency and time information of the phonemes. The speaker identification information is identification information for identifying a speaker who utters a voice.
The audio database 11 includes a hard disk device, a magneto-optical disk device, a semiconductor memory,
It is configured by a recording medium such as a CD-ROM or a combination thereof.

The correction target voice designation unit 12 receives an input of an instruction to select a synthetic voice that is a correction target from among the synthesized voices stored in the synthesized voice storage unit 13. The correction target voice specifying unit 12 receives an input for specifying the synthesized voice data, for example, by text data representing the utterance content of the synthesized voice data to be corrected. For the correction target voice designation unit 12, for example, an input device such as a keyboard or a mouse is used.
The synthesized speech storage unit 13 stores synthesized speech data including speech signal data, speech prosody information, utterance content (notation), and speaker identification information.

The voice input unit 14 receives an input of sample voice, which is a voice serving as a sample, using a microphone or the like.
The voice text input unit 15 receives input of text data representing the utterance content of the sample voice received by the voice input unit 14. For the voice text input unit 15, for example, an input device such as a keyboard or a mouse is used.

The voice analysis unit 16 analyzes the sample voice input from the voice input unit 14 based on the text data received by the voice text input unit 15 and generates prosody information of the sample voice. Specifically, the speech analysis unit 16 uses an acoustic model having speech signal data and phoneme labels by executing forced alignment of speech recognition technology, and from the text data received by the speech text input unit 15. The acoustic model corresponding to the obtained phoneme label is applied to each sample voice, the boundary in the time axis direction with respect to the adjacent phoneme label is detected, the start time and the end time for each phoneme are determined, and the time information of the phoneme is obtained.
Furthermore, the voice analysis unit 16 generates time change information of the fundamental frequency of the sample voice in a voiced section (voiced voice section that is voice accompanied by vocal cord vibration), and the voiced section is generated with respect to the generated fundamental frequency value. By performing smoothing using a spline function or the like using the analysis value of the above, the time change information of the fundamental frequency with a smooth change is generated. At this time, the voice analysis unit 16 uses the fundamental frequency values of the voiced sections before and after the voiceless section for a voiceless section (voiced voice section that is voice without accompanying vocal cord vibration) for which the fundamental frequency value cannot be obtained. Using the interpolated value, the fundamental frequency of the silent section is interpolated.

  The sample voice designating unit 17 includes, for example, kana notation, kanji notation, and phoneme label notation for the utterance content of the voice (sample voice) that becomes a sample in the case of correction among the sound data stored in the sound database 11. And accept either. Specifically, the sample voice designating unit 17 accepts designation of any one of the other language speaker voice, the other language speaker voice, and the other language speaker voice. The foreign language speaker voice is a voice having the same correction target voice and speaker identification information and having a different utterance content from the correction target voice. The other-language / other-speaker voice is a voice having different speech to be corrected and speaker identification information and having different utterance contents from the voice to be corrected. The same-speech other-speaker voice is a voice in which the correction target voice and the speaker identification information are different, and the correction target voice and the utterance content are the same. For example, an input device such as a keyboard or a mouse is used for the sample voice designation unit 17.

The sample voice search unit 18 reads out from the voice database 11 at least one of prosody information of the other language speaker voice, the other language speaker voice, and the synonym speaker voice.
Further, the sample voice search unit 18 searches the voice database 11 using text data representing the utterance contents and speaker identification information of the correction target voice received by the sample voice specifying unit 17 as a key, and as a result, Get voice.

The prosody selection unit 19 includes prosody information generated by the voice analysis unit 16 (at least one of the other-language synonym voice, the other-language other-speaker voice, and the same-word other-speaker voice) and the sample. The prosody information of the sample voice (at least one of the other-language synonym voice, the other-language other-speaker voice, and the same-word other-speaker voice) read by the voice search unit 18 or the synthesized voice to be corrected From the prosodic information, a tone sample (tone sample data that is a sample of tone), a pitch sample (pitch sample data that is a sample of pitch), and a time information sample (phoneme timing) that are used for target parameter determination processing Prosody selection processing is performed for selecting each of the time sample data). Here, the pitch is the pitch of the sound, and is represented by, for example, the fundamental frequency of the sound. For example, the maximum, minimum, or average value of the fundamental frequency value included in the time change information of the fundamental frequency is used as the pitch. The fundamental frequency is the lowest frequency among the harmonic components of speech. The tone is an arrangement (pitch accent) of the pitch of the voice, and is represented by a time series of relative or absolute changes in the value of the fundamental frequency.
Details of the prosody selection processing will be described later with reference to the drawings.

  The target parameter determination unit 20 performs target parameter determination processing based on the prosodic information of the tone sample, the pitch sample, and the time information sample selected by the prosody selection unit 19 to determine the target parameter. The target parameter is information (prosodic information) including time change information of a fundamental frequency having a target prosody and time information of phonemes having a target prosody.

The voice data stored in the voice database 11 will be described in more detail with reference to FIG. FIG. 2 is a schematic diagram showing the configuration of audio data stored in the audio database 11.
The speech data is data that is stored for each speech in association with speech signal data, prosodic information, speaker identification information, kanji notation, kana notation, and phoneme label notation. The audio signal data is data corresponding to an audio waveform, for example, array data based on time series of amplitude values. The prosody information includes time change information of the fundamental frequency and time information of the phonemes. The time change information of the fundamental frequency is data in which values of the fundamental frequency are arranged for each elapsed time. The phoneme time information represents the start time and the end time of each phoneme with the start time of the speech as a time reference. The phoneme will be described later.
The speaker identification information is identification information for identifying a voice speaker. The kanji notation is information in which kanji representing speech utterance contents are arranged in the order of utterances. Kana notation is information in which kana representing the utterance content of speech is arranged in the order of utterances. The phoneme label notation is information in which phoneme labels representing speech utterance contents are arranged in the order of utterances.

For example, in FIG. 2, the first line of the data in the figure is the audio signal data in which the time change information “FRQ1” of the basic frequency of the audio of the audio signal data “WAVE1” and the time information “TIME1” of the phoneme are prosodic information. , The speech identification information of this speech is “A01”, the kanji notation of the speech content of this speech is “Hokkaido”, the kana notation is “Hokkaido”, and the phoneme label of the speech content of this speech The notation is “hoQkaido:”.
Further, the fourth line of the data in the figure represents voice data whose utterance content is 1 mora, the time change information “FRQ4” of the fundamental frequency of the voice of the voice signal data “WAVE4”, and the time information of phonemes. “TIME4” is associated with the speech signal data as prosodic information, the speaker identification information of this speech is “A01”, and there is no kanji notation of the speech utterance content (“−” in FIG. 2). Kana notation is “A”, and the phoneme label notation of the utterance content of this voice is “a”.
The voice database 11 can be newly stored when voice data is newly created. In addition, the voice database 11 obtains voice data created by another device that creates voice and newly creates the voice data. It is also possible to memorize.

  FIG. 3 is a waveform diagram of an audio signal. The vertical axis in the waveform diagram of FIG. 3 represents the amplitude, and the horizontal axis represents the time that has elapsed with reference to the time when the voice was started. Data in which the amplitude value of the audio signal is arranged for each elapsed time is the audio signal data. Specifically, FIG. 3 is a waveform diagram in a case where “Hokkaiido” which is WAVE1 in the first line of the data in FIG. 2 is spoken.

  FIG. 4 is a graph showing a change of the fundamental frequency according to time. This time change information of the fundamental frequency is data in which values of the fundamental frequency are arranged for each elapsed time. In FIG. 4, the vertical axis represents the fundamental frequency, and the horizontal axis represents the elapsed time. Specifically, FIG. 4 is a graph showing FRQ1 in the first row of the data in FIG.

FIG. 5 is a schematic diagram showing text data representing time information of phonemes. For example, in FIG. 5, TIME1 on the first line is the name of the time information data of phonemes, and the second to eleventh lines are the time information of phonemes for each phoneme. Note that a phoneme is a minimum unit of sound in phonology, and each vowel and consonant corresponds to one phoneme. In addition, each of the repellent sound, the long sound, and the prompt sound corresponds to one phoneme.
In the data from the second row to the eleventh row, the first column shows the time to the start time of each phoneme in units of 1 / 10,000 second with the start time of the voice as a time reference, and the second column shows the start time of the voice. The time until the end point of each phoneme is expressed in units of 1 / 10,000 second as a time reference, and the third column shows phoneme labels of phonemes. For example, in FIG. 5, “0 4750 sil” indicates that the period from when the voice starts until 0.475 seconds elapses is a silent section. Further, “4750 5100 h” represents that the phoneme from the time when 0.475 seconds elapses to the time when 0.51 seconds elapses with respect to the start time of the sound is “h”. Note that the phoneme label sil indicates that there is no phoneme, the phoneme label Q indicates a prompt sound, and the phoneme label o: indicates a long sound of “o”. Although the case where the time is in units of 1 / 10,000 seconds has been described as an example here, the time may be expressed in other units such as 1 / 1000th of a second (millisecond).

FIG. 6 is a conceptual diagram for explaining conditions when the user selects another language voice for which the sample voice designating unit 17 accepts input.
There are two types of other language voices: another language other speaker's voice and another language same speaker's voice. The priority of prosodic information of other language speech changes when selecting a tone sample, a pitch sample, and a time information sample, depending on whether or not it is another language speech.
When the other language voice is designated from the sample voice designation unit 17, the user selects a voice having the same number of mora in the speech to be corrected and the utterance content. Mora is a phonological unit of sound length. In Japanese, generally, two characters of kana correspond to 1 mora for stuttering, and one character of kana corresponds to 1 mora for other than stuttering. One mora is composed of one or more phonemes.
FIG. 6A is a diagram showing a mora delimiter and phoneme delimitation of a specific example “Ao No” of the speech to be corrected, and FIG. 6B is a mora delimiter of a specific example “Shiro No” of another language sound. It is a figure showing a phoneme division | segmentation.
In the case of FIG. 6, the number of mora of “Shiro No” is 5, which matches the number of mora of the correction target voice “Ao No”. Therefore, “Shiro No” satisfies the condition that the number of mora is the same as the correction target voice, and the user can select it as the other language voice.

  Note that the user can also select a speech that has the same number of mora in the utterance content as the correction target speech and a different number of phonemes from the correction target speech. For example, as shown in FIG. 6, the number of phonemes of “Shiro No” shown in FIG. 6B is 7, and the number of phonemes of the correction target speech “Ao No” shown in FIG. The number of phonemes does not match because it is 5, but “Shiro No” satisfies the condition because the number of mora matches as described above. Therefore, the user can select a speech having the same number of mora as the correction target speech and a different number of phonemes as the other language speech.

FIG. 7 is a schematic diagram showing an example of a priority order in which the prosody selection unit 19 selects a sample voice. Specifically, in FIG. 7, a combination of a tone sample, a pitch sample, and a time information sample is associated with the priority order.
Examples of voice candidates include the following voices. Listed below are the types of audio.
(1) Voice received by the voice input unit (a) Speech to be corrected and the same utterance content (b) Utterance content and number of same phonemes different from the correction target voice (c) 2) Speech stored in the speech database (a) Utterance content, same phoneme number, and speaker different from the correction target speech (b) Utterance content, same mora number, same speaker different from the correction target speech (c) Correction target speech Utterance content, same phoneme number, other speaker (d) utterance content, same mora number, other speaker different from correction target speech (e) correction target speech, same utterance content, other speaker (3) correction target speech

For the tone sample, pitch sample, and time information sample, (1) (a) to (1) (c) and (2) (a) to (2) (e), respectively, Any one of (3) is selected.
For example, in the first line of FIG. 7, a combination in which (2) (a) is a tone sample, (3) is a pitch sample, and (3) is a time information sample is associated with priority 1. Has been.
In the second line of FIG. 7, the combination (2) (a) is a tone sample, (2) (b) is a pitch sample, and (3) is a time information sample. Is associated with.
Here, 1 is the highest priority.

  Specifically, the prosody selection unit 19 selects one tone tone sample from the sample speech received by the speech input unit 14 or the sample speech obtained by the search from the speech database 11 by the sample speech search unit 18. For the pitch sample, the prosody selection unit 19 uses the sample voice (the target voice and the voice of the same speaker) obtained by the sample voice search unit 18 from the voice database 11 or the correction obtained from the synthesized voice storage unit 13. Select one of the target voices. The prosodic information selection unit 19 obtains the phoneme time information from the sample speech received by the speech input unit 14, the sample speech obtained from the speech database 11 by the sample speech search unit 18, and the synthesized speech storage unit 13. One of the selected correction target voices is selected.

  The prosody selection unit 19 selects a sample voice selected for the tone sample, a sample voice selected for the pitch sample, and a sample voice selected for the phoneme time information according to the priority of each possible combination. Select the highest priority combination.

For example, the input of the sample voice is not received from the voice input unit 14 and the input of the sample voices (2) (a), (2) (b) is received from the sample voice specifying unit 17 and the correction target voice is stored. When (3) is received from the voice storage unit 13, the priority corresponding to the combination in which the tone sample is (2) (a), the pitch sample is (3), and the time information sample is (3) is 1 In addition, since the priorities corresponding to other combinations of these sample sounds are 2, 3, and 7, the prosody selecting unit 19 selects a combination having a priority of 1.
Also, for example, a sample voice (1) (a) is received from the voice input unit 14, a sample voice (2) (b) is received from the sample voice designation unit 17, and the synthesis target voice is stored. When (3) is received from the voice storage unit 13, the priority corresponding to the combination in which the tone sample is (1) (a), the pitch sample is (3), and the time information sample is (3) is 4. Since the priority corresponding to the combination in which the tone sample is (1) (a), the pitch sample is (2) (b), and the time information sample is (3) is 5, the prosody selecting unit 19 , A combination with a priority of 4 is selected.

In addition, in FIG. 7, it is not restricted to the combination shown in figure, You may make it change a combination and change a priority.
Further, for example, when the information shown in FIG. 7 is stored in advance in a storage device such as a hard disk and the prosody selection processing is performed, the prosody selection unit 19 refers to this storage device and uses the sample according to the priority order. Prosodic information can be selected.

Next, details of the target parameter determination unit 20 will be described.
FIG. 8 is an explanatory diagram for explaining a case where time change information of the fundamental frequency of the tone sample is calculated using an average value of the fundamental frequency in the time direction (hereinafter referred to as “time average value”). In FIG. 8, the vertical axis represents the fundamental frequency, and the horizontal axis represents time.

FIG. 8A and FIG. 8B show a graph (a) showing the time change of the fundamental frequency of the tone sample and a graph (A) showing the time change of the fundamental frequency of the pitch sample. Further, FIG. 8B is a graph (C) showing the time change of the fundamental frequency calculated by the target parameter determination unit 20 based on the average value of the fundamental frequency of the pitch sample and the average value of the fundamental frequency of the tone sample. ).
When calculating the time change information of the fundamental frequency of the tone sample using the time average value of the fundamental frequency, the target parameter determination unit 20 first calculates the time average value of the fundamental frequency of the tone sample (FIG. 8A) and The time average value of the fundamental frequency of the pitch sample (FIG. 8A) is calculated. Then, the target parameter determination unit 20 makes the time average value of the fundamental frequency of the tone sample (FIG. 8B) the same as the time average value of the fundamental frequency of the pitch sample (FIG. 8A). Such time change information of the fundamental frequency of the tone sample is calculated. Specifically, the target parameter determination unit 20 calculates the difference between the time average value of the fundamental frequency of the pitch sample and the time average value of the fundamental frequency of the tone sample, and uses the calculated difference as the fundamental frequency of the tone sample. Calculate time-series data of the added sum. FIG. 8B shows a graph of time change information of the fundamental frequency of the calculated tone sample.
Thus, by calculating the fundamental frequency of the tone sample having an average value that is the same as the average value of the fundamental frequency of the pitch sample, the time of the fundamental frequency of the tone sample close to the pitch of the pitch sample. Change information is obtained.

Next, with reference to FIG. 9, from the start of speech to the end of utterance of each speech unit of the tone sample, the total length of the tone sample totaling the total time from the start of speech to the end of speech of the tone sample of the tone sample A case will be described in which the overall length of the tone sample is changed without changing the time ratio (ratio of the speech segment to the overall length). Here, the speech segment is speech waveform data constituting the synthesized speech. In the example of FIG. 9, a case where the unit of the speech unit is a mora will be described.
FIG. 9 is a conceptual diagram illustrating a case where the overall length of the tone sample is changed without changing the ratio of the speech segment to the overall length. The upper part of FIG. 9 represents the time information of the phonemes of the time information sample, the middle part of FIG. 9 represents the time information of the phonemes of the tone sample, and the lower part of FIG. 9 represents the time information of the phonemes of the target parameters. The horizontal axis in FIG. 9 indicates time.
When changing the overall length of the tone sample without changing the ratio of the speech unit to the overall length, the target parameter determination unit 20 first sums the time from the start of speech to the end of speech of each speech unit in the time information sample. The total length of the time information sample and the total length of the tone sample are calculated. Then, the target parameter determination unit 20 changes the time information of the phoneme of the tone sample so that the total length of the time information sample matches the overall length of the tone sample without changing the ratio of the speech segment to the overall length. Specifically, the target parameter determination unit 20 calculates the total length of the time information sample and the total length of the tone sample. Then, the target parameter determination unit 20 calculates the ratio of the speech segment to the overall length, and calculates the ratio of the overall length of the tone sample to the overall length of the time information sample relative to the time information of the phoneme of the tone sample. By changing the time information, the time information of the phoneme of the tone sample in which the total length of the time information sample and the total length of the tone sample coincide with each other is obtained. The target parameter determining unit 20 determines the obtained phoneme time information as the phoneme time information of the target parameter.
As a result, even if the total length of the time information sample and the total length of the tone sample are different, it is possible to obtain the total length of the tone sample in accordance with the total length of the time information sample, and to adjust the total length of the tone sample. it can.

FIG. 10 is an explanatory diagram for explaining a case where the time change information of the fundamental frequency of the tone sample in which the pitch of the tone sample is changed in accordance with the pitch of the pitch sample is changed according to the time information sample. When the time change information of the fundamental frequency of the tone sample in which the pitch of the tone sample is changed according to the pitch of the pitch sample is changed according to the time information sample, the target parameter determination unit 20 sets the pitch of the pitch sample. The time change information (FIG. 10 (c)) of the fundamental frequency of the tone sample that is close to the time length of the basic frequency time change information of the tone sample whose pitch is changed according to the time information sample and the time information of the phoneme By changing the overall length so as to match, the time change information of the fundamental frequency of the target parameter is generated.
Specifically, the target parameter determination unit 20 calculates the ratio of the total length of the time information of the tone sample with the pitch changed to the total length of the time information of the time information sample, and calculates the calculated ratio as By multiplying the total length of the time information of the tone sample whose pitch is changed, the time information of the tone sample whose pitch is changed is updated. For example, in FIG. 10D, the time change information (FIG. 10C) of the fundamental frequency of the tone sample that is close to the pitch of the pitch sample is updated to match the overall length of the phoneme time information. The time change information of the later fundamental frequency is shown.

  The target parameter determination unit 20 changes the fundamental frequency of the tone sample based on the height, the maximum value, or the minimum value of the fundamental frequency of the pitch sample instead of the average value of the fundamental frequency of the pitch sample. It may be. The target parameter determination unit 20 changes the fundamental frequency of the tone sample based on one of the average value, maximum value, and minimum value of the fundamental frequency of the pitch sample, and the height of the basic frequency of the pitch sample. May be.

A case will be described in which the fundamental frequency of the tone sample is changed based on the maximum value, minimum value, and pitch range of the fundamental frequency of the above-described pitch sample. FIG. 11 is an explanatory diagram illustrating a case where the target parameter determination unit 20 calculates time change information of the fundamental frequency of the tone sample based on the maximum value of the fundamental frequency of the pitch sample. When calculating the time change information of the fundamental frequency of the tone sample based on the maximum value of the fundamental frequency of the pitch sample, the target parameter determination unit 20 first calculates the maximum value of the fundamental frequency of the pitch sample. Then, the target parameter determination unit 20 calculates the maximum value of the fundamental frequency of the tone sample (FIG. 11A) and the maximum value of the basic frequency of the pitch sample, and the basic frequency of the pitch sample (FIG. B) Calculate the fundamental frequency of the pitch sample with the same maximum value. Specifically, the target parameter determination unit 20 calculates a difference between the maximum value of the basic frequency of the pitch sample and the maximum value of the basic frequency of the tone sample, and the basic frequency at each time point of the tone sample is calculated to the calculated difference. The time series data of the sum that is added is generated.
11 (a) is a graph showing the fundamental frequency of the tone sample, FIG. 11 (a) is a graph showing the fundamental frequency of the pitch sample, and FIG. 11 (c) is a graph of the pitch sample. It is a graph of the sum which added the fundamental frequency in each time of a tone sample to the difference of the maximum value of a fundamental frequency and the maximum value of the fundamental frequency of a tone sample.

  Next, a case where the target parameter determination unit 20 calculates time change information of the fundamental frequency of the tone sample based on the minimum value of the fundamental frequency of the pitch sample will be described. When calculating time change information of the fundamental frequency of the tone sample based on the minimum value of the fundamental frequency of the pitch sample, the target parameter determining unit 20 firstly determines the minimum value of the basic frequency of the pitch sample and the basic frequency of the tone sample. The minimum value of is calculated. Then, the target parameter determination unit 20 calculates the fundamental frequency of the tone sample such that the minimum value of the fundamental frequency of the tone sample and the minimum value of the basic frequency of the pitch sample are the same value. Specifically, the target parameter determination unit 20 calculates a difference between the minimum value of the fundamental frequency of the pitch sample and the minimum value of the basic frequency of the tone sample, and the basic frequency at each time point of the tone sample is calculated to the calculated difference. The time series data of the sum that is added is generated.

FIG. 12 is an explanatory diagram for explaining a case where the target parameter determination unit 20 changes the fundamental frequency of the tone sample based on the pitch range and the average value of the pitch sample. When the target parameter determination unit 20 changes the fundamental frequency of the tone sample based on the pitch range and the average value of the fundamental frequency of the pitch sample, the target parameter determination unit 20 first determines the maximum and minimum values of the fundamental frequency of the pitch sample. The width of the value (difference in value) and the width between the maximum value and the minimum value of the fundamental frequency of the tone sample are calculated. Next, the target parameter determination unit 20 sets the width between the maximum value and the minimum value of the fundamental frequency of the tone sample to the same value as the width of the maximum value and the minimum value of the fundamental frequency of the pitch sample calculated previously. Thus, time change information of the fundamental frequency of the tone sample is generated. Specifically, the target parameter determination unit 20 calculates a ratio between the maximum and minimum widths of the fundamental frequency of the tone sample and the maximum and minimum widths of the basic frequency of the pitch sample, and the tone sample. Time series data of a value obtained by multiplying the fundamental frequency at each time point by this ratio is generated.
FIG. 12A is a graph of the fundamental frequency of the tone sample. FIG. 12A is a graph of the fundamental frequency of the tone sample that is the same as the width of the maximum value and the minimum value of the fundamental frequency of the tone sample. The graph is shown in FIG.
Here, the target parameter determination unit 20 is a time series of the sum obtained by adding the average value of the fundamental frequencies of the pitch samples to the fundamental frequency at each time point of the fundamental frequencies of the obtained tone samples (FIG. 12A). Calculate the data. A graph of the fundamental frequency of the tone sample obtained at this time is shown in FIG.

FIG. 13 shows the time of the phoneme of the tone sample so that the target parameter determination unit 20 matches the length of the speech unit of the time information sample and the length of the speech unit of the tone sample between the corresponding speech units. It is a conceptual diagram showing the process concept in the case of changing information. The length of a speech unit is the time from the start of speech of one speech unit to the end of speech. The corresponding speech unit is a speech unit in which the order counted from the head of the arranged speech units is the same in the time information sample and the tone sample.
The upper part of FIG. 13 represents the length of the mora of the tone sample, the middle part of FIG. 13 represents the length of the mora of the target parameter, and the lower part of FIG. 13 represents the length of the mora of the time information sample. The length of the mora is a total value of the lengths (time) of phonemes included in the mora. The horizontal axis in FIG. 13 indicates time.
When changing the time information of the phoneme of the tone sample so that the length of the time information sample mora matches the length of the corresponding tone sample mora, first, the target parameter determination unit 20 first sets the time information sample (lower). The length of each mora is calculated. Then, the target parameter determination unit 20 changes the length of the mora of the tone sample (upper) to match the length of the mora of the corresponding time information sample (lower). For example, the target parameter determination unit 20 matches the length of the mora of the tone sample with the length of the mora of the corresponding time information sample. The target parameter determination unit 20 changes the time information of the phonemes of the tone sample based on the length of the mora obtained by this processing. That is, the time information of the phoneme of the target parameter (middle stage) is obtained.

8 and 11 to 12, according to the pitch sample, (1) a process for changing the pitch of the tone sample based on the time average value of the fundamental frequency, and (2) based on the maximum value of the fundamental frequency. Processing to change the pitch of the tone sample, (3) processing to change the pitch of the tone sample based on the minimum value of the fundamental frequency, and (4) tone sample based on the difference between the maximum value and the minimum value of the fundamental frequency. Although the processing for changing the pitch is described, as the processing for changing the pitch, any one of (1) to (4) may be applied in accordance with a user instruction.
9 and 13, (5) a process of changing the overall length without changing the ratio of each speech unit, and (6) the time information of each phoneme and the time information of each phoneme in the time information sample. The process of changing the overall length by changing the corresponding speech units to match each other has been described, but as the process of changing the overall length, one of (5) and (6) May be applied in accordance with user instructions.
In addition, using FIG. 10, (7) the time change information of the fundamental frequency of the tone sample in which the pitch of the tone sample is changed according to the pitch of the pitch sample based on the time average value of the fundamental frequency, The process of changing the overall length without changing the ratio of the speech segments has been described. In (7), when the pitch of the tone sample is changed, it is not changed based on the time average value of the fundamental frequency, but one of the above (2), (3), and (4) is performed. It may be. Further, in (7), the case where the overall length of the tone sample after changing the pitch is changed, and (5) the case where the overall length is changed without changing the ratio of each speech unit has been described. The total length may be changed by changing the time information of each phoneme and the time information of each phoneme in the time information sample so that the corresponding speech segments match each other. As described above, after changing the pitch of the tone sample according to the pitch of the pitch sample according to any one of (1) to (4), the pitch of the tone sample whose pitch is changed according to (5) or (6). The overall length can be changed. The user may select which of the processes (1) to (4) and the processes (5) and (6) to apply.

Next, a processing procedure of the entire target parameter determination device 1 will be described.
FIG. 14 is a flowchart showing a processing procedure of the entire target parameter determination device 1.
As shown in the figure, in step S01, the correction target voice specifying unit 12 first receives an input for specifying the correction target voice. Specifically, the correction target voice designation unit 12 accepts designation of an arbitrary correction target voice by accepting input of the utterance content of the correction target voice and the speaker identification information.

  Next, in step S03, it is determined whether or not the designation of the sample voice has been completed. If the designation of the sample voice has been completed, the process proceeds to step S09. If the designation of the sample voice has not been completed, the process proceeds to step S04. This determination is performed by, for example, either the number of times the loop from step S04 to S08 is repeated (for example, the number specified by the user) or the presence / absence of a designation end command.

Next, in step S04, it is determined whether or not the voice input unit 14 has received sample voice input. When the voice input unit 14 receives an input of the sample voice, the process proceeds to step S05, and the voice text input unit 15 receives an input of text data representing the utterance content of the sample voice received by the voice input unit 14. . Next, in step S06, the speech analysis unit 16 performs speech analysis processing on the input sample speech, and based on the text data input by the speech text input unit 15, the input speech (synonymous speech) (Or other language speech) prosodic information is generated, and the process proceeds to step S03.
On the other hand, in step S04, if the voice input unit 14 has not received the input of the sample voice, it is determined in step S07 whether the sample voice specifying unit 17 has received the input of the text data specifying the sample voice. To do. When the sample voice designating unit 17 has not accepted the input of text data designating the sample voice, the process proceeds to step S03, and when the sample voice designating unit 17 accepts the input of text data designating the sample voice. In step S08, the sample voice search unit 18 converts the voice data of the designated sample voice (any other-speaker voice, other-speaker voice, same-speaker other-speaker voice) into the voice database 11. From step S03, and proceeds to step S03.

  On the other hand, when the designation of the sample voice is completed in step S03 (step S03: YES), the prosody information selection unit 19 in step S09, the prosody of the correction target voice that the correction target voice designation unit 12 has received the input. Information is read from the synthesized speech storage unit 13.

Next, in step S10, the prosody selection unit 19 calculates sample speech or prosody information from which prosody information has been read between the start of processing of the entire target parameter determination device 1 and the processing of step S10. From the sample sounds, the sound suitable for each of the tone sample, the pitch sample, and the time information sample is selected according to the priority order.
Next, in step S11, the target parameter determination unit 20 generates a tone sample that matches the pitch of the pitch sample based on the time change information of the basic frequency of the tone sample and the time change information of the basic frequency of the pitch sample. The time change information of the fundamental frequency is calculated.
For example, the target parameter determination unit 20 calculates the time average value of the basic frequency of the tone sample and the time average value of the basic frequency of the pitch sample, and calculates the time average value of the basic frequency of the pitch sample and the basic frequency of the tone sample. The basic frequency of the tone sample is adjusted to the pitch of the pitch sample by calculating the time series data of the sum of the calculated time difference and the sum of the calculated difference and the basic frequency of the tone sample. Get time change information.

Next, in step S12, the target parameter determination unit 20 calculates phoneme time information that is a target parameter based on the time information of the phonemes of the tone sample and the time information of the phonemes of the time information sample.
For example, the target parameter determination unit 20 calculates the overall length of the tone sample based on the time information of the phonemes of the tone sample and the overall length of the time information sample based on the time information of the phonemes of the time information sample. Then, the target parameter determination unit 20 calculates the ratio of the speech segment to the overall length, converts the time information of the phoneme of the tone sample into the time information of each phoneme of the tone sample, and the overall length of the tone sample with respect to the overall length of the time information sample. To obtain the time information of the phoneme of the tone sample in which the total length of the time information sample and the total length of the tone sample coincide with each other. The target parameter determining unit 20 determines the obtained phoneme time information as the phoneme time information of the target parameter.

Next, in step S13, the target parameter determination unit 20 is the target parameter based on the time change information of the fundamental frequency of the tone sample according to the pitch of the pitch sample and the time information of the phoneme of the target parameter. It is obtained by calculating time change information of the fundamental frequency.
For example, the target parameter determination unit 20 calculates the ratio of the total length of the time information of the tone sample with the pitch changed to the total length of the time information of the time information sample, and the pitch is calculated based on the calculated pitch. By multiplying the overall length of the time information of the changed tone sample, the time information of the tone sample whose pitch has been changed is updated and obtained as a target parameter.
When the target parameter is obtained by updating the time information of the tone sample whose pitch has been changed, the process of the entire flowchart is terminated.

  In the target parameter determination apparatus 1 configured as described above, a user who corrects the correction target voice utters a sample voice having the target prosody to the voice input unit 14 or inputs an instruction to the sample voice designation unit 17. Thus, the target parameter is determined. Therefore, the user can easily determine the target parameter only by inputting an instruction to speak or select the sample voice without specifically examining the value of the target parameter.

  Further, in the target parameter determination device 1, even if the speech content is different from the speech to be corrected, such as the other language speaker voice or the other language speaker voice, the tone sample, the pitch sample, and the time information sample Can be used to determine target parameters. Therefore, in the target parameter determination device 1, the voice uttered by the corrector or the voice designated from the voice database 11 does not necessarily have the same utterance content as the correction target voice. Therefore, it is possible to improve the degree of freedom of designation of the corrector's utterance or the voice database 11, and it becomes easier to determine the target parameter.

<Modification>
In the first embodiment described above, the prosody selection unit 19 uses the same-speech speech, other-speech speech, other-speaker speech, other-speaker speech, and synonym-speech for the phoneme time information. The total time length of the voice of each person's voice (the voice obtained from the voice database 11 or the voice received by the voice input unit 14) is not normalized with the total time length of the voice section of the correction target voice. However, the overall time length of each speech section is normalized so as to match the overall time length of the speech section of the correction target speech, and is selected from the normalized phoneme time information. Anyway. In this normalization, for example, the target parameter determination unit 20 calculates the total time information sample length of the correction target speech and the overall length of the speech to be normalized. Then, the target parameter determination unit 20 calculates the ratio of the speech segment to the entire length of the correction target speech, and changes the calculated ratio by multiplying the time information of the speech phoneme to be normalized.
By normalizing in this way, the target parameter can be obtained without changing the overall length of the sound to be corrected.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
FIG. 15 is a block diagram showing a functional configuration of the synthesized speech correction apparatus 2 according to the embodiment. As shown in the figure, the synthesized speech correction apparatus 2 includes each functional unit included in the target parameter determination apparatus 1 according to the first embodiment shown in FIG. Parts corresponding to the respective functional units of the target parameter determination device 1 in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

The correction unit 21 selects a sample speech unit according to the target parameter obtained by the target parameter determination unit 20, selects the speech signal data of the speech unit, and selects the speech signal data of the speech unit of the speech to be corrected. Then, by replacing the selected speech signal data with the selected speech signal data, the prosody of the speech to be modified is modified to generate a synthesized speech having a prosody close to the target prosody.
Specifically, the correction unit 21 calculates, for each speech unit, a difference between the fundamental frequency of the target parameter and the fundamental frequency of the speech to be modified, and a speech unit whose difference exceeds a predetermined threshold (hereinafter, (Referred to as “speech segment to be corrected”).
When detecting the speech unit to be corrected, the correcting unit 21 (a) a difference in fundamental frequency at the start time of the speech unit, (b) a difference in fundamental frequency at the end time of the speech unit, (c) Difference in fundamental frequency at an intermediate point between the start and end of a speech unit, (d) Difference in average value of fundamental frequencies in the range from start to end of speech unit, (e) From start of speech unit The time until the end is detected based on whether one of the definite integral values of the absolute value of the fundamental frequency difference in the range exceeds a predetermined threshold value.
Next, the correction unit 21 (1) the detected speech element to be corrected matches the phoneme label, and (2) the correction target section (the correction target speech in the time change information of the fundamental frequency of the target parameter). The speech signal data of the speech unit having the fundamental frequency closest to the fundamental frequency of the segment corresponding to the time from the start to the end of the segment is read from the speech database 11. Then, the correcting unit 21 converts the audio signal data of the section to be corrected into the audio signal data of the read speech segment and the corresponding prosody information (time change information of the fundamental frequency and time information of the phoneme). Then, it is newly registered in the synthesized voice storage unit 13 as synthesized voice.
The correction unit 21 obtains time change information of the fundamental frequency by performing frequency analysis again on the newly registered synthesized speech as a whole.

The correction unit 21 determines more specifically whether or not the condition (2) is satisfied as follows. First, the correction unit 21 sets one or more preset indexes among the average value of the fundamental frequency, the start value, the end value, and the phoneme time information of the same part of the target prosody, with a minimum error. Using a cost function of waveform-connected speech synthesis that includes a term for evaluating the existence, a speech unit of the same kind that minimizes the error is retrieved from the speech database 11. And it determines with the audio | voice signal data of the obtained audio | voice unit being the audio | voice signal data which satisfy | fill the conditions of (2).
Then, the correction unit 21 corrects the prosody of the correction target speech by rewriting the audio signal data of the target speech unit to be corrected to the selected audio signal data.

FIG. 16 is an explanatory diagram showing a change state of time change information of the fundamental frequency in the correction target voice before and after the correction. The upper part of FIG. 16A represents time change information of the fundamental frequency, and the lower part of FIG. 16A represents time information of phonemes. The time change information of the fundamental frequency in the upper part of FIG. 16A and the time information of the phonemes in the lower part of FIG. 16A are information of the same sound. FIG. 16A is a graph showing the time change of the fundamental frequency of the target speech, and FIG. 16A is a graph showing the time change of the fundamental frequency of the target parameter.
In FIG. 16A, the correcting unit 21 detects a speech unit in which the difference between the time change information of the basic frequency of the target speech to be corrected and the time change information of the basic frequency of the target parameter is greater than or equal to a predetermined value. The speech unit (the fourth speech unit (“I”) counted from the head (“A”)) is determined to be the speech unit to be corrected. Next, the correction unit 21 reads out the speech signal data of the speech unit that satisfies the above-described conditions and the time change information of the fundamental frequency from the speech database 11. Then, the correcting unit 21 rewrites the audio signal data of the speech unit to be corrected to the audio signal data of the audio unit that satisfies the above-described conditions, and performs audio analysis on the rewritten audio signal data to obtain prosody information. Recreate it.
FIG. 16B is a graph showing the time change of the fundamental frequency of the synthesized speech after correction, and FIG. 16B is a graph showing the time change of the fundamental frequency of the target parameter. By such correction processing, the time change information of the basic frequency of the correction target voice is corrected so as to approach the time change information of the basic frequency of the target parameter.
When correcting the time information of the phoneme of the synthesized speech, the correcting unit 21 corrects the time information of the phoneme to be corrected of the correction target speech by rewriting the time information of the phoneme obtained from the speech database 11. I do.

  The synthesized speech correcting apparatus 2 configured as described above determines a target parameter by designating a speech having a prosody that is a correction target, and corrects the synthesized speech based on the determined target parameter. . Therefore, the user can easily correct the synthesized speech without specifically considering the target parameter.

<Modification>
In the second embodiment described above, the correction unit 21 corrects the audio signal data by updating the audio signal data for a section in which the difference between the basic frequency of the correction target voice and the basic frequency of the target parameter exceeds a predetermined threshold. However, the correction may be performed not only on the section exceeding the predetermined threshold but on all sections, or a part of the sections (for example, as shown in FIG. 16A). The speech unit corresponding to the section) may be designated by the user to be corrected. Further, the size of the weight of the cost function described above may be arbitrarily changed in accordance with an instruction input from the user.
Further, when selecting speech segments that satisfy the conditions (1) and (2) above, the speech data of the same speaker may be selected, or the speech data of other speakers may be selected. Also good.
Note that in other devices such as a device that performs speech synthesis without using the target prosody created by the target parameter determination unit 20, speech unit selection based on accent information obtained from language analysis of speech content may be performed. is there.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.
FIG. 17 is a block diagram showing a functional configuration of the synthesized speech correction apparatus 3 according to the embodiment. As shown in the figure, the synthesized speech correction device 3 includes each functional unit included in the target parameter determination device 1 according to the first embodiment shown in FIG. 1 and a correction unit 31. Parts corresponding to the respective functional units of the target parameter determination device 1 in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

  The correcting unit 31 corrects the synthesis target speech to be corrected using the target parameter determined by the target parameter determining unit 20. At this time, there are (1) a method of replacing only the speech unit to be corrected by prosody conversion, and (2) a method of replacing the entire speech to be corrected by prosody conversion.

  In the case of the method of replacing only the speech unit to be modified in (1) by prosody conversion, the modification unit 31 corresponds to the speech unit to be modified based on the time change information of the fundamental frequency given as the target parameter. Either (a) time change information of the fundamental frequency, (b) an average value of the fundamental frequency (time average value), or (c) a maximum value of the fundamental frequency of the section is used. Further, the correcting unit 31 determines whether the power of the sound signal is (d) the power of the sound signal of the sound unit to be corrected, (e) the average value of (d) described above, or (f) the (d) described above. One of the maximum values of is used. Further, the correcting unit 31 extracts and uses the data of the speech unit to be corrected from the time information based on the time information of the phoneme given as the target parameter. And the correction part 31 performs the process which converts an audio | voice signal using these values. Note that existing technology is used for the audio signal conversion processing itself. Here, which value of the above (a), (b), and (c) is used depends on a preset value stored in advance. Further, which value of (d), (e), and (f) is used depends on a preset value stored in advance.

  In the case of the method (2) of replacing the entire synthesized speech to be corrected by prosody conversion, the correction unit 31 is based on the time change information of the fundamental frequency given as the target parameter. Or (b) an average value (time average) of fundamental frequencies calculated from time change information of the fundamental frequency is used. Further, the correcting unit 31 uses the power of the correction target sound as the power of the corrected sound signal. Further, the correcting unit 31 uses the phoneme time information given as the target parameter as it is. And the correction part 31 performs the process which converts an audio | voice signal using these values. The use of the existing technology as the audio signal conversion process itself is the same as in the above case. Here, which value of (a) and (b) is used depends on a preset value stored in advance.

  The correcting unit 31 includes speech signal data obtained by prosody conversion by any one of the above methods (1) or (2), and corresponding prosodic information (basic frequency time change information and phoneme time information). ) Are newly registered as synthesized speech in the synthesized speech storage unit 13.

  FIG. 18 is a conceptual diagram illustrating a case where a speech unit to be corrected is replaced by prosody conversion. For example, when the speech unit “I” of “ANO” is targeted for correction and the prosody conversion is performed by the method (1) described above, for example, the time of the fundamental frequency corresponding to the speech unit “I” The change information changes from the graph shown in FIG. 18A to a graph shown in FIG.

  The synthesized speech correction apparatus 3 configured as described above determines a target parameter by designating a speech having a prosody as a correction target by the user, similarly to the synthesized speech correction apparatus 2 in the second embodiment. Then, the synthesized speech is corrected based on the determined target parameter. Therefore, the user can easily correct the synthesized speech without specifically considering the target parameter.

In the case where a part or all of the functions of the target parameter determination device 1, the synthesized speech correction device 2, and the synthesized speech correction device 3 in the above-described embodiment are realized by a computer, a program for realizing the functions of these devices May be recorded on a computer-readable recording medium, and a program recorded on the recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
As mentioned above, although embodiment of this invention was explained in full detail with reference to drawings, the concrete structure is not restricted to this embodiment, The design etc. of the range which does not deviate from the summary of this invention are included.

It is a block diagram showing the functional structure of the target parameter determination apparatus by 1st Embodiment. It is the schematic showing the structure of the audio | voice data memorize | stored by an audio | voice database. It is a wave form diagram showing the change of the amplitude of an audio | voice signal. It is a graph showing the change according to the time of a fundamental frequency. It is the schematic showing the outline of the time information of a phoneme. It is a conceptual diagram explaining the conditions when a user selects the other language voice which the sample audio | voice designation | designated part 17 receives an input. It is a schematic diagram showing an example of the priority which selects the audio | voice data used as a sample, when the prosody selection part 19 performs a prosody selection process. It is explanatory drawing explaining the case where the time change information of the fundamental frequency of a tone sample is calculated using the time average value of the value of a fundamental frequency. It is a conceptual diagram explaining the case where the overall length of a tone sample is changed without changing the ratio of the speech segment to the overall length. It is explanatory drawing explaining the case where the time change information of the fundamental frequency of the tone sample which the pitch of the tone sample was changed according to the pitch of the pitch sample is changed according to the time information sample. It is explanatory drawing explaining the case where the target parameter determination part 20 calculates the time change information of the fundamental frequency of a tone sample based on the maximum value of the fundamental frequency of a pitch sample. It is explanatory drawing explaining the case where the target parameter determination part 20 changes the fundamental frequency of a tone sample based on the height range and average value of the fundamental frequency of a pitch sample. The target parameter determination unit 20 changes the time information of the phoneme of the tone sample so that the length of the speech unit of the time information sample and the length of the tone unit of the tone sample coincide with each other. It is a conceptual diagram showing the processing concept in the case. It is a flowchart showing the process sequence of the whole target parameter determination apparatus. It is a block diagram showing the function structure of the synthetic | combination voice correction apparatus 2 by 2nd Embodiment. It is explanatory drawing showing the change state of the time change information of the fundamental frequency in the audio | voice for correction before and behind correction. It is a block diagram showing the function structure of the synthetic | combination voice correction apparatus 3 by 3rd Embodiment. It is a conceptual diagram explaining the case where the speech element of correction object is converted and replaced.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Target parameter determination apparatus 11 Audio | voice database 12 Correction object audio | voice designation | designated part 13 Synthetic audio | voice storage part (correction object audio | voice storage part)
14 Voice input part 15 Voice text input part (utterance content acquisition part)
16 speech analysis unit 17 sample speech designation unit 18 sample speech search unit 19 prosody selection unit 20 target parameter determination unit 2, 3 synthesized speech correction device 21, 31 correction unit

Claims (7)

A voice data storage unit that stores voice, utterance content of the voice, time change information of the fundamental frequency of the voice, and time information of phonemes that represent the timing of phonemes included in the voice;
Sample tone data having time change information of the fundamental frequency of speech and time information of phonemes representing the timing of phonemes included in the speech is acquired, and a tone sample that is a sample of tone according to the type of the sample speech data Data and pitch sample data that is a sample of pitch and time sample data that is a sample of the timing of phonemes, and time variation information of the fundamental frequency of each of the selected pitch sample data and tone sample data; A prosody selection unit that acquires time information of the phonemes of the selected tone sample data and the time sample data;
By changing the time change information of the fundamental frequency included in the tone sample data according to the time change information of the basic frequency included in the pitch sample data, and further matching the time information of the phonemes included in the time sample data. While determining the time change information of the fundamental frequency as the target parameter, the time information of the phoneme as the target parameter is adjusted by matching the time information of the phoneme included in the tone sample data with the time information of the phoneme included in the time sample data. A target parameter determination unit to determine;
A target parameter determination device comprising: The target parameter determination device according to claim 1, wherein
A voice input unit that accepts voice input;
An utterance content acquisition unit for acquiring utterance content corresponding to the voice;
A voice analysis unit that calculates time change information of the fundamental frequency of the voice and time information of the phoneme of the voice based on the voice and the utterance content received by the voice input unit;
Further comprising
The prosody selection unit acquires the speech having the time change information of the fundamental frequency and the time information of the phoneme calculated by the speech analysis unit as the sample speech data.
A target parameter determination device characterized by that. The target parameter determination device according to claim 1, wherein
A sample voice designation unit that accepts input of an instruction to select sample voice data;
A sample voice search unit for obtaining the sample voice data by searching the voice data storage unit based on an instruction received by the sample voice designation unit;
The prosody selection unit obtains sample voice data obtained by the sample voice search unit;
A target parameter determination device characterized by that. The target parameter determination device according to claim 1, wherein
A correction target voice designation unit that receives an input of an instruction to select a correction target voice to be corrected;
A sample voice search unit that obtains a sample voice having the same notation as the correction target voice by searching the voice data storage unit;
The prosody selection unit obtains sample voice data obtained by the sample voice search unit;
A target parameter determination device characterized by that. The target parameter determination device according to claim 1, wherein
A correction target voice storage unit for storing a correction target voice;
A sample voice search unit that obtains sample voice data that is different from the notation of the voice to be corrected and has the same number of phonemes or mora from the voice data storage unit;
Further comprising
The prosody selection unit obtains sample voice data obtained by the sample voice search unit;
A target parameter determination device characterized by that. A target parameter determination device according to claim 1;
A correction target voice storage unit for storing a correction target voice;
A correction unit that reads out the correction target speech and corrects the correction target speech based on time change information of the fundamental frequency and time information of phonemes determined by the target parameter determination device;
A synthesized speech correction apparatus comprising: A computer having a voice data storage unit that stores voice, speech utterance content, time change information of the fundamental frequency of the voice, and time information of phonemes representing timing of phonemes included in the voice in association with each other. ,
Sample tone data having time change information of the fundamental frequency of speech and time information of phonemes representing the timing of phonemes included in the speech is acquired, and a tone sample that is a sample of tone according to the type of the sample speech data Data and pitch sample data that is a sample of pitch and time sample data that is a sample of the timing of phonemes, and time variation information of the fundamental frequency of each of the selected pitch sample data and tone sample data; Prosody selection means for acquiring time information of each of the selected tone sample data and time sample data;
By changing the time change information of the fundamental frequency included in the tone sample data according to the time change information of the basic frequency included in the pitch sample data, and further matching the time information of the phonemes included in the time sample data. While determining the time change information of the fundamental frequency as the target parameter, the time information of the phoneme as the target parameter is adjusted by matching the time information of the phoneme included in the tone sample data with the time information of the phoneme included in the time sample data. Target parameter determining means for determining,
Computer program to function as.

Images