JP5328703B2 - Prosody pattern generator - Google Patents

Description

  The present invention relates to a prosodic pattern generation device that generates a prosodic pattern from an input user's utterance.

Conventionally, a technique for correcting to a prosodic pattern intended by the user by using prosodic information extracted from the user's utterance has been proposed. Here, the prosody information indicates the prosody (pitch, phoneme duration, power, etc.) itself extracted from the utterance. The prosodic pattern indicates a prosody arranged in quality and format that can be used in synthesized speech.
For example, in Patent Document 1, a pitch pattern of speech input from a microphone is determined, a pitch pattern candidate is generated from input text data, and the pitch pattern determined from the input speech is compared with the pitch pattern candidate. Thus, an audio output device that sets a pitch pattern of audio with a high degree of matching is disclosed. Further, in Patent Document 2, when an unknown word is registered in the notation-compatible prosodic dictionary, the notation and reading of the unknown word are acquired, and voice information corresponding to the unknown word is acquired via voice input, A speech synthesizer that registers prosody extracted from information in a notation-compatible prosody dictionary is disclosed.

JP 2009-53522 A JP 2009-48008 A

  However, conventionally, a pitch pattern candidate generated from text data is selected and used as a corrected pitch pattern, and the pitch pattern generated from the text data is an artificial prosodic pattern generated based on a rule. There was a problem that it was a regular prosodic pattern and lacked naturalness. In addition, in order to obtain pitch pattern (prosodic pattern) candidates based on text data, it is difficult to generate pitch pattern (prosodic pattern) candidates corresponding to text data in unknown words, new words, and the like. there were.

  Furthermore, because prosody is extracted from utterances of unknown words and used as prosody of unknown words, prosody (prosodic information) cannot always be successfully extracted from uttered speech information due to the influence of the user's voice quality and recording environment. . In addition, since the utterance is not limited to a professional narrator, the user's utterance cannot be obtained, and the extracted prosody (prosodic information) is accompanied by local deterioration. There was also a problem that occurred.

The present invention has been made to solve the above-described problems, and a real voice prosody pattern, which is a prosodic pattern in which correction of extraction errors and the like is performed on real voice prosody information that is prosodic information extracted from speech data. A prosody pattern dictionary that stores multiple words is prepared in advance, and from this prosody pattern dictionary, a real voice prosody pattern of similar text such as unknown words or new words is selected by selecting a real voice prosody pattern close to the real voice prosody information of the input voice data It is an object of the present invention to provide a prosody pattern generation apparatus that generates and outputs a natural voice real prosody pattern even when there is no voice, or even in a voice quality or recording environment of a user who cannot obtain stable utterance or real voice prosody information.
In the following description, the real voice prosody pattern and the real voice pitch pattern are described as the prosodic pattern and the pitch pattern unless otherwise specified.

The prosody pattern generation device according to the present invention receives an input of speech data and text data, extracts prosodic information from the speech data, and generates a real voice prosody information extracting unit that associates the prosodic information with the text data. A prosodic pattern dictionary that stores a plurality of prosodic patterns, and a similar prosodic pattern search unit that searches the prosodic pattern dictionary for one or more prosodic patterns similar to the real voice prosodic information part or the whole and outputs them as similar prosodic patterns The real voice prosody information input from the real voice prosody information extraction unit is processed to approximate the prosody pattern stored in the prosodic pattern dictionary and output to the similar prosody pattern search unit, and the similar prosody Converts the pattern into a user-recognizable format and presents it, requiring the user to select a similar prosodic pattern And similar prosodic pattern presenting unit that, among the similar prosodic pattern similar prosody pattern presentation unit is presented, which is constituted to include the prosody pattern output unit for outputting the similarity prosody pattern selected by the user.

  Search prosody patterns similar to part or whole of real voice prosody information generated based on speech data and text data from the prosodic pattern dictionary, present them to the user as similar prosodic patterns, and output the similar prosodic pattern selected by the user Thus, a prosodic pattern similar to the real voice prosodic information can be selected from the prosodic pattern dictionary storing the prosody pattern subjected to correction and maintenance, and a highly natural prosodic pattern can be generated. Also, prosody patterns with high naturalness can be generated even when similar text data such as unknown words or new words does not exist. Furthermore, a stable prosodic pattern can be generated without being affected by the sound collection situation.

It is a block diagram which shows the structure of the speech synthesis system which has a prosodic pattern generation apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the similarity calculation method and similar pitch pattern search method of the prosodic pattern generation apparatus by Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the prosodic pattern generation apparatus by Embodiment 1 of this invention. It is a block diagram which shows the other structural example of the real voice pitch information extraction part by Embodiment 1 of this invention. It is a block diagram which shows the other structural example of the similar pitch pattern presentation part by Embodiment 1 of this invention. It is explanatory drawing which shows the production | generation of the pitch pattern from the utterance containing degradation of the prosody pattern production | generation apparatus by Embodiment 1 of this invention. It is a block diagram which shows the structure of the speech synthesis system which has a prosodic pattern generation apparatus by Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the prosodic pattern generation apparatus by Embodiment 2 of this invention. It is a block diagram which shows the structure of the speech synthesis system which has a prosodic pattern generation apparatus by Embodiment 3 of this invention. It is a flowchart which shows operation | movement of the prosodic pattern generation apparatus by Embodiment 3 of this invention. It is explanatory drawing which shows the example of a process of the similar pitch pattern of the prosodic pattern generation apparatus by Embodiment 3 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a speech synthesis system having a prosodic pattern generation device according to Embodiment 1 of the present invention. In the first embodiment, description will be made using a pitch pattern generation apparatus that handles only pitches as prosodic information and prosodic patterns. Therefore, hereinafter, the prosodic pattern generation device is described as a pitch pattern generation device. The same applies to the second embodiment and the third embodiment.
The speech synthesis system 100 includes an input device 1, a pitch pattern generation device (prosodic pattern generation device) 2, and an output device 3.

Next, details of each component will be described. The input device 1 includes a voice input unit 11, a text input unit 12, and a pitch pattern selection unit 13.
The voice input unit 11 includes, for example, a microphone, collects utterances, generates voice data, and outputs the voice data to the pitch pattern generation device 2. In addition to directly recording the utterance from the microphone, the voice data may be read from a recording device (not shown) that stores prerecorded utterance data, or the voice data may be collected from the stream data. Any other configuration can be used as long as the audio data can be obtained. Further, the voice data may be a user's own voice or another user's voice. The text input unit 12 collects text data (for example, reading, notation, pause information) corresponding to the utterance, creates text data, and outputs the text data to the pitch pattern generation device 2. The pitch pattern selection unit 13 outputs the pitch pattern selection result input by the user to the pitch pattern generation device 2 as pitch pattern selection information.

  A pitch pattern generation device (prosodic pattern generation device) 2 includes a real voice pitch information extraction unit (real voice prosody information extraction unit) 21, a similar pitch pattern search unit (similar prosody pattern search unit) 22, and a pitch pattern dictionary (prosody pattern dictionary) 23. , A similar pitch pattern presentation unit (similar prosody pattern presentation unit) 24, and a pitch pattern output unit (prosody pattern output unit) 25, and searches for similar pitch patterns based on speech data and text data input from the input device 1. The retrieved similar pitch pattern is output to the output device 3 and presented to the user, and the similar pitch pattern corresponding to the pitch pattern selection information input from the input device 1 is output as a pitch pattern.

  The real voice pitch information extraction unit 21 analyzes the voice data input from the voice input unit 11 to extract pitch information, and obtains real voice pitch information in which the text data input from the text input unit 12 is associated with the pitch information. Generated and output to the similar pitch pattern search unit 22. Note that a known processing method such as a cepstrum method or an autocorrelation function method can be applied to the extraction of pitch information. Since the processing method is well known, detailed description thereof is omitted.

  The similar pitch pattern search unit 22 has a similarity calculation unit 22a. The similarity calculation unit 22a calculates the similarity between the real voice pitch information input from the real voice pitch information extraction unit 21 and the pitch pattern stored in the pitch pattern dictionary 23 (hereinafter referred to as an accumulated pitch pattern). Based on the similarity, the pattern search unit 22 searches for one or more stored pitch patterns that are similar to the portion of the real voice pitch information (for example, a unit of a continuous voiced sound section) or the whole, and uses the searched stored pitch patterns as similar pitch patterns. Is output to the similar pitch pattern presentation unit 24.

Here, a calculation example of similarity and a search example of a similar pitch pattern will be specifically described with reference to FIG. FIG. 2 is an explanatory diagram showing a similarity calculation method and a similar pitch pattern search method of the pitch (prosodic) pattern generation device according to the first embodiment.
As shown in FIG. 2, when the real voice pitch information “interchange” is input, the similarity calculation unit 22 a stores a plurality of accumulated pitch patterns stored in the pitch pattern dictionary 23 (pitch pattern 1 and pitch pattern 2 in FIG. 3). , Pitch pattern 3...

Next, the input real voice pitch information and the read accumulated pitch pattern 1 are divided into voiced sound sections, and the distance and likelihood (d ₁₁ , d ₁₂ , d ₁₃ , d ₁₄ ) are calculated for each divided voiced sound section. To do. Furthermore, the sum of distances and likelihoods (d ₁₁ + d ₁₂ + d ₁₃ + d ₁₄ ) of each voiced sound section is calculated. The sum of the calculated distance and likelihood is used as the similarity between the real voice pitch information and the accumulated pitch pattern.

This process is executed for all the accumulated pitch patterns 2, 3..., And the similarity ((d ₂₁ + d ₂₂ + d ₂₃ + d ₂₄ ), (d ₃₁ + d ₃₂ + d ₃₃ + d ₃₄ )...) Is calculated. To do. The similar pitch pattern search unit 22 compares the similarities calculated by the similarity calculating unit 22a, and sets the upper rank of the similarities or the accumulated pitch pattern whose similarity is equal to or greater than the threshold as the similar pitch pattern.

  The pitch pattern dictionary 23 is a pitch pattern in which corrections such as extraction errors and pitch fluctuations are corrected for the real voice pitch information extracted from the voice data, and text data corresponding to the pitch pattern in which the maintenance is performed. This is a memory in which a plurality of (notation, reading, etc.) are registered. The pitch pattern stored in the pitch pattern dictionary 23 is referred to as an accumulated pitch pattern.

The similar pitch pattern presentation unit 24 outputs the similar pitch pattern input from the similar pitch pattern search unit 22 to the pitch pattern output unit 25. Furthermore, the similar pitch pattern presentation unit 24 includes an output conversion unit 24a, converts the similar pitch pattern into a format that can be confirmed by the user, and outputs the similar pitch pattern to the output device 3 as an output similar pitch pattern.
As a format that can be confirmed by the user, there is a display of a table in which the time change of the pitch pattern is quantified or an illustrated graph. Also, a synthesized speech using a pitch pattern is generated using a known speech synthesizer and is output to the output device 3 where a synthesized speech such as a speaker can be viewed, or the pitch pattern for the speech synthesizer is synthesized with a speech synthesis function and speech. For example, it can be appropriately configured to output to the output device 3 having an output function.
The output similar pitch pattern output to the output device 3 is presented to the user via the similar pitch pattern output unit 31. The user selects an optimum pitch pattern from the presented output similar pitch patterns, and inputs a pitch pattern selection result via the pitch pattern selection unit 13.

  The pitch pattern output unit 25 selects and outputs a pitch pattern to be output from the similar pitch patterns input from the similar pitch pattern presentation unit 24 based on the pitch pattern selection information input from the pitch pattern selection unit 13. .

  The output device 3 includes a similar pitch pattern output unit 31 and is configured by, for example, a device having a function of displaying the illustrated graph. The similar pitch pattern for output input from the similar pitch pattern presentation unit 24 is presented to the user.

Next, the operation of the speech synthesis system 100 will be described. FIG. 3 is a flowchart showing the operation of the pitch (prosodic) pattern generation device according to Embodiment 1 of the present invention. Hereinafter, description will be given according to this flowchart.
Voice data and text data are input from the voice input unit 11 and the text input unit 12 to the real voice pitch information extraction unit 21 (step ST1). The real voice pitch information extraction unit 21 analyzes the input voice data and extracts pitch information (step ST2). The pitch information is, for example, data obtained by extracting the pitch at regular time intervals. Also, the extraction interval is not limited to a certain time interval, and can be changed as appropriate according to the application and data, such as several representative points obtained by dividing the voiced sound interval into equal intervals, points where the pitch changes rapidly, and the like.

  Further, the real voice pitch information extraction unit 21 associates the extracted pitch information with the text data, and outputs it to the similar pitch pattern search unit 22 as real voice pitch information (step ST3). When the real voice pitch information is input in step ST3, the similarity calculation unit 22a of the similar pitch pattern search unit 22 reads the stored pitch pattern from the pitch pattern dictionary 23, and the read stored pitch pattern and the input real voice pitch information. Is calculated (step ST4). Further, the similar pitch pattern search unit 22 searches for one or more accumulated pitch patterns similar to a portion of the real voice pitch information (for example, a unit of continuous voiced sound segments) or the whole based on the similarity calculated in step ST4, It outputs to the similar pitch pattern presentation part 24 as a similar pitch pattern (step ST5).

  The similar pitch pattern presentation unit 24 outputs the similar pitch pattern input in step ST5 to the pitch pattern output unit 25, and the output conversion unit 24a converts the similar pitch pattern into a format that can be confirmed by the user. It outputs to the similar pitch pattern output part 31 of the output device 3 as a pattern, and the searched similar pitch pattern is shown to a user (step ST6).

  Thereafter, the user selects a desired pitch pattern from the output similar pitch patterns presented by the similar pitch pattern output unit 31, and the pitch pattern selection information as a selection result is sent via the pitch pattern selection unit 13 to the pitch pattern output device 3. (Step ST7). The pitch pattern output unit 25 selects and outputs a pitch pattern to be output from the similar pitch patterns input in step ST5 based on the pitch pattern selection information input in step ST7 (step ST8), and performs processing. finish.

Next, another configuration example of the real voice pitch information extraction unit 21 is shown. FIG. 4 is a block diagram illustrating another configuration example of the real voice pitch information extraction unit.
The real voice pitch information extraction unit 21 may be additionally provided with an additional information acquisition unit 21a that performs linguistic analysis of text data to acquire additional information and gives it to the real voice pitch information. Specifically, when text data with only notation is input, the text data is subjected to language analysis, and additional information such as reading and part-of-speech information is acquired and given to the real voice pitch information.

  Furthermore, as another configuration example, a segmentation unit 21b is added to the real voice pitch information extraction unit 21, for example, segmenting utterance speech data for each phoneme as phoneme information using a known speech recognition technique from input text data. May be provided. The real voice pitch information extraction unit 21 extracts real voice pitch information associated with the phoneme information obtained in the segmentation unit 21b.

Furthermore, as another configuration example, a pitch pattern generation specifying unit 21c that specifies a location where a pitch pattern is to be generated from audio data may be additionally provided in the real voice pitch information extracting unit 21. More specifically, for example, when generating the pitch pattern of the “interchange” part in the generation of the synthesized speech “this interchange”, the pitch pattern designation unit for the utterance data of the user “this interchange”. Specifies that only the “interchange” part is to be extracted from the real voice pitch information.
In the example of FIG. 4, a configuration in which the additional information acquisition unit 21a, the segmentation unit 21b, and the pitch pattern generation designation unit 21c are all provided at the same time is shown, but it is not necessary to provide all at the same time. You can do it.

Next, details of the similarity calculation unit 22a of the similar pitch pattern search unit 22 and an example of a calculation method will be described.
When the data format of the real voice pitch information and the accumulated pitch pattern is different (for example, the real voice pitch information calculates the pitch every 5 ms, while the accumulated pitch pattern calculates the pitch every 50 ms. In other words, the similarity is calculated after converting the data format of the real voice pitch information into the data format of the accumulated pitch pattern.

  For calculating the distance and likelihood of the similarity calculation unit 22a, for example, a square error or inner product between pitch patterns after normalizing the pitch pattern to a predetermined n-order vector, DP (dynamic planning), etc. Method) Processing such as a matching method and a statistical method using an HMM (Hidden Markov Model) can be applied. Since the DP matching method and the statistical method using the HMM are publicly known, detailed description thereof is omitted. For example, when using HMM, create HMM for real voice pitch information with one or more number of states and Gaussian distribution for each voiced sound section (or unit of phoneme, syllable, word, sentence, etc.) And used to calculate likelihood.

Next, a similarity calculation method will be exemplified.
<Example of calculation method>
1. The similarity is calculated by focusing on the accumulated pitch pattern in which the number of real voice pitch information and the number of voiced sound sections are the same.
2. A plurality of voiced sound sections in the real voice pitch information are concatenated to calculate the similarity as one voiced sound section.
3. A plurality of voiced sound sections of the accumulated pitch pattern are concatenated to calculate the similarity as one voiced sound section.
Calculation method example 2 above. And 3. Is that a voiced / unvoiced utterance error or pitch extraction error may result in the division of one voiced sound segment, or two voiced sound segments. It is taken into consideration.
4). The similarity is calculated from the distance and likelihood of the voiced sound section excluding the voiced sound section estimated to have partial voiced / unvoiced utterance errors and pitch extraction errors (for example, the distance is short or the likelihood is low) Only high-order voiced segments are used to calculate similarity).
5. The unit for calculating the distance and likelihood is not only a voiced sound section but a fixed time unit. If real voice pitch information and accumulated pitch patterns separated by phonemes, syllables, words, and sentences are obtained, calculation is made using the distance and likelihood in those units.

Next, another configuration example of the search operation of the similar pitch pattern search unit 22 is shown.
In addition to searching for similar pitch patterns using real voice pitch information, the similar pitch pattern detection unit inputs text data of uttered speech, voiced / unvoiced sound information, phoneme information, syllable information, part of speech information, language It is also possible to perform a search combining the appearance positions of the voices, and to perform a search from a linguistic storage pitch pattern that is assumed that the real voice pitch information and the storage pitch pattern are likely to match. For example, by searching from the stored pitch pattern of text with the same consonant type (burst consonant, frictional consonant, etc.), the pitch pattern transition between consonant vowels is similar. A pitch pattern with a sense of convergence is selected from the pitch patterns.

Furthermore, the similar pitch pattern search unit 22 can be configured to be used for the similar pitch pattern search after adjusting the accumulated pitch pattern. An example of adjustment is shown below.
1. A similar pitch pattern is searched after the accumulated pitch pattern is stretched and deformed on the time axis in accordance with the time length of the input voice. Thereby, a similar pitch pattern can be searched from pitch patterns having different speaking speeds.
2. After adjusting the average pitch of the accumulated pitch patterns in accordance with the average pitch of the real voice pitch information, the similar pitch pattern is searched. As a result, a similar pitch pattern can be retrieved from pitch patterns with different voice pitches.
3. A similar pitch pattern is searched after adjusting the pitch change width of the stored pitch pattern in accordance with the pitch change width of the real voice pitch information. As a result, a similar pitch pattern can be retrieved from pitch patterns having different tones.

Next, another configuration example of the similar pitch pattern presentation unit 24 is shown. FIG. 5 is a block diagram illustrating another configuration example of the similar pitch pattern presentation unit 24.
The output conversion unit 24a of the similar pitch pattern presenting unit 24 gives the text data of the pitch pattern to be generated in addition to the text data of the pitch pattern to be generated in the output of the similar pitch pattern for output, and combines them as a similar pitch pattern for output. An additional text data composition unit 24b may be provided. For example, if the text data of the generated pitch pattern is “Interchange” and the text data that is assumed to be used in synthesized speech as “Next Interchange” is input, the generated “Interchange” pitch pattern is changed. Generate a similar output pitch pattern that uses the next interchange.

  Further, the similar pitch pattern presentation unit 24 may be provided with a trigger input unit 24c that receives an instruction to select a similar pitch pattern input by the user. The similar pitch pattern presentation unit 24 outputs the selected similar pitch pattern to the output device 3 as an output similar pitch pattern.

  Further, the similar pitch pattern presentation unit 24 may be provided with a narrowing unit 24d for narrowing down the similar pitch pattern by the user. For example, the user narrows down the similar pitch pattern by using the narrowing unit 24d under conditions such as “the voice pitch is close to utterance” or “the part where the voice pitch changes is close to the utterance”, and outputs the pitch pattern after narrowing down Is output to the output device 3 as a similar pitch pattern.

  As described above, according to the first embodiment, the real voice pitch information extraction unit 21 extracts the real voice pitch information from the voice data based on the user's utterance, and the accumulated pitch stored in the real voice pitch information and the pitch pattern dictionary 23. The similarity calculation unit 22a that calculates the similarity from the pattern, the similarity pitch pattern search unit 22 that searches for one or more stored pitch patterns close to the real voice pitch information, and outputs the similar pitch pattern selected by the user as a pitch pattern Since the pitch pattern output unit 25 is provided, a pitch close to the real voice pitch information of the input voice data from the pitch pattern dictionary 23 storing a plurality of pitch patterns of the real voice subjected to correction such as extraction error correction. It is possible to search for patterns and generate highly natural pitch patterns. That.

  Moreover, according to this Embodiment 1, since it comprised so that the similar pitch pattern search part 22 which selects the accumulation | storage pitch pattern close | similar to the real voice pitch information of the input audio | voice data could be provided, similar texts, such as an unknown word and a new word Even when there is no data accumulation pitch pattern, a highly natural pitch pattern can be generated.

  Furthermore, according to the first embodiment, the pitch pattern dictionary 23 that stores in advance a plurality of real voice pitch patterns that have been corrected, such as correction of extraction errors of real voice pitch information extracted from voice data, is provided, and the input voice Since the similar pitch pattern search unit 22 for searching for a pitch pattern close to the real voice pitch information of the data is provided, a user who cannot obtain a stable utterance or pitch pattern as in the explanatory diagram shown in FIG. A pitch pattern that does not include deterioration can also be generated from real voice pitch information.

  Further, in the first embodiment, when the additional information acquisition unit 21a for analyzing the input text data and acquiring additional information is additionally provided in the real voice pitch information extraction unit 21, only the notation or the like is provided. Even when the input information is small, additional information such as reading and part-of-speech information useful for similar pitch pattern search and generation of synthesized speech in the output similar pitch pattern can be obtained.

  Further, in the first embodiment, when the real voice pitch information extraction unit 21 is additionally provided with a segmentation unit 21b that performs segmentation of utterance data for each phoneme using speech recognition technology from input text data. Since the similar pitch pattern search can be performed in units of phoneme information segmented in the similar pitch pattern search, the accuracy of similarity calculation can be improved.

  Further, in the first embodiment, when the real pitch information extraction unit 21 is additionally provided with a pitch pattern generation designation unit 21c that designates a location where a pitch pattern is desired to be generated from the voice data, it is actually generated. Since a sentence using the pitch pattern is uttered and the real voice pitch information can be extracted from the utterance considering the language environment before and after, it is possible to generate a highly natural pitch pattern with good connection between the front and back.

  Furthermore, in the first embodiment, in the search for similar pitch patterns, when similarity is calculated as a single segment by concatenating a plurality of voiced sound intervals of accumulated pitch patterns in the real voice pitch information or in the pitch pattern dictionary 23 Includes a pitch pattern that is close to the originally desired pitch pattern, even when voiced sections of real voice pitch information extracted due to partial voiced / unvoiced voicing errors or pitch extraction errors are divided or integrated. Searchable.

  Further, in the first embodiment, when similarities are calculated in similarity pitch pattern search, the distances and likelihoods of some sections are used without using the distances and likelihoods of all sections. Can calculate the similarity from the section excluding the section estimated to have partial voiced / unvoiced utterance errors and pitch extraction errors.

  Furthermore, in the first embodiment, in the search for the similar pitch pattern, when the text data is used together with the real voice pitch information for the search, the similarity between the real voice pitch information and the pitch pattern is high. The search target can be narrowed down to the text data in the pitch pattern dictionary 23 that is assumed to be easily obtained, and a pitch pattern similar in transition can be obtained by reducing the search processing amount or searching for an accumulated pitch pattern that is close to the text data. This makes it easier to search for similar pitch patterns.

  Furthermore, in the first embodiment, in the search for similar pitch patterns, when the configuration is such that the speaking speed, average pitch, pitch change width, etc. of the accumulated pitch pattern are processed according to the real voice pitch information, the accumulated pitch A pitch pattern close to the real voice pitch information can be generated even with real voice pitch information such as the opposite sex greatly different from the pattern.

  Furthermore, in this Embodiment 1, in addition to the text data of the pitch pattern to produce | generate, the text data before and behind that is given to the output conversion part 24a of the similar pitch pattern presentation part 24, and those pitch patterns are combined together for the output similar When the text data composition unit 24b configured as a pitch pattern is additionally provided, similar pitch patterns can be confirmed in consideration of the language environment before and after the sentence in which the actually generated pitch pattern is used. It is possible to select a similar pitch pattern that is well connected and highly natural.

  Furthermore, in the first embodiment, when the similar pitch pattern presenting unit 24 is additionally provided with a trigger input unit 24c for receiving selection of the similar pitch pattern output from the user, all the similar pitch patterns are output. Since it is not necessary to generate a similar pitch pattern, the amount of processing can be reduced. Further, when it is desired to confirm a similar pitch pattern again, the operation and procedure can be simplified.

  Further, in the first embodiment, when the similar pitch pattern presenting unit 24 is additionally provided with a narrowing unit 24d for narrowing down similar pitch patterns presented under the conditions desired by the user, the processing amount can be reduced and the user can be reduced. The similar pitch pattern can be easily selected.

  Furthermore, in the first embodiment, the output conversion unit 24a of the similar pitch pattern presenting unit 24 is configured to output the synthesized speech and output it in a format that can be confirmed by the user. The user can select the synthesized speech to be used in an auditory manner, and even a user who has no expertise or experience regarding speech information such as pitch can easily generate a pitch pattern.

  In the first embodiment, the pitch pattern has been described. However, the present invention is not limited to this, and for example, it can be performed on a parameter for controlling the pitch pattern.

  In the first embodiment, the pitch information and the pitch pattern that handle only the pitch as the pitch information and the pitch pattern have been described. However, in addition to the pitch, the pitch information that combines other pitches such as phonological continuation length and power, and A pitch pattern is also applicable.

  In the first embodiment, the description is made using Japanese as the input language, but the present invention can also be applied to other languages such as English and Chinese.

Embodiment 2. FIG.
In the pitch pattern generation device shown in the first embodiment, the configuration in which the real voice pitch information extracted by the real voice pitch information extraction unit 21 is used as it is for the similar pitch pattern search is shown, but in the second embodiment, the real voice pitch information is used. A configuration used for searching for a similar pitch pattern after processing is performed so as to approach the accumulated pitch pattern of the pitch pattern dictionary 23 is shown.

  FIG. 7 is a block diagram showing a configuration of a speech synthesis system having a pitch (prosody) pattern generation apparatus according to the second embodiment. The pitch pattern generation apparatus 2 according to the first embodiment includes a real voice pitch information processing unit (real voice prosody information). (Processing part) 26 is additionally provided. In the following description, the same or equivalent parts as those of the speech synthesis system according to the first embodiment are denoted by the same reference numerals as those used in the first embodiment, and description thereof is omitted or simplified.

  The real voice pitch information extraction unit 21 outputs the extracted real voice pitch information to the real voice pitch information processing unit 26. The real voice pitch information processing unit 26 processes the average pitch and the like so as to approximate the accumulated pitch pattern with respect to the real voice pitch information input from the real voice pitch information extraction unit 21, and sends the processed pitch information to the similar pitch pattern search unit 22. Output. The similar pitch pattern search unit 22 searches the pitch pattern dictionary 23 for a stored pitch pattern similar to the whole or part of the real voice pitch information based on the processed real voice pitch information input from the real voice pitch information processing unit 26. Then, the retrieved accumulated pitch pattern is output to the similar pitch pattern presentation unit 24 as a similar pitch pattern.

Next, a specific processing method of the real voice pitch information processing unit 26 will be described below.
A. The average pitch of the real voice pitch information is adjusted to match the average pitch of the pitch pattern dictionary 23 to be searched. Thereby, it is possible to search even when utterances having greatly different pitches such as the opposite sex are inputted.
B. The pitch change width of the real voice pitch information is adjusted according to the pitch change width of the pitch pattern dictionary 23 to be searched. As a result, even utterances with greatly different pitches such as the opposite sex can be searched.
C. The average time length of the real voice pitch information (speech) is expanded or contracted on the time axis according to the average time length of the utterance in the pitch pattern dictionary 23 to be searched. As a result, it is possible to search even utterances with greatly different speaking speeds.
D. When the user intentionally changes the prosody (higher or lower, faster or slower, etc.) instead of processing based on the average value or range of the real voice pitch information as in A to C above. In consideration of the above, the real voice pitch information may be processed based on a preset parameter.
E. The pitch considered to be an extraction error in the input real voice pitch information is corrected to obtain processed real voice pitch information.
For example, comparison with surrounding pitches, sections of pitches that are assumed to be double pitch or half pitch from the user's range, sections in which pitches are locally extracted by devoicing or voice leakage, and voice For sections where the extracted real voice pitch information is estimated to be abnormal, such as sections where the pitch fluctuates due to tremors, interpolation or averaging is performed from surrounding pitches, or when similar pitch pattern search unit 22 searches Set as a section not considered.
F. One or more pieces of processed real voice pitch information are generated using a conversion policy that compensates for pitch characteristics that are likely to deteriorate in the user's utterance in advance. For example, it is assumed that the inflection is too large (small) and the pitch pattern change width is small (large). The longer the sentence, the slower the speech speed. For example, the pitch pattern dictionary 23 is processed so as to maintain a high pitch from the beginning to the end of the word, assuming a dark utterance.
G. In addition to the real voice pitch information input from the real voice pitch information extraction unit 21, text data such as utterance reading, voiced / unvoiced information, and phoneme duration is input, and interpolation processing combining the text data is performed. For example, if there is a section with no pitch in the vowel chain section that is originally voiced utterance from text data, it is determined that the pitch is missing or is an extraction error, and pitch interpolation is performed.

Next, the operation of the pitch pattern generation apparatus according to the second embodiment will be described with reference to the flowchart of FIG. Note that the same reference numerals as those used in FIG. 3 are assigned to steps performing the same processing as in the first embodiment, and description thereof will be omitted or simplified.
The real voice pitch information extraction unit 21 associates the extracted pitch information with the text data input from the text input unit 12, and outputs it to the real voice pitch information processing unit 26 as real voice pitch information (step ST11).

The real voice pitch information processing unit 26 analyzes the real voice pitch information input in step ST11, and performs a pitch pattern dictionary 23 such as correction of an average pitch, correction of a change width, and correction of a portion estimated to be an extraction error. The processing is performed so as to be close to the stored accumulated pitch pattern, and is output to the similar pitch pattern search unit 22 as processed real voice pitch information (step ST12). When the processed real voice pitch information is input in step ST12, the similarity calculation unit 22a of the similar pitch pattern search unit 22 reads the stored pitch pattern from the pitch pattern dictionary 23, and the read processed pitch is input to each stored pitch pattern. The similarity of pitch information is calculated (step ST13).
Since the subsequent processing is the same as that of the first embodiment, description thereof is omitted.

  As described above, according to the second embodiment, the real voice pitch information processing unit 26 that performs processing for bringing the real voice pitch information closer to the accumulated pitch pattern in the pitch pattern dictionary 23 is provided, and the processed real voice pitch information is converted into the similar pitch pattern. Since it is configured to be used for the search, even the user's real voice pitch information that is greatly different from the accumulated pitch pattern in the pitch pattern dictionary 23 such as the opposite sex and the tone can be used for the similar pitch pattern search.

  In addition, according to the second embodiment, the pitch that is considered to be an extraction error in the input real voice pitch information is corrected and used as a similar pitch pattern search as processed real voice pitch information. It is possible to improve the search accuracy of the similar pitch pattern from the user who cannot obtain the pitch and the real voice pitch information in the recording environment. Thereby, a user's burden, such as re-reading, can be reduced.

  Further, according to the second embodiment, in the processing of the real voice pitch information, the processing is not performed based on the average value or the change width of the real voice pitch information, but is processed based on a preset parameter. Therefore, even when the user intentionally changes the pitch (higher or lower, faster or slower) and uses it for similar pitch pattern search after processing that reflects the change. can do.

  Further, according to the second embodiment, since the configuration is such that the real voice pitch information is processed using the conversion policy that compensates for the pitch characteristics that are likely to deteriorate in the user's utterance (for example, the inflection tends to be large), the user can The real voice pitch information can be brought close to the accumulated pitch pattern in the pitch pattern dictionary 23 without changing the normal utterance method.

  Further, according to the second embodiment, in addition to the real voice pitch information, text data such as utterance reading, voiced / unvoiced information, and phoneme duration is input, and correction processing is performed in combination with the text data. Therefore, it is possible to detect the presence or absence of pitch extraction error from the mismatch of the number of unvoiced sound sections or voiced sound sections, and to identify and estimate the location of pitch extraction errors in the voiced sound section from the phoneme duration, improving the correction accuracy Can be made.

  In the second embodiment, the configuration in which the real voice pitch information processing unit 26 processes the real voice pitch information is shown. However, a user may provide means for processing the real voice pitch information. As a result, the user himself / herself can appropriately correct the real voice pitch information accompanying the utterance error or the pitch extraction error.

  In the second embodiment, a configuration is shown in which one processing is performed on the real voice pitch information. However, a plurality of processing may be performed on the real voice pitch information. Thereby, the number and variation of the similar pitch patterns searched in the similar pitch pattern search increase, and it becomes easy to generate a pitch pattern close to the user's desire.

  Although the second embodiment has been described using the pitch pattern, the present invention is not limited to this, and for example, it can be performed on a parameter for controlling the pitch pattern.

  In the second embodiment, the pitch information and the pitch pattern that handle only the pitch as the pitch information and the pitch pattern have been described. However, in addition to the pitch, the pitch information that combines other pitches such as phonological continuation length and power, and A pitch pattern is also applicable.

Embodiment 3 FIG.
In the first and second embodiments, the configuration in which the similar pitch pattern searched by the similar pitch pattern searching unit 22 is presented to the user as it is or is output as the pitch pattern is shown. In the third embodiment, the similar pitch pattern search is performed. A configuration in which the similar pitch pattern searched by the unit 22 is processed to be closer to real voice pitch information and then presented to the user or output as a pitch pattern is shown.

  FIG. 9 is a block diagram showing the configuration of the pitch (prosody) pattern generation apparatus according to the third embodiment. A pitch pattern candidate generation unit (prosody pattern candidate generation unit) 27 is added to the pitch pattern generation apparatus 2 of the second embodiment. It is additionally provided. In the following description, the same reference numerals as those used in the first and second embodiments are given to the same or corresponding parts as the constituent elements of the pitch pattern generating apparatus according to the first and second embodiments. The description is omitted or simplified.

  Based on the real voice pitch information input from the real voice pitch information extraction unit 21, the similar pitch pattern search unit 22 searches the pitch pattern dictionary 23 for one or more accumulated pitch patterns that are similar to the part or the whole of the real voice pitch information. The retrieved accumulated pitch pattern is output to the pitch pattern candidate generation unit 27 as a similar pitch pattern. The pitch pattern candidate generation unit 27 processes the similar pitch pattern input from the similar pitch pattern search unit 22, generates one or more pitch pattern candidates that are close to the real voice pitch information of the input voice data, and creates a new one. One of the similar pitch patterns is output to the similar pitch pattern presentation unit 24.

Next, the operation of the pitch pattern generation apparatus according to the third embodiment will be described with reference to the flowchart of FIG. Note that the same reference numerals as those used in FIG. 8 are attached to steps for performing the same processing as in the second embodiment, and description thereof will be omitted or simplified.
Based on the similarity calculated in step ST13, the similar pitch pattern search unit 22 searches for one or more pitch patterns that are similar to a part of the real voice pitch information (for example, a unit of continuous voiced voice segments) or the whole, and the similar pitch It outputs to the similar pitch pattern candidate production | generation part 27 as a pattern (step ST21).

The pitch pattern candidate generating unit 27 processes the similar pitch pattern input in step ST21, generates a similar pitch pattern close to the real voice pitch information, and outputs the similar pitch pattern to the similar pitch pattern presenting unit 24 (step ST22).
Since the subsequent processing is the same as in the first and second embodiments, description thereof is omitted.

Next, a specific example of the pitch pattern candidate generation process is shown. FIG. 11 is an explanatory view showing pitch pattern candidate generation processing of the pitch (prosodic) pattern generation device according to the third embodiment.
In the pitch pattern candidate generation process, for example, a plurality of similar pitch patterns are connected, and one or more pitch pattern candidates close to the real voice pitch information of the audio data input to the input device 1 are generated. In the example of FIG. 11, when the generated real voice pitch information “interchange” is inputted, the similar pitch pattern search unit 22 reads the pitch voice information of “inter” corresponding to the first and second voiced sound sections from the pitch pattern dictionary 23. A similar pitch pattern of utterances having two voiced sound segments of “performance” similar to the “sound” and two voiced sound segments of “heights” similar to the real voice pitch information of “change” corresponding to the third and fourth voiced sound segments A similar pitch pattern of the utterance possessed is searched and output to the pitch pattern candidate generation unit 27. The pitch pattern candidate generation unit 27 connects the two input similar pitch patterns, and obtains a new similar pitch pattern close to the real voice pitch information “interchange”.

In addition to connecting similar pitch patterns based on the number of voiced sound sections shown in FIG. 11, the following processing method can be applied.
a. If the real voice pitch information divided in units of phonemes, syllables, words, and sentences and the pitch patterns in the pitch pattern dictionary 23 are obtained, similar pitch patterns are connected using these units.
b. In addition to connecting the pitch pattern as it is, the pitch pattern may be connected after the smoothing process or the interpolation process is performed so that the transition of the connection portion is smooth and the pitch is changed.
c. The connection is made after deformation so that the average pitch of similar pitch patterns to be connected and the speaking speed of the change width of the pitch are improved.
d. In addition to the text data of the pitch pattern to be generated, the text data before and after the text data are given, and the pitch of the connection portion between the similar pitch pattern and the front and back pitch patterns is subjected to processing such as smoothing or smoothing processing.
e. The user provides means for correcting the similar pitch pattern, and performs, for example, local pitch correction or interpolation by weighted addition of the real voice pitch information and the similar pitch pattern.
f. Using the user's pointed location (time information, phonological information, etc.) and the user's pointed content (accent position, voice pitch, intonation, etc.) as input, process similar pitch patterns with differences in the pointed location and pointed content .

  As described above, according to the third embodiment, the similar pitch pattern searched by the similar pitch pattern search unit 22 is combined with the partial similar pitch pattern to approximate the real voice pitch information of the input voice data. Since the pitch pattern candidate generation unit 27 is set to be one of the similar pitch patterns, the variation of the similar pitch patterns is increased even in the pitch pattern dictionary of the same scale, and the similar pitch close to the real voice pitch information. A pattern can be obtained more easily.

  Further, according to the third embodiment, when combining similar partial pitch patterns, if the connection is made after changing the pitch so that the transition of the connection portion is smooth, Discontinuity caused by connection of similar pitch patterns of utterances in different environments can be eliminated.

  Further, according to the third embodiment, when partial similar pitch patterns are combined, the average pitch of each similar pitch pattern, the change width of the pitch, and the speech speed are deformed so as to improve the connection and then connected. In such a case, a similar pitch pattern having a good connection and high naturalness can be generated even in the connection of similar pitch patterns having high similarity to the real voice pitch information but different inflections.

  Further, according to the third embodiment, in the generation of pitch pattern candidates, in addition to the text data of the similar pitch pattern to be generated, the text data before and after the text data is given, and the connection portion between the generated similar pitch pattern and the front and rear pitch patterns If the pitch is processed so as to improve the connectivity, the similar pitch with good connection and naturalness will be taken into consideration even before and after the language environment in the sentence where the actually generated pitch pattern is used A pattern can be generated.

  Further, according to the third embodiment, when the pitch pattern candidate is generated so that the user includes a means for correcting the similar pitch pattern, the pitch pattern desired by the user can be easily generated.

  Here, when the pitch pattern generated by the pitch pattern candidate generation unit 27 is selected, the pitch pattern dictionary 23 may store the pitch pattern and use it as a new pitch pattern for the search. As the number of pitch pattern variations increases, a pitch pattern reflecting the user's personality can be added to the pitch pattern dictionary 23.

  In the third embodiment, the pitch pattern candidate generation unit 27 is additionally provided in the pitch pattern generation apparatus of the second embodiment. However, the pitch pattern candidate generation is performed in the pitch pattern generation apparatus of the first embodiment. A portion 27 may be additionally provided.

  In the third embodiment, the pitch pattern has been described. However, the present invention is not limited to this, and for example, it can be performed on a parameter for controlling the pitch pattern.

  In the third embodiment, the pitch information and the pitch pattern that handle only the pitch are described as the pitch information and the pitch pattern. However, in addition to the pitch, the pitch information that combines other pitches such as phonological continuation length and power, and A pitch pattern is also applicable.

  DESCRIPTION OF SYMBOLS 1 Input device, 2 Pitch pattern production | generation apparatus, 3 Output device, 11 Voice input part, 12 Text input part, 13 Pitch pattern selection part, 21 Real voice pitch information extraction part, 21a Additional information acquisition part, 21b Segmentation part, 21c Pitch pattern Generation designation unit, 22 Similar pitch pattern search unit, 22a Similarity calculation unit, 23 Pitch pattern dictionary, 24 Similar pitch pattern presentation unit, 24a Output conversion unit, 24b Text data synthesis unit, 24c Trigger input unit, 24d Narrowing unit, 25 Pitch pattern output unit, 26 Real voice pitch information processing unit, 27 Pitch pattern candidate generation unit, 31 Similar pitch pattern output unit, 100 Speech synthesis system.

Claims (3)

A real voice prosody information extraction unit that accepts input of voice data and text data, extracts prosody information from the voice data, and generates real voice prosody information in which the prosody information is associated with the text data;
A prosodic pattern dictionary storing a plurality of prosodic patterns;
A similar prosodic pattern search unit that searches one or more prosodic patterns similar to part or the whole of the real voice prosodic information from the prosodic pattern dictionary and outputs them as similar prosodic patterns;
The real voice prosody information input from the real voice prosody information extraction unit is processed to approximate the prosody pattern stored in the prosodic pattern dictionary, and is output to the similar prosody pattern search unit; ,
A similar prosodic pattern presenting unit that presents the similar prosodic pattern converted into a user-recognizable format and requests the user to select the similar prosodic pattern;
A prosodic pattern generation device comprising: a prosodic pattern output unit that outputs a similar prosodic pattern selected by the user among the similar prosodic patterns presented by the similar prosodic pattern presentation unit. Accepts input of speech data and text data and extracts prosodic information from the speech data
And generating real voice prosody information in which the prosody information is associated with the text data.
An information extractor;
A prosodic pattern dictionary storing a plurality of prosodic patterns;
A similar prosodic pattern search unit that searches one or more prosodic patterns similar to part or the whole of the real voice prosodic information from the prosodic pattern dictionary and outputs them as similar prosodic patterns;
The similar prosodic pattern is converted into a user-recognizable format and presented to the user.
A similar prosodic pattern presentation unit that requests selection of a similar prosodic pattern;
Processing the similar prosodic pattern, and the real voice prosody pattern candidates approximating to the prosodic information generated one or more prosodic pattern candidate generating unit that outputs the prosody pattern candidate to the similar prosodic pattern presenting unit as the similar prosodic pattern ,
Of the similar prosodic patterns presented by the similar prosodic pattern presentation unit, the user selects
Prosody pattern generator you comprising the prosodic pattern output unit for outputting the-option was similar prosodic patterns. 3. The prosodic pattern generation device according to claim 1, wherein the similar prosodic pattern presentation unit generates and presents synthesized speech based on the similar prosodic pattern.

Images