JP2008292587A - Rhythm creating device, rhythm creating method and rhythm creating program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a rhythm creating device, a rhythm creating method and a rhythm creating program, capable of crating a corrected rhythm pattern by correcting an extraction error of a voice rhythm pattern extracted from human utterance, without spoiling naturalness and expression of human utterance, and even without spending time and effort. <P>SOLUTION: The rhythm creating device 2 comprises: a pitch pattern extraction section 26b for extracting a voice pitch pattern which shows a rhythm of human voice from a voice data; and a correction rhythm creation section 27 for creating a correction pitch pattern on the basis of highly reliable extraction pattern by a pitch pattern extraction section 26b in the voice pitch pattern, instead of less reliable extraction pattern by the pitch pattern extraction section 26b in the voice pitch pattern, and a regular pitch pattern created by a pitch pattern creation section 24b. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention accepts an arbitrary text and a human voice that reads out the content of the text, and generates a prosody pattern based on the received arbitrary text and human voice, a prosody generation method, and , Prosody generation program.

  In recent years, speech synthesis technology for converting text into speech and outputting it has been used in various systems or apparatuses. For example, there are an IVR (Automatic Voice Response) system, an in-vehicle information terminal, an operation method guidance on a mobile phone, reading out an e-mail, a support system for visually handicapped and speech handicapped. In such a speech synthesis technology, it is difficult to generate a synthesized speech that is natural as human speech and rich in expressiveness.

  In other words, the prosody of synthesized speech is generally based on linguistic analysis such as morphological analysis that analyzes word reading and parts of speech in text, analysis of clauses and dependency, accent settings, intonation settings, pause and speech speed settings, etc. To be determined. However, with the current processing technology, it is difficult to perform an analysis that takes into account the meaning of the sentence and the context before and after like a human being, and the analysis result may include an error. For this reason, the synthesized speech generated by the speech synthesis technique may include portions where the prosody that determines how to speak, such as voice pitch, intonation, and rhythm, is unnatural compared to human speech.

Therefore, as a method of improving the quality of the synthesized speech prosody, when the text to be synthesized is determined in advance, the speech prosodic pattern is extracted from the human speech and the synthesized speech is generated using the extracted speech prosodic pattern as it is. Methods are known (see, for example, Patent Documents 1 to 4). In this method, extraction of the human utterance and its speech prosody pattern is required in advance, but since the synthesized speech is generated using the speech prosody pattern extracted from the human utterance, it is natural as human speech, Synthetic speech rich in expressiveness can be generated.
JP-A-10-153998 JP-A-9-292897 Japanese Patent Laid-Open No. 11-14383 JP-A-7-140996

  However, in the above-described conventional method, when the extraction accuracy of the speech prosody pattern extracted from the human utterance is low, that is, when the extraction error of the speech prosody pattern occurs, the problem is that the prosody becomes an unnatural synthetic speech. Produce.

  Specifically, in order to extract speech prosodic patterns from human speech, phoneme labeling technology that detects the start and end points of each phoneme in human speech, and the pitch at each time during human speech are detected. A pitch extraction technique is required. Various excellent methods have been developed for these techniques. However, since human utterances are very diverse and irregular, it is impossible to extract speech prosodic patterns with 100% accuracy. For this reason, the user needs to correct the extraction error of the speech prosodic pattern using a GUI device or the like. This work requires specialized knowledge about audio, and is time consuming and time consuming.

  The present invention has been made in view of the above-mentioned problems, and its purpose is to extract a speech prosody pattern extracted from a human utterance without impairing the naturalness and expressiveness of the human utterance. In addition, it is an object of the present invention to provide a prosody generation device, a prosody generation method, and a prosody generation program capable of generating a modified prosody pattern by correcting without taking time and effort.

  In order to achieve the above object, the prosody generation device according to the present invention generates a phonetic character string data indicating a reading of the text by performing a text analysis on the text input unit to which an arbitrary text is input and the text. Based on a language processing unit, the phonetic character string data, and a prosody generation rule, a regular prosody generation unit that generates a regular prosody pattern indicating the prosody of the text, and voice data of human speech read out from the text A speech input unit for converting to speech, a speech prosody extraction unit for extracting a speech prosody pattern indicating the prosody of the human speech from the speech data, and a speech prosody extraction unit for extracting the speech prosody pattern from the speech data In the speech prosody pattern, the speech prosody extraction unit extracts a pattern having the reliability greater than or equal to a threshold value. A reliability determination unit that determines a pattern with high extraction reliability and determines a pattern with a reliability less than a threshold among the speech prosodic patterns as a pattern with low extraction reliability by the speech prosody extraction unit; Based on the regular prosodic pattern and the regular prosody pattern of the speech prosody pattern, which are highly reliable in extraction by the speech prosody extractor, instead of the pattern of the prosody pattern which is not reliable by the speech prosody extractor. A modified prosody generation unit for generating a modified prosody pattern is provided. The regular prosodic pattern, the speech prosodic pattern, and the modified prosodic pattern are, for example, pitch patterns representing a voice pitch change pattern.

  According to the prosody generation device of the present invention, the modified prosody pattern generated by the modified prosody generation unit has high extraction reliability by the speech prosody extraction unit instead of the pattern with low extraction reliability by the speech prosody extraction unit. It is a pattern generated based on a pattern and a regular prosodic pattern. That is, the modified prosody pattern generated by the modified prosody generation unit does not use a pattern with low extraction reliability by the speech prosody extraction unit, and a pattern with high extraction reliability by the speech prosody extraction unit and an appropriate rule. It is a pattern generated based on the prosodic pattern. This makes it possible to correct errors in the extraction of speech prosodic patterns extracted from human utterances without compromising the naturalness and expressiveness of human utterances and without taking time and effort. Can be generated.

  In the prosody generation device according to the present invention, the modified prosody generation unit modifies the regular prosody pattern so as to approximate a pattern with high reliability of extraction by the speech prosody extraction unit of the speech prosody pattern, It is preferable to include a prosody complementing unit that generates a modified prosody pattern by connecting the regular prosody pattern and a pattern having high extraction reliability by the speech prosody extraction unit among the speech prosody patterns.

  According to the above configuration, the modified prosodic pattern generated by the prosody complementing unit is transformed into an appropriate regular prosody pattern so as to approximate a pattern with high reliability of extraction by the speech prosody extracting unit, A pattern generated by connecting a pattern with high extraction reliability by the speech prosody extraction unit. This makes it possible to correct errors in the extraction of speech prosodic patterns extracted from human utterances without compromising the naturalness and expressiveness of human utterances and without taking time and effort. Can be generated.

  In the prosody generation device according to the present invention, the modified prosody generation unit modifies the regular prosody pattern so as to approximate a pattern with high reliability of extraction by the speech prosody extraction unit of the speech prosody pattern, It is preferable to include a prosody modification unit that generates a modified prosody pattern by using a modified regular prosody pattern without using a pattern with high extraction reliability by the speech prosody extraction unit among the speech prosody patterns. .

  According to the above configuration, the modified prosody pattern generated by the prosody modification unit is modified by an appropriate regular prosody pattern so as to approximate a pattern with high reliability of extraction by the speech prosody extraction unit, and extracted by the speech prosody extraction unit. This is a pattern generated by using a modified regular prosodic pattern without using a pattern with high reliability. This makes it possible to correct errors in the extraction of speech prosodic patterns extracted from human utterances without compromising the naturalness and expressiveness of human utterances and without taking time and effort. Can be generated.

  To achieve the above object, a prosody editing system according to the present invention includes the prosody generation device and a GUI device that edits at least one of the phonetic character string data and the modified prosody pattern generated by the prosody generation device. It is characterized by that.

  According to the prosody editing system of the present invention, the GUI device edits at least one of the phonetic character string data and the modified prosody pattern generated by the prosody generating device, so that the phonetic character string generated by the prosody generating device is edited. The user can make fine adjustments to at least one of the data and the modified prosodic pattern.

  To achieve the above object, a speech synthesis system according to the present invention includes the prosody generation device and a speech synthesis device that generates and outputs synthesized speech based on the modified prosodic pattern generated by the prosody generation device. It is characterized by that.

  According to the speech synthesis system of the present invention, the speech synthesizer generates and outputs a synthesized speech based on the modified prosodic pattern generated by the prosody generation device. Therefore, the output synthesized speech is a natural speech possessed by a human utterance. Synthetic speech with sex and expressive power.

  In order to achieve the above object, a speech synthesis system according to the present invention includes the prosody generation device, a GUI device that edits at least one of the phonetic character string data and the modified prosody pattern generated by the prosody generation device, And a speech synthesizer that generates and outputs synthesized speech based on at least one of the modified prosodic pattern generated by the prosody generating device and the modified prosodic pattern edited by the GUI device. .

  According to the speech synthesis system of the present invention, the speech synthesizer generates and outputs a synthesized speech based on at least one of the modified prosodic pattern generated by the prosody generating device and the modified prosodic pattern edited by the GUI device. The output synthesized speech is a synthesized speech having the naturalness and expressiveness that human speech has.

  In order to achieve the above object, in the prosody generation method according to the present invention, a text input unit included in a computer performs a text input process in which an arbitrary text is input, and a language processing unit included in the computer performs language analysis on the text. Thus, a language processing step for generating phonetic character string data indicating the reading of the text, and a regular prosody generation unit provided in the computer are based on the phonetic character string data and statistical prosody data. A regular prosody generation step of generating a regular prosody pattern indicating the prosody of the text, a speech input unit included in the computer, a speech input step of converting a human speech read out from the text into speech data, and the computer A speech prosody extraction unit comprising a speech prosody parameter indicating the prosody of the human speech from the speech data; A speech prosody extraction step for extracting a speech pattern, and a reliability determination unit included in the computer obtains the reliability of the extraction when the speech prosody pattern is extracted from the speech data in the speech prosody extraction step And determining a pattern having a reliability greater than or equal to a threshold value among the speech prosodic patterns as a pattern having a high extraction reliability by the speech prosody extraction step, and selecting a pattern having the reliability less than the threshold among the speech prosodic patterns A reliability determination step for determining a pattern with low extraction reliability by the phonetic prosody extraction step, and a pattern with low extraction reliability by the phonetic prosody extraction step among the phonetic prosody patterns by the modified prosody generation unit included in the computer Instead of the pattern of the phonetic prosody pattern that is highly reliable for extraction by the phonetic prosody extraction step, and Characterized in that it comprises a modified prosody generation step of generating a modified prosody pattern based on the serial rule prosody pattern.

  In order to achieve the above object, the prosody generation program according to the present invention generates a phonetic character string data indicating a reading of the text by performing a text input process in which an arbitrary text is input and language analysis of the text. Based on language processing, the phonetic character string data, and data on statistical prosody, regular prosody generation processing for generating a regular prosody pattern indicating the prosody of the text, and human speech read out from the text The speech prosody pattern is extracted from the speech data by speech input processing for converting to speech data, speech prosody extraction processing for extracting speech prosody patterns indicating the prosody of the human speech from the speech data, and speech prosody extraction processing. The reliability of the extraction at the time of extraction is acquired, and the reliability of the speech prosodic pattern is equal to or greater than a threshold value A turn is determined as a pattern with high extraction reliability by the speech prosody extraction process, and a pattern whose reliability is less than a threshold is determined as a pattern with low extraction reliability by the speech prosody extraction process. A pattern having a high reliability of extraction by the speech prosody extraction process among the speech prosodic patterns, instead of a pattern having a low reliability of the extraction by the speech prosody extraction process of the speech prosody pattern, and The computer is caused to execute a modified prosody generation process for generating a modified prosody pattern based on the regular prosody pattern.

  It should be noted that the prosody generation method and prosody generation program according to the present invention achieve the same effects as the above-mentioned prosody generation apparatus.

  As described above, the prosody generation device, the prosody generation method, and the prosody generation program according to the present invention impair the naturalness and expressiveness of the human utterance due to the extraction error of the speech prosodic pattern extracted from the human utterance. In addition, there is an effect that it is possible to generate a modified prosodic pattern by correcting without taking time and effort.

  Hereinafter, more specific embodiments of the present invention will be described in detail with reference to the drawings.

[Embodiment 1]
FIG. 1 is a block diagram showing a schematic configuration of a speech synthesis system 1 according to the present embodiment. That is, the speech synthesis system 1 according to the present embodiment includes a prosody generation device 2 and a speech synthesis device 3. The prosody generation device 2 and the speech synthesis device 3 are connected to each other by wire or wireless. The prosody generation device 2 is a device that accepts an arbitrary text and a human voice that reads out the text, and generates a modified prosody pattern based on the received arbitrary text and the human voice. The speech synthesizer 3 is a device that receives the modified prosodic pattern generated by the prosody generating device 2 and generates and outputs synthesized speech based on the received modified prosodic pattern. The prosody generation device 2 and the speech synthesis device 3 are configured by general-purpose computers such as personal computers and server machines, for example. The prosody generation device 2 and the speech synthesizer 3 may be configured by, for example, a computer incorporated in an electronic device such as an in-vehicle information terminal, a mobile phone, or a home appliance. The prosody generation device 2 and the speech synthesis device 3 may exist in the same hardware, or may exist in different hardware.

(Configuration of prosody generation device)
The prosody generation device 2 includes a text input unit 21, a word dictionary 22, a language processing unit 23, a regular prosody generation unit 24, a speech input unit 25, a speech prosody extraction unit 26, and a modified prosody generation unit 27.

  An arbitrary text is input to the text input unit 21. In the present embodiment, it is assumed that the text input unit 21 has input text representing “Please push the push button according to the voice guidance”. For example, the text input unit 21 may accept text input from the user via an input device such as a keyboard or a mouse, or may accept text by reading data recorded in a memory provided in the computer. . The text input unit 21 outputs the input text to the language processing unit 23.

  The word dictionary 22 stores a plurality of word notations, readings, parts of speech, and accent information. Accent information is data indicating an accent type, for example. For example, when the prosody generation device 2 reads a recording medium on which word data is recorded, the word dictionary 22 stores the above-described word notation, reading, part of speech, and accent information.

  The language processing unit 23 performs morphological analysis on the text output from the text input unit 21 using the word dictionary 22. The text is divided into a plurality of words by performing morphological analysis using the word dictionary 22 in the language processing unit 23. FIG. 2 is a conceptual diagram showing a result of the morphological analysis performed on the text by the language processing unit 23 according to the present embodiment. As shown in FIG. 2, the language processing unit 23 generates a part of speech and a reading by using the word dictionary 22 for each divided word. Part of speech includes common nouns, verb conjunctions, adjectives, adjective verbs, case particles, connective particles, and the like. Here, common nouns, verb usage forms, adjectives, adjective verbs, etc. are classified as independent words. Case particles, connection particles, etc. are classified as appendages. Reading indicates the reading of a word. The reading includes an accent nucleus. Here, the accent nucleus is a position where the accent shifts from “high” to “low”. In the present embodiment, the accent nucleus is represented by a symbol “′”, for example, “O'N”. The morpheme analysis method includes, for example, the Viterbi algorithm and the longest match method, but the morpheme analysis method used in the present embodiment is not limited to a specific one.

  Further, the language processing unit 23 generates a plurality of clauses and their readings based on the result of morphological analysis performed on the text output from the text input unit 21. FIG. 3 is a conceptual diagram showing a plurality of clauses generated by the language processing unit 23 according to the present embodiment and their readings. As shown in FIG. 3, the language processing unit 23 generates four clauses “to voice guidance”, “accordingly”, “push button”, and “please press”. A phrase is an independent word followed by an appendix. For example, in the phrase “for voice guidance”, the common noun “speech” and the compound noun “voice guidance” of “guidance” are treated as one independent word, and then the case particle (adjunct). "" Is connected. In addition, the language processing unit 23 generates a reading by appropriately setting an accent nucleus for the generated phrase according to an arbitrary accent coupling rule. For example, the words “sound”, “guidance”, and “ni” are combined with the accents of “O'N'SE”, “GA'I DANCE”, and “NI”, and the phrase “ONSEGA 'IDANNI” is generated. Is done.

  Furthermore, the language processing unit 23 analyzes the dependency (modification) relationship between the plurality of generated clauses according to an arbitrary rule. In this embodiment, the language processing unit 23 specifies dependency relationships such as “follow voice guidance → follow”, “follow → press”, and “push push button →”.

  The language processing unit 23 generates phonetic character string data based on the results of language analysis such as morphological analysis and dependency analysis. The phonetic character string data is data indicating reading of text. In the present embodiment, the language processing unit 23 generates phonetic character string data indicating “ONSEGA 'IDANNI_SHITAGATTE, push button' TAO_OSITECHDASA '.”. Here, “_” is a symbol representing the boundary of an accent phrase. An accent phrase is a unit constituting an accent and generally corresponds to the above-mentioned phrase. “,” Is a symbol that represents the boundary of the accent phrase and also represents the boundary of the phrase. A phrase is a unit when syntactically analyzing a sentence or a clause, and consists of a plurality of words. In other words, in the present embodiment, “ONSEGA’IDANNI_SHITAGATTE” and “PUSHBO’TANO_Oshitekadasa ′” are each one phrase. “′” Is a symbol representing an accent nucleus. The format of the phonetic character string data is merely an example, and the way of expressing the phonetic character string data is not limited to this. The language processing unit 23 outputs the generated phonetic character string data to the regular prosody generation unit 24 and the speech prosody extraction unit 26.

  The regular prosody generation unit 24 converts the phonetic character string data output from the language processing unit 23 into a phoneme symbol string. In the present embodiment, the regular prosody generation unit 24 converts the phonetic character string data “ONSEGA’IDANNI_SHITAGATTE, push-bo” TANO_OSITECHDASA’i. Here, “Q” is a symbol representing a pose. “N” is a symbol representing “n”, and is represented by capital letters to distinguish it from “ni” which is a symbol representing “ni”. The regular prosody generation unit 24 generates a regular prosody pattern based on the converted phoneme symbol string. The regular prosody pattern includes a phoneme time length pattern, a regular pitch pattern, and a power pattern. For this reason, the regular prosody generation unit 24 includes a phoneme time length generation unit 24a, a pitch pattern generation unit 24b, and a power generation unit 24c.

  The phoneme time length generation unit 24a has a phoneme time length table in which data indicating statistical phoneme time lengths in human speech is recorded. The phoneme time length generator 24a extracts data from the phoneme time length table based on each phoneme of the phoneme symbol string, and generates a phoneme time length pattern by combining the extracted data. The phoneme time length table includes, for example, data indicating the phoneme time length of the phoneme “a”, data indicating the phoneme time length of the phoneme “i”, data indicating the phoneme time length of the phoneme “u”,. Are recorded in order.

The pitch pattern generation unit 24b generates a regular pitch pattern by superimposing the accent phrase component generated from the accent phrase on the phrase component generated from the phrase. FIG. 4 is a conceptual diagram showing a state in which an accent phrase component is superimposed on a phrase component. As shown in FIG. 4, the phrase component F _1, it is superimposed accent phrase component A ₁ and A _2, the phrase component F _2, accent phrase component A ₃ and A ₄ are superimposed. Here, the phrase components F ₁ and F ₂ are represented as a right-down triangular model. That is, generally speaking, a human voice is high at the beginning, but the voice pitch gradually decreases due to a decrease in subglottic pressure or the like. That is, the phrase components F ₁ and F ₂ are voice components that express the characteristic that the pitch decreases with time. The right-down triangle model is data relating to a statistical regular pitch pattern, and is recorded in advance in a memory (not shown) of the pitch pattern generation unit 24b.

Accent phrase components A _{1 to} A ₄ are represented as trapezoidal models. Here, for example, consider the case of the accent phrase component A ₁ . The phoneme symbol string “oNse-gaidaNsuni” corresponding to the accent phrase component A ₁ corresponds to the phonetic character string data “Onsega 'Idanni”. That is, in general, the voice of the part “Onsega” before the accent core is high and the voice of the part “Idannis” after the accent core is low in human speech. That is, the accent phrase component A ₁ is a component that represents a high characteristic of the phoneme symbol string “oNse-ga”. Similarly, the accent phrase component A ₂ is a component that represents a characteristic that the phoneme symbol string “shitagaqte” is high. The accent phrase component A ₃ is a component that represents a high characteristic of the phoneme symbol string “puqshbo”. The accent phrase component A ₄ is a component that represents a high characteristic of the phoneme symbol string “ositekudasa”. The trapezoidal model is data relating to a statistical regular pitch pattern, and is recorded in advance in a memory (not shown) of the pitch pattern generation unit 24b.

The pitch pattern generation unit 24b sets the pattern of the outer shape when the accent phrase component is superimposed on the phrase component as a regular pitch pattern. FIG. 5 is a conceptual diagram illustrating an example of a regular pitch pattern generated by the pitch pattern generation unit 24b according to the present embodiment. As shown in FIG. 5, regular pitch pattern, the phrase component F _1, is superimposed accent phrase component A ₁ and A _2, and the phrase component F _2, accent phrase component A ₃ and A ₄ are superimposed It is the pattern of the outer shape in the case.

  The power generation unit 24c has a power value table in which power values unique to each phoneme are recorded. The power value is data relating to statistical power and is a value representing the loudness of the voice. The power generation unit 24c extracts a power value from the power value table based on each phoneme in the phoneme symbol string. Here, in general, even for the same phoneme, the higher the regular pitch pattern, the larger the power value, and the lower the regular pitch pattern, the smaller the power value. The power generation unit 24c generates a power pattern by correcting the power value extracted from the power value table according to the level of the regular pitch pattern.

  That is, the regular prosodic pattern including the phoneme duration pattern, the regular pitch pattern, and the power pattern generated by the above method is a statistically valid prosodic pattern, but is an average prosodic pattern. The prosodic pattern is somewhat poor in expressiveness. The prosody generation unit 24 outputs a regular prosody pattern including a phoneme time length pattern, a regular pitch pattern, and a power pattern to the modified prosody generation unit 27. Note that the phoneme time length pattern, the regular pitch pattern, and the power pattern generation method are not limited to the above methods. Also, in the above, an example is shown in which statistical data is used to generate phoneme time length patterns, regular pitch patterns, and power patterns, but phoneme time length patterns based on prosodic generation rules generated heuristically. It is also possible to generate regular pitch patterns and power patterns.

  The voice input unit 25 has a function of receiving a human voice read out from the text received by the text input unit 21. For this reason, the voice input unit 25 is constituted by a microphone, for example. In the present embodiment, the voice input unit 25 receives a human voice that reads “Please push the push button according to the voice guidance.” The voice input unit 25 converts the received human voice into digital voice data that can be processed by a computer. The voice input unit 25 outputs the converted voice data to the voice prosody extraction unit 26. Note that the voice input unit 25 is a digital voice recorded on a recording medium such as a CD (Compact Disc) or an MD (Mini Disc) in addition to analog voice obtained by reproducing a human voice previously recorded by a recording device. Audio data or digital audio data transmitted via a wired or wireless communication network may be directly received. Further, the voice input unit 25 may have a function of expanding the compressed voice data when the received voice data is compressed.

  Similar to the regular prosody generation unit 24, the phonetic prosody extraction unit 26 converts the phonogram character string data output from the language processing unit 23 into a phoneme symbol string. In the present embodiment, the phonetic prosody extraction unit 26 converts the phonetic character string data “ONSEGA’IDANNI_SHITAGATTE, push-bo” TANO_OSITECHDASA’i. The speech prosody extraction unit 26 extracts a speech prosody pattern from the speech data output from the speech input unit 25 based on the converted phoneme symbol string. Note that the phonetic prosody pattern includes a phoneme time length pattern, a voice pitch pattern, and a power pattern. Therefore, the speech prosody extraction unit 26 includes a phoneme time length extraction unit 26a, a pitch pattern extraction unit 26b, a reliability determination unit 26c, and a power extraction unit 26d.

  The phoneme time length extraction unit 26a has a phoneme model in which data obtained by statistically modeling information on which phoneme is likely to be a feature amount is recorded. The phoneme time length extraction unit 26a extracts modeled data from the phoneme model based on each phoneme in the phoneme symbol string. The phoneme time length extraction unit 26a specifies a section of voice data that is most similar to the extracted data by collating the extracted data with the voice data. The phoneme time length extraction unit 26a extracts a phoneme time length pattern from speech data by setting a phoneme boundary in the specified section. Such an extraction method is generally called phoneme labeling. Note that the phoneme model is expressed using parameters such as MFCC (Mel Frequency Cepstral Coefficients). In general, the voice data output from the voice input unit 25 is also converted into a parameter such as MFCC and then verified by a matching method such as HMM (Hidden Markov Model) or DP (Dynamic Programming).

  The pitch pattern extraction unit 26b extracts an audio pitch pattern from the audio data by using a correlation processing method. Here, the correlation processing method is a method utilizing the fact that the correlation processing is strong against waveform phase distortion. In the present embodiment, a case where an autocorrelation function (ACF) is used as an example of a correlation processing method will be described, but the present invention is not limited to this. For example, instead of the autocorrelation function, other correlation processing methods such as modified correlation, SIFT algorithm, and average amplitude difference function (AMDF) may be used. Further, instead of the correlation processing method, other methods such as a waveform processing method and a spectrum processing method may be used.

  Here, the autocorrelation function is a function representing how much similarity the audio data itself has. The autocorrelation function is defined by the following (Equation 1). In the following (Equation 1), φ (m) represents a correlation value. x (n) represents a time series of audio data. N represents the number of samples of audio data that are cut out and used for analysis. m is 0, 1, 2,..., N−1.

  That is, the pitch pattern extraction unit 26b calculates the correlation value φ (m) by applying the time series x (n) of the audio data to the above (Equation 1). The pitch pattern extraction unit 26b extracts a maximum value (peak value) from the calculated correlation value φ (m), and calculates a reciprocal of the period of the maximum value, thereby extracting a sound pitch pattern from the sound data. At this time, the reliability determination unit 26c acquires the extraction reliability when the pitch pattern extraction unit 26b extracts the audio pitch pattern from the audio data. In the present embodiment, the reliability determination unit 26c uses the correlation value φ (m) calculated by the pitch pattern extraction unit 26b as the reliability. In addition, the reliability determination unit 26c determines that a pattern having a reliability greater than or equal to a threshold among the voice pitch patterns is a pattern with high extraction reliability by the pitch pattern extraction unit 26b. On the other hand, the reliability determination unit 26c determines a pattern whose reliability is less than the threshold among the voice pitch patterns as a pattern with low extraction reliability by the pitch pattern extraction unit 26b.

  Hereinafter, the voice pitch pattern extraction processing by the pitch pattern extraction unit 26b and the reliability determination processing by the reliability determination unit 26c will be specifically described with reference to FIGS. FIG. 6 is a conceptual diagram showing a time series x (n) of audio data of an arbitrary vowel. When the time series x (n) of the audio data shown in FIG. 6 is applied to the above (Equation 1), the correlation value φ (m) is obtained. FIG. 7 is a conceptual diagram showing the correlation value φ (m) when the time series x (n) of the audio data shown in FIG. 6 is applied to the above (Equation 1). As shown in FIG. 7, the correlation value φ (m) has a maximum value at time points A, B, and C, but the pitch pattern extraction unit 26 b selects the maximum value M at the time point C having the largest value. . The pitch pattern extraction unit 26b extracts the voice pitch pattern from the voice data by calculating the reciprocal of the period T of the maximum value M at the time point C.

  Here, the reliability determination unit 26c determines whether or not the maximum value M at the time point C is equal to or greater than the threshold value S. That is, if the maximum value M is equal to or greater than the threshold value S, the reliability determination unit 26c determines that the extraction reliability by the pitch pattern extraction unit 26b is high. On the other hand, if the maximum value M is less than the threshold value S, the reliability determination unit 26c determines that the extraction reliability of the pitch pattern extraction unit 26b is low. In the example shown in FIG. 6, since the maximum value M at the time point C is equal to or greater than the threshold value S, the reliability determination unit 26c determines that the extraction reliability by the pitch pattern extraction unit 26b is high. That is, generally, the time series of voice data such as vowels a, i, u, e, o, repelling N, semi-vowels y, w, and nasal sounds n, m has a clear periodicity (see, for example, FIG. 6). ), The maximum value of the correlation value φ (m) tends to be equal to or greater than the threshold value S, and the extraction reliability by the pitch pattern extraction unit 26c is increased. On the other hand, since the time series of voice data such as voiced plosive / friction sounds b, d, g, j, and z has ambiguous periodicity, the maximum value of the correlation value φ (m) tends to be less than the threshold value S, and the pitch The extraction reliability by the pattern extraction unit 26c is lowered. It should be noted that the time series of voice data such as unvoiced plosive / friction sounds p, t, k, s, sh, h, prompting sound q, pause pose Q, etc. has no periodicity, so no local maximum value is observed, resulting in the extraction of pitch. Not. Here, even if it is a vowel, a repellent sound, a semi-vowel, a nasal sound, etc., if there is a voiced plosive / friction sound, a pose Q, etc. immediately before or after, the maximum value that is the correlation value φ (m) tends to be less than the threshold value S, The extraction reliability by the pitch pattern extraction unit 26c is lowered. The threshold S is recorded in advance in a memory (not shown) of the reliability determination unit 26c.

  FIG. 8 is a conceptual diagram showing an example of a voice pitch pattern extracted by the pitch pattern extraction unit 26b according to the present embodiment. As shown in FIG. 8, in the audio pitch pattern, a pitch determined to have high extraction reliability by the reliability determination unit 26c is represented by a solid line pattern, and extraction reliability is low by the reliability determination unit 26c. The pitch determined as is represented by a dotted line pattern. That is, the dotted line pattern in FIG. 8 is a pattern in which the reliability determination unit 26c has determined that the extraction reliability is low, and thus the pitch pattern extraction unit 26b has a voice pitch pattern extraction error. Probability is high. That is, if a synthesized speech is generated using the speech pitch pattern shown in FIG. 8 as it is, there is a high possibility that the synthesized speech has an unnatural prosody in the phoneme portion corresponding to the dotted line pattern.

  The power extraction unit 26d extracts a power pattern from the audio data output from the audio input unit 25. The power pattern is calculated by setting a fixed window length of, for example, about 20 msec in the audio data and taking the square sum of the audio data in this window.

  The speech prosody extraction unit 26 outputs the speech prosody pattern including the phoneme time length pattern, speech pitch pattern, and power pattern extracted by the above method to the modified prosody generation unit 27. Note that the method for extracting the phoneme time length pattern, the voice pitch pattern, and the power pattern is not limited to the above method.

  The modified prosody generation unit 27 uses a pattern having a high extraction reliability by the pitch pattern extraction unit 26b in the voice pitch pattern instead of a pattern having a low extraction reliability by the pitch pattern extraction unit 26b. A corrected pitch pattern is generated based on the regular pitch pattern. For this reason, the modified prosody generation unit 27 includes a prosody supplementing unit 27a.

  The prosody complementing unit 27a extracts a pattern with high reliability of extraction by the pitch pattern extracting unit 26b from the speech pitch patterns output from the speech prosody extracting unit 26. FIG. 9 is a conceptual diagram showing an example of a pattern with high extraction reliability by the pitch pattern extraction unit 26b among the voice pitch patterns shown in FIG. That is, the pattern shown in FIG. 9 is a pattern in which only the solid line pattern is extracted from the voice pitch pattern shown in FIG.

Further, the prosody complementing unit 27a uses the regular pitch pattern output from the regular prosody generation unit 24 so as to approximate a pattern (see FIG. 9) with high reliability of extraction by the pitch pattern extraction unit 26b among the voice pitch patterns. Deform. Here, the time series in the accent phrase of the pattern shown in FIG. 9 is P (n), the time series in the accent phrase of the regular pitch pattern shown in FIG. 5 is Q (n), and the time series in the accent phrase of the modified regular pitch pattern Let the sequence be Q ′ (n). In the present embodiment, the prosody complementing unit 27a transforms the time series Q (n) into the time series Q ′ (n) by using the following (Equation 2) and (Equation 3). In (Expression 2), P _d represents the amount of inclination change of Q (n). T _s represents the time expansion / contraction rate of Q (n). T _m represents the time movement width of Q (n). F _s represents the pitch expansion / contraction rate of Q (n). F _m represents the pitch movement width of Q (n). In (Expression 3), D represents an error between P (n) and Q ′ (n). That is, the prosody complementing unit 27a according to the present embodiment calculates P _d , T _s , T _m , F _s , and F _m in (Equation 2) so that the error D in (Equation 3) is minimized, Based on the calculated P _d , T _s , T _m , F _s , and F _m , the time series Q (n) is transformed into the time series Q ′ (n). The prosody complementer 27a performs this for each accent phrase. Note that the method of transforming the time series Q (n) into the time series Q ′ (n) is not limited to this. For example, the prosody complementing unit 27a may perform processing for each phrase, or may use any known mathematical formula instead of the following (Equation 2) and (Equation 3).

  FIG. 10 is a conceptual diagram illustrating an example of a regular pitch pattern modified to approximate a pattern (see FIG. 9) determined to have high extraction reliability by the pitch pattern extraction unit 26b in the audio pitch pattern. . As shown in FIG. 10, the modified regular pitch pattern is represented by a dotted line pattern. The solid line pattern shown in FIG. 10 is the pattern shown in FIG.

  The prosody complementing unit 27a generates a corrected pitch pattern by connecting the regular pitch pattern modified as described above and a pattern having high reliability of extraction by the pitch pattern extracting unit 26c among the voice pitch patterns. That is, the prosody complementing unit 27a uses the solid line pattern shown in FIG. 10 as it is, and connects the solid line pattern and the dotted line pattern. Further, the prosody complementing unit 27a performs smoothing according to any known technique in order to smooth the connection portion between the solid line pattern and the dotted line pattern. FIG. 11 is a conceptual diagram showing an example of a pitch pattern smoothed by the prosody complementing unit 27a. The circles shown in FIG. 11 are connected portions between the solid line pattern and the dotted line pattern, and indicate a smoothed portion. By performing such processing, a corrected pitch pattern is generated. FIG. 12 is a conceptual diagram showing an example of a corrected pitch pattern generated by the prosody complementing unit 27a.

  The prosody complementing unit 27a also corrects the extraction errors for the phoneme time length pattern and the power pattern output from the speech prosody extracting unit 26, respectively. For example, in the case of a phoneme time length pattern, first, the reliability determination unit 26c calculates the reliability of this extraction when the phoneme time length extraction unit 26a extracts a phoneme time length pattern from speech data. For example, the reliability determination unit 26c uses the similarity calculated by comparing the modeled data extracted from the phoneme model with each section of the speech data as the reliability. In addition, the reliability determination unit 26c determines a pattern having a reliability greater than or equal to a threshold among the phoneme time length patterns as a pattern with high extraction reliability by the phoneme time length extraction unit 26a. On the other hand, the reliability determination unit 26c determines a pattern whose reliability is less than the threshold among the phoneme time length patterns as a pattern with low reliability of extraction by the phoneme time length extraction unit 26a. Thus, the prosody complementing unit 27a uses the phoneme time length extraction unit 26a of the phoneme time length pattern to extract the reliability of the phoneme time length pattern, instead of the pattern having a low extraction reliability of the phoneme time length extraction unit 26a. A modified phoneme time length pattern is generated based on the pattern having a high height and the phoneme time length pattern generated by the phoneme time length generation unit 24a. Further, for example, in the case of a power pattern, the prosody complementing unit 27a corrects the extraction error according to any known method to generate a corrected power pattern.

  The prosody complementing unit 27a outputs the modified prosody pattern including the modified phoneme time length pattern, the modified pitch pattern, and the modified power pattern generated by the above method to the speech synthesizer 3.

  By the way, the above-mentioned prosody generation device 2 is also realized by installing a program in an arbitrary computer such as a personal computer. That is, in the text input unit 21, the language processing unit 23, the regular prosody generation unit 24, the speech input unit 25, the speech prosody extraction unit 26, and the modified prosody generation unit 27, the CPU of the computer realizes these functions. It is embodied by operating according to a program. Therefore, a program for realizing the functions of the text input unit 21, the language processing unit 23, the regular prosody generation unit 24, the speech input unit 25, the speech prosody extraction unit 26, and the modified prosody generation unit 27, or a recording in which the program is recorded. A medium is also an embodiment of the present invention. The word dictionary 22 is embodied by a built-in storage device of a computer or a storage device accessible from this computer.

(Configuration of speech synthesizer)
The speech synthesizer 3 includes a waveform dictionary 31, a waveform generator 32, and a synthesized speech output unit 33.

  The waveform dictionary 31 stores a plurality of waveform data. For example, the waveform synthesizer 3 reads the recording medium on which the waveform data is recorded, so that the waveform data is stored in the waveform dictionary 31.

  The waveform generation unit 32 generates a waveform of synthesized speech using the waveform dictionary 31 based on the modified prosodic pattern output from the prosody generation device 2. The waveform generation unit 32 outputs the generated synthesized speech waveform to the synthesized speech output unit 33.

  The synthesized speech output unit 33 generates synthesized speech based on the synthesized speech waveform output from the waveform generating unit 32. The synthesized speech output unit 33 outputs the generated synthesized speech to the outside of the speech synthesizer 3. That is, since the synthesized speech output by the synthesized speech output unit 33 uses the modified prosodic pattern generated by the prosody generating device 2, it becomes a synthesized speech having the naturalness and expressiveness that human speech has.

  By the way, the speech synthesizer 3 described above can also be realized by installing a program in an arbitrary computer such as a personal computer. In other words, the waveform generation unit 32 and the synthesized speech output unit 33 are realized by the CPU of the computer operating according to a program that realizes these functions. Therefore, a program for realizing the functions of the waveform generation unit 32 and the synthesized speech output unit 33 or a recording medium on which the program is recorded is also an embodiment of the present invention. The waveform dictionary 31 is embodied by a built-in storage device of a computer or a storage device accessible from this computer.

  The configuration of the speech synthesis system 1 has been described above, but the configuration of the speech synthesis system 1 is not limited to the configuration illustrated in FIG. For example, a voice recognition unit may be provided instead of the text input unit 21 in the prosody generation device 2.

  FIG. 13 is a block diagram illustrating a schematic configuration of a speech synthesis system 1a according to a modification of the present embodiment. 13, components having the same functions as those in FIG. 1 are denoted by the same reference numerals. The prosody generation device 2 includes a speech recognition unit 28 instead of the text input unit 21 shown in FIG. The voice recognition unit 28 has a function of recognizing human voice. Therefore, the voice recognition unit 28 converts the voice data output from the voice input unit 25 into a feature amount. The speech recognition unit 28 uses the converted feature amount as a recognition result while referring to an acoustic model and a language model (both not shown) as a recognition result. Output. That is, the voice recognition unit 28 outputs the recognition result to the language processing unit 23. Thereby, since it is not necessary for a user to input a text into the prosody generation apparatus 2, it becomes possible to reduce a user's effort.

(Operation of speech synthesis system)
Next, the operation of the speech synthesis system 1 according to the above configuration will be described with reference to FIG.

  FIG. 14 is a flowchart showing an example of the operation of the speech synthesis system 1. That is, as shown in FIG. 14, the text input unit 21 receives an arbitrary text (step Op1). The language processing unit 23 performs language analysis on the text input in step Op1 using the word dictionary 22 (step Op2). The language analysis includes the morphological analysis and dependency analysis described above. The language processing unit 23 generates phonetic character string data indicating the reading of the text based on the result of the language analysis in step Op2 (step Op3). The regular prosody generation unit 24 converts the phonetic character string data generated in step Op3 into a phoneme symbol string, and generates a regular prosody pattern based on the converted phoneme symbol string (step Op4). The regular prosody pattern includes a phoneme time length pattern, a regular pitch pattern, and a power pattern.

  The voice input unit 25 receives the human voice read out from the text input in step Op1, and converts the received human voice into voice data (step Op5). The phoneme prosody extraction unit 26 converts the phonetic character string data generated in step Op3 into a phoneme symbol string, and based on the converted phoneme symbol string, a phoneme prosody pattern is converted from the voice data converted in step Op5. Extract (Step Op6). Note that the phonetic prosody pattern includes a phoneme time length pattern, a voice pitch pattern, and a power pattern. Here, for example, the pitch pattern extraction unit 26b of the speech prosody extraction unit 26 uses the autocorrelation function defined in the above (Equation 1) to generate a speech pitch pattern from the speech data converted in step Op5. To extract.

  The reliability determination unit 26c calculates the reliability of this extraction when the pitch pattern extraction unit 26b extracts the audio pitch pattern from the audio data (Step Op7). In the present embodiment, the reliability determination unit 26c uses the correlation value φ (m) calculated by the pitch pattern extraction unit 26b as the reliability. Further, the reliability determination unit 26c determines that a pattern having a reliability greater than or equal to a threshold among the voice pitch patterns is a pattern with high extraction reliability by the pitch pattern extraction unit 26b, and the reliability is less than the threshold among the voice pitch patterns. The pattern is determined to be a pattern with low extraction reliability by the pitch pattern extraction unit 26b (Step Op8).

  The prosody complementing unit 27a modifies the regular pitch pattern generated in step Op4 so as to approximate the pattern determined to have high reliability in step Op8 (see FIG. 9) (step Op9). For example, the prosody complementing unit 27a deforms the regular pitch pattern by using the above (Equation 2) and (Equation 3). Then, the prosody complementing unit 27a uses the pattern determined to be highly reliable in Step Op8 as it is, the pattern determined to be highly reliable in Step Op8, and the regular pitch pattern deformed in Step Op9. Are connected (step Op10). The prosody complementing unit 27a performs smoothing according to any known method to generate a corrected pitch pattern in order to smooth the connected portions connected in step Op10 (step Op11). Then, the prosody complementing unit 27a outputs the modified prosody pattern including the modified pitch pattern generated in step Op11 to the speech synthesizer 3 (step Op12).

  Next, the waveform generation unit 32 of the speech synthesizer 3 generates a synthesized speech waveform using the waveform dictionary 31 based on the modified prosodic pattern output in step Op12 (step Op13). The synthesized speech output unit 33 generates synthesized speech based on the waveform of the synthesized speech generated in step Op13 (step Op14). The synthesized speech output unit 33 outputs the synthesized speech generated in step Op14 to the outside of the speech synthesizer 3 (step Op15).

  As described above, according to the prosody generation device 2 according to the present embodiment, the corrected pitch pattern generated by the prosody complementation unit 27a is appropriately approximated to a pattern with high reliability of extraction by the pitch pattern extraction unit 26b. This is a pattern generated by deforming a regular pitch pattern and connecting the deformed regular pitch pattern and a pattern with high extraction reliability by the pitch pattern extraction unit 26b. As a result, the correction pitch pattern can be corrected by correcting the extraction error of the voice pitch pattern extracted from the human voice without compromising the naturalness and expressiveness of the human voice and without taking time and effort. Can be generated.

[Embodiment 2]
FIG. 15 is a block diagram illustrating a schematic configuration of the speech synthesis system 10 according to the present embodiment. That is, the speech synthesis system 10 according to the present embodiment includes a prosody generation device 4 instead of the prosody generation device 2 shown in FIG. In FIG. 15, components having the same functions as those in FIG. 1 are given the same reference numerals, and detailed descriptions thereof are omitted.

  The prosody generation device 4 includes a modified prosody generation unit 41 instead of the modified prosody generation unit 27 shown in FIG. The modified prosody generation unit 41 is also realized by the computer CPU operating according to a program that realizes this function.

  The modified prosody generation unit 41 uses a pattern having a high extraction reliability by the pitch pattern extraction unit 26b in the voice pitch pattern, instead of a pattern having a low extraction reliability by the pitch pattern extraction unit 26b in the voice pitch pattern, and A corrected pitch pattern is generated based on the regular pitch pattern. For this reason, the modified prosody generation unit 41 includes a prosody modification unit 41a.

  The prosody modification unit 41a extracts a pattern having high extraction reliability by the pitch pattern extraction unit 26b from the speech pitch patterns output from the speech prosody extraction unit 26 (see FIG. 9). Further, the prosody modification unit 41a generates a regular prosody so as to approximate a pattern (see FIG. 9) with high reliability of extraction by the pitch pattern extraction unit 26b among the speech pitch patterns output from the speech prosody extraction unit 26. The regular pitch pattern output from the unit 24 is deformed (see FIG. 10). The processing up to this point is the same as the processing of the prosody complementing unit 27a shown in FIG.

  FIG. 16 is a conceptual diagram showing only a modified regular pitch pattern by removing the thick line pattern shown in FIG. The prosody modification unit 41a performs smoothing according to any known method in order to smooth the boundary portion of the accent phrase in the modified regular pitch pattern. FIG. 17 is a conceptual diagram showing an example of a pitch pattern smoothed by the prosody modification unit 41a. A circle mark shown in FIG. 17 indicates a smoothed portion that is a boundary portion of an accent phrase in the modified regular pitch pattern. By performing such processing, a corrected pitch pattern is generated. FIG. 18 is a conceptual diagram illustrating an example of a corrected pitch pattern generated by the prosody correcting unit 41a. The prosody modification unit 41 a outputs the modified pitch pattern shown in FIG. 18 to the speech synthesizer 3.

  As described above, according to the prosody generation device 4 according to the present embodiment, the corrected pitch pattern generated by the prosody correction unit 41a is appropriately approximated to a pattern with high extraction reliability by the pitch pattern extraction unit 26b. This pattern is generated by using a modified regular pitch pattern without modifying a regular regular pitch pattern and using a pattern with high extraction reliability by the pitch pattern extraction unit 26b. As a result, the correction pitch pattern can be corrected by correcting the extraction error of the voice pitch pattern extracted from the human voice without compromising the naturalness and expressiveness of the human voice and without taking time and effort. Can be generated.

[Embodiment 3]
FIG. 19 is a block diagram showing a schematic configuration of the speech synthesis system (prosody editing system) 11 according to the present embodiment. That is, the speech synthesis system 11 according to the present embodiment includes a GUI (Graphical User Interface) device 5 in addition to the speech synthesis system 1 shown in FIG. The GUI device 5 and the prosody generation device 2 are connected to each other by wire or wireless. The GUI device 5 and the speech synthesizer 3 are connected to each other by wire or wireless. In FIG. 19, configurations having the same functions as those in FIG. 1 are given the same reference numerals, and detailed descriptions thereof are omitted. Further, in FIG. 19, illustrations of the constituent members 21 to 27 of the prosody generation device 2 and the constituent members 31 to 33 of the speech synthesizer 3 are omitted. Further, the GUI device 5 may be provided in the speech synthesis system 1a shown in FIG. 13 and the speech synthesis system 10 shown in FIG.

  The GUI device 5 is a device that allows the user to edit the phonetic character string data and the modified prosody pattern generated by the prosody generation device 2. Therefore, the GUI device 5 has a user interface function that presents the phonetic character string data and the modified prosodic pattern to the user and can edit the phonetic character string data and the modified prosodic pattern presented using the input device. provide. Therefore, the GUI device 5 includes a display unit 51 and an editing unit 52. The display unit 51 and the editing unit 52 described above are also realized by the computer CPU operating according to a program that realizes this function.

  The display unit 51 includes an arbitrary display device such as a liquid crystal display, an organic EL display, a plasma display, or a CRT display. The editing unit 52 includes an arbitrary input device such as a keyboard, a mouse, a numeric keypad, and a touch panel.

  FIG. 20 is a conceptual diagram illustrating an example of a display screen displayed on the display unit 51. As shown in FIG. 20, the display screen of the display unit 51 includes a text editing unit 51a, a language processing button 51b, a language processing result editing unit 51c, a rule prosody generation button 51d, a rule prosody pattern display unit 51e, a voice input button 51f, A speech prosody extraction button 51g, a speech prosody pattern display unit 51h, an automatic correction button 51i, a modified prosody pattern display unit 51j, and a waveform generation button 51k are provided.

  The text editing unit 51a allows the user to input arbitrary text. In the example illustrated in FIG. 20, text representing “please push the push button according to the voice guidance” is input to the text editing unit 51 a by the user. The text may be input to the text editing unit 51a by the user specifying a text file prepared in advance in the GUI device 5 and opening the specified text file.

  The language processing button 51b is a button for instructing the language processing unit 23 of the prosody generation device 2 to perform language analysis of the text input to the text editing unit 51a.

  The language processing result editing unit 51c displays the phonetic character string data generated based on the result of language analysis by the language processing unit 23. In the example shown in FIG. 20, the phonetic character string data “ONSEGA 'IDANNI_SHITAGATTE, push-bo' TAO_OSITECHDASAY '” is displayed in the language processing result editing unit 51c. The language processing result editing unit 51c has a function of allowing the user to edit the displayed phonetic character string data. Thereby, when the language analysis by the language processing unit 23 is wrong, that is, when the displayed phonetic character string data is wrong, for example, the user changes the position of the accent nucleus, By changing the boundary of the phrase, it is possible to change to the correct phonetic character string data.

  The rule prosody generation button 51d is a button for instructing the rule prosody generation unit 24 of the prosody generation device 2 to generate a rule prosody pattern based on the phonetic character string data displayed in the language processing result editing unit 51c. It is.

  The regular prosody pattern display unit 51 e displays the regular prosody pattern generated by the regular prosody generation unit 24. In the example shown in FIG. 20, the regular prosody pattern display unit 51e displays a regular pitch pattern and a phoneme duration pattern among the regular prosodic patterns. The regular prosodic pattern display unit 51e may display a power pattern.

  The voice input button 51f is a button for causing the GUI device 5 to input a human voice read out from the text input to the text editing unit 51a. For example, when the user designates the voice input button 51 f and reads out the text, the voice of the human who reads the text is recorded in the GUI device 5. For this reason, a microphone is built in or connected to the GUI device 5. Note that when the user instructs the voice input button 51f, a voice data file is displayed, and a human voice may be input to the GUI device 5 by pointing to the displayed voice data file.

  The speech prosody extraction button 51g is a button for instructing the speech prosody extraction unit 26 of the prosody generation device 2 to extract a speech prosody pattern from the human speech input by the speech input button 51f.

  The speech prosody pattern display unit 51 h displays the speech prosody pattern extracted by the speech prosody extraction unit 26. In the example shown in FIG. 20, the speech prosody pattern display unit 51 h displays a speech pitch pattern and a phoneme time length pattern among speech prosodic patterns. In the voice pitch pattern, the pitch determined to be high in extraction reliability by the reliability determination unit 26c is represented by a solid line pattern, and the pitch determined to be low in extraction reliability by the reliability determination unit 26c. It is represented by a dotted line pattern. Note that a power pattern may be displayed in the speech prosody pattern display portion 51h.

  The automatic correction button 51i is displayed on the speech pitch pattern with high extraction reliability displayed on the speech prosody pattern display unit 51h and on the regular prosody pattern display unit 51e with respect to the prosody complement unit 27a of the prosody generation device 2. It is a button for instructing to generate a corrected pitch pattern based on the regular pitch pattern. The automatic correction button 51i is a button for instructing generation of a corrected phoneme time length pattern in addition to generating a corrected pitch pattern.

  The modified prosodic pattern display unit 51j displays the modified prosodic pattern generated by the prosody complementing unit 27a. In the example shown in FIG. 20, the modified prosody pattern display unit 51j displays a modified pitch pattern and a modified phoneme time length pattern among the modified prosodic patterns. The modified prosody pattern display unit 51e may display a modified power pattern. Here, in the present embodiment, the modified prosody pattern display unit 51j moves the displayed modified pitch pattern by the user operating the input device, and newly resets the modified pitch pattern. it can. As an example, when the user touches the correction pitch pattern to which the mouse pointer is to be moved and moves (drags) the touched position (instructed position) upward or downward and drops it at the desired position, the correction is performed. The pitch pattern is arranged at a desired position that has been moved. Note that the modified prosodic pattern display unit 51j preferably displays the modified pitch pattern superimposed on the spectrogram.

  The waveform generation button 51k is a button for instructing the waveform generation unit 32 of the speech synthesizer 3 to generate a synthesized speech waveform based on the modified prosodic pattern displayed on the modified prosodic pattern display unit 51e. Thereby, the speech synthesizer 3 can output the synthesized speech based on the waveform of the synthesized speech generated by the waveform generation button 51k. Therefore, the user can change the modified pitch pattern displayed on the modified prosodic pattern display unit 51j based on the synthesized speech output from the speech synthesizer 3.

  As described above, according to the speech synthesis system 11 according to the present embodiment, the GUI device 5 edits at least one of the phonogram character string data and the modified prosody pattern generated by the prosody generation device 2. The user can make fine adjustments to at least one of the phonetic character string data and the modified prosodic pattern generated by the generation device 2.

  In the first to third embodiments, the modified prosody pattern output from the prosody generation device or the GUI device is output to the speech synthesizer, and the speech synthesizer generates and outputs synthesized speech based on the modified prosodic pattern. Although the example to do was demonstrated, it is not limited to this. For example, a prosody dictionary for speech synthesis, a waveform dictionary for speech synthesis, an acoustic model for speech recognition, and the like may be generated using the modified prosody pattern output from the prosody generation device or the GUI device.

  That is, the present invention is not limited to the first to third embodiments described above, and various modifications can be made within the scope of the claims. That is, embodiments obtained by combining technical means appropriately changed within the scope of the claims are also included in the technical scope of the present invention.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
A text input part for inputting arbitrary text;
A language processing unit that generates phonetic character string data indicating the reading of the text by performing language analysis on the text;
A regular prosody generation unit that generates a regular prosody pattern indicating the prosody of the text based on the phonetic character string data and the prosody generation rules;
A voice input unit for converting the voice of a human who has read the text into voice data;
A speech prosody extraction unit that extracts a speech prosody pattern indicating the prosody of the human speech from the speech data;
When the speech prosody extraction unit extracts the speech prosody pattern from the speech data, the speech prosody extraction unit acquires the reliability of the extraction, and among the speech prosody patterns, the reliability of the speech prosody pattern is greater than or equal to a threshold by the speech prosody extraction unit A reliability determination unit that determines a pattern with high extraction reliability and determines a pattern having a reliability less than a threshold among the speech prosody patterns as a pattern with low extraction reliability by the speech prosody extraction unit;
Instead of a pattern with low extraction reliability by the speech prosody extraction unit in the speech prosody pattern, a pattern with high extraction reliability by the speech prosody extraction unit in the speech prosody pattern, and a regular prosody pattern A prosody generation device comprising: a modified prosody generation unit that generates a modified prosody pattern based on the prosody pattern.

(Appendix 2)
The modified prosody generation unit includes:
The regular prosody pattern is modified so as to approximate a pattern with high extraction reliability by the speech prosody extraction unit of the speech prosody pattern, and the speech prosody extraction unit of the speech prosody pattern is modified. The prosody generation device according to claim 1, further comprising a prosody complementing unit that generates a modified prosody pattern by connecting a pattern having high extraction reliability.

(Appendix 3)
The modified prosody generation unit includes:
The regular prosody pattern is modified so as to approximate a pattern with high extraction reliability by the speech prosody extraction unit of the speech prosody pattern, and the extraction reliability by the speech prosody extraction unit of the speech prosody pattern is high The prosody generation device according to claim 1, further comprising a prosody modification unit that generates a modified prosody pattern by using a modified regular prosody pattern without using a pattern.

(Appendix 4)
The prosodic generation device according to any one of appendices 1 to 3, wherein the regular prosodic pattern, the speech prosodic pattern, and the modified prosodic pattern are pitch patterns representing a voice pitch change pattern.

(Appendix 5)
The prosody generation device according to any one of appendices 1 to 4,
A prosody editing system comprising: a GUI device that edits at least one of the phonetic character string data and the modified prosody pattern generated by the prosody generation device.

(Appendix 6)
The prosody generation device according to any one of appendices 1 to 4,
A speech synthesis system comprising: a speech synthesizer that generates and outputs synthesized speech based on the modified prosodic pattern generated by the prosody generation device.

(Appendix 7)
The prosody generation device according to any one of appendices 1 to 4,
A GUI device for editing at least one of the phonetic character string data and the modified prosody pattern generated by the prosody generation device;
A speech synthesizer that generates and outputs synthesized speech based on at least one of the modified prosodic pattern generated by the prosody generating device and the modified prosodic pattern edited by the GUI device; A speech synthesis system.

(Appendix 8)
A text input unit provided in the computer, a text input process in which arbitrary text is input;
A language processing step of the computer includes a language processing step of generating phonetic character string data indicating reading of the text by analyzing the text.
A regular prosody generation step of generating a regular prosody pattern indicating a prosody of the text based on the phonetic character string data and the prosody generation rule,
A voice input step in which the voice input unit included in the computer converts human voice read out from the text into voice data;
A speech prosody extraction step in which the speech prosody extraction unit provided in the computer extracts a speech prosody pattern indicating the prosody of the human speech from the speech data;
A reliability determination unit included in the computer acquires the reliability of the extraction when the speech prosody pattern is extracted from the speech data in the speech prosody extraction step, and the reliability of the speech prosody pattern Is determined to be a pattern having a high extraction reliability by the speech prosody extraction step, and a pattern having a reliability less than the threshold among the speech prosody extraction steps has a low extraction reliability by the speech prosody extraction step A reliability determination step for determining a pattern;
The modified prosody generation unit included in the computer has an extraction reliability of the speech prosody pattern extracted from the speech prosody pattern by the speech prosody extraction step instead of a pattern having a low reliability of extraction by the speech prosody extraction step. A prosody generation method comprising: a high pattern; and a modified prosody generation step of generating a modified prosody pattern based on the regular prosody pattern.

(Appendix 9)
Text input processing where arbitrary text is input,
Linguistic processing for generating phonetic character string data indicating the reading of the text by language analysis of the text;
A regular prosody generation process for generating a regular prosody pattern indicating the prosody of the text based on the phonetic character string data and the prosody generation rules;
A voice input process for converting a human voice read out from the text into voice data;
A speech prosody extraction process for extracting a speech prosody pattern indicating the prosody of the human speech from the speech data;
When the speech prosody pattern is extracted from the speech data in the speech prosody extraction process, the reliability of the extraction is obtained, and the speech prosody extraction is performed for a pattern having the reliability greater than or equal to a threshold among the speech prosody patterns. A reliability determination process for determining a pattern with high reliability of extraction by processing, and determining a pattern having the reliability less than a threshold among the speech prosodic patterns as a pattern with low extraction reliability by the speech prosody extraction process;
Instead of a pattern with low extraction reliability by the voice prosody extraction process in the phonetic prosody pattern, a pattern with high extraction reliability by the voice prosody extraction process and a regular prosody pattern in the voice prosody pattern A prosody generation program that causes a computer to execute a modified prosody generation process that generates a modified prosody pattern based on the computer program.

  As described above, the present invention accepts an arbitrary text and a human voice that is read out from the text, and generates a prosodic pattern based on the received arbitrary text and the human voice. It is useful as a method or prosody generation program.

1 is a block diagram showing a schematic configuration of a speech synthesis system according to a first embodiment of the present invention. It is a conceptual diagram which shows the result of the morphological analysis which the language processing part in the prosody generation apparatus of the said speech synthesis system performed with respect to character string data. It is a conceptual diagram which shows the some clause produced | generated by the said language processing part, and its reading. It is a conceptual diagram which shows the state by which the accent phrase component was superimposed on the phrase component. It is a conceptual diagram which shows an example of the regular pitch pattern produced | generated by the pitch pattern production | generation part in the said prosody production | generation apparatus. It is a conceptual diagram which shows the time series of the audio | voice data of arbitrary vowels. It is a conceptual diagram which shows the correlation value at the time of applying the time series of the audio | voice data shown in FIG. 6 to an autocorrelation function. It is a conceptual diagram which shows an example of the audio | voice pitch pattern extracted by the pitch pattern extraction part in the said prosody generation apparatus. FIG. 9 is a conceptual diagram illustrating an example of a pattern determined as having high extraction reliability by the reliability determination unit in the prosody generation device in the speech pitch pattern illustrated in FIG. 8. It is a conceptual diagram which shows an example of the regular pitch pattern deform | transformed so that it might approximate to the pattern shown in FIG. It is a conceptual diagram which shows an example of the pitch pattern smoothed by the prosody complementation part in the said prosody generation apparatus. It is a conceptual diagram which shows an example of the correction pitch pattern produced | generated by the said prosody complement part. It is a block diagram which shows schematic structure of the speech synthesis system which concerns on the modification of the 1st Embodiment of this invention. It is a flowchart which shows an example of operation | movement of the said speech synthesis system. It is a block diagram which shows schematic structure of the speech synthesis system which concerns on the 2nd Embodiment of this invention. FIG. 11 is a conceptual diagram showing only a regular pitch pattern that is modified by removing the thick line pattern shown in FIG. 10. It is a conceptual diagram which shows an example of the pitch pattern smoothed by the prosody modification part in the prosody generation apparatus of the said speech synthesis system. It is a conceptual diagram which shows an example of the correction pitch pattern produced | generated by the said prosody correction | amendment part. It is a block diagram which shows schematic structure of the speech synthesis system which concerns on the 3rd Embodiment of this invention. It is a conceptual diagram which shows an example of the display screen displayed on the display part in the GUI apparatus of the said speech synthesis system.

Explanation of symbols

1, 1a, 10, 11 Speech synthesis system 2, 4 Prosody generation device 3 Speech synthesizer 5 GUI device 21 Text input unit 23 Language processing unit 24 Regular prosody generation unit 24a Phoneme time length generation unit 24b Pitch pattern generation unit 24c Power generation Unit 25 speech input unit 26 speech prosody extraction unit 26a phoneme time length extraction unit 26b pitch pattern extraction unit 26c reliability determination unit 26d power extraction unit 27, 41 modified prosody generation unit 27a prosody complement unit 41a prosody modification unit

Claims (7)

A text input part for inputting arbitrary text;
A language processing unit that generates phonetic character string data indicating the reading of the text by performing language analysis on the text;
A regular prosody generation unit that generates a regular prosody pattern indicating the prosody of the text based on the phonetic character string data and the prosody generation rules;
A voice input unit for converting the voice of a human who has read the text into voice data;
A speech prosody extraction unit that extracts a speech prosody pattern indicating the prosody of the human speech from the speech data;
When the speech prosody extraction unit extracts the speech prosody pattern from the speech data, the speech prosody extraction unit acquires the reliability of the extraction, and among the speech prosody patterns, the reliability of the speech prosody pattern is greater than or equal to a threshold by the speech prosody extraction unit A reliability determination unit that determines a pattern with high extraction reliability and determines a pattern having a reliability less than a threshold among the speech prosody patterns as a pattern with low extraction reliability by the speech prosody extraction unit;
Instead of a pattern with low extraction reliability by the speech prosody extraction unit in the speech prosody pattern, a pattern with high extraction reliability by the speech prosody extraction unit in the speech prosody pattern, and a regular prosody pattern A prosody generation device comprising: a modified prosody generation unit that generates a modified prosody pattern based on the prosody pattern. The modified prosody generation unit includes:
The regular prosody pattern is modified so as to approximate a pattern with high extraction reliability by the speech prosody extraction unit of the speech prosody pattern, and the speech prosody extraction unit of the speech prosody pattern is modified. The prosody generation device according to claim 1, further comprising a prosody complementing unit that generates a modified prosody pattern by connecting a pattern having high extraction reliability. The modified prosody generation unit includes:
The regular prosody pattern is modified so as to approximate a pattern with high extraction reliability by the speech prosody extraction unit of the speech prosody pattern, and the extraction reliability by the speech prosody extraction unit of the speech prosody pattern is high The prosody generation device according to claim 1, further comprising a prosody modification unit that generates a modified prosody pattern by using a modified regular prosody pattern without using a pattern.   The prosody generation device according to any one of claims 1 to 3, wherein the regular prosody pattern, the speech prosody pattern, and the modified prosody pattern are pitch patterns representing a change pattern of voice pitch. The prosody generation device according to any one of claims 1 to 4,
A prosody editing system comprising: a GUI device that edits at least one of the phonetic character string data and the modified prosody pattern generated by the prosody generation device. A text input unit provided in the computer, a text input process in which arbitrary text is input;
A language processing step of the computer includes a language processing step of generating phonetic character string data indicating reading of the text by analyzing the text.
A regular prosody generation step of generating a regular prosody pattern indicating a prosody of the text based on the phonetic character string data and the prosody generation rule,
A voice input step in which the voice input unit included in the computer converts human voice read out from the text into voice data;
A speech prosody extraction step in which the speech prosody extraction unit provided in the computer extracts a speech prosody pattern indicating the prosody of the human speech from the speech data;
A reliability determination unit included in the computer acquires the reliability of the extraction when the speech prosody pattern is extracted from the speech data in the speech prosody extraction step, and the reliability of the speech prosody pattern Is determined to be a pattern having a high extraction reliability by the speech prosody extraction step, and a pattern having a reliability less than the threshold among the speech prosody extraction steps has a low extraction reliability by the speech prosody extraction step A reliability determination step for determining a pattern;
The modified prosody generation unit included in the computer has an extraction reliability of the speech prosody pattern extracted from the speech prosody pattern by the speech prosody extraction step instead of a pattern having a low reliability of extraction by the speech prosody extraction step. A prosody generation method comprising: a high pattern; and a modified prosody generation step of generating a modified prosody pattern based on the regular prosody pattern. Text input processing where arbitrary text is input,
Linguistic processing for generating phonetic character string data indicating the reading of the text by language analysis of the text;
A regular prosody generation process for generating a regular prosody pattern indicating the prosody of the text based on the phonetic character string data and the prosody generation rules;
A voice input process for converting a human voice read out from the text into voice data;
A speech prosody extraction process for extracting a speech prosody pattern indicating the prosody of the human speech from the speech data;
When the speech prosody pattern is extracted from the speech data in the speech prosody extraction process, the reliability of the extraction is obtained, and the speech prosody extraction is performed for a pattern having the reliability greater than or equal to a threshold among the speech prosody patterns. A reliability determination process for determining a pattern with high reliability of extraction by processing, and determining a pattern having the reliability less than a threshold among the speech prosodic patterns as a pattern with low extraction reliability by the speech prosody extraction process;
Instead of a pattern with low extraction reliability by the voice prosody extraction process in the phonetic prosody pattern, a pattern with high extraction reliability by the voice prosody extraction process and a regular prosody pattern in the voice prosody pattern A prosody generation program that causes a computer to execute a modified prosody generation process that generates a modified prosody pattern based on the computer program.

Images