JP5687611B2 - Phrase tone prediction device - Google Patents

Description

  The present invention relates to a phrase end tone predicting apparatus that predicts a phrase end tone that frequently appears in conversation from text information.

  In conventional speech-to-speech synthesis for the purpose of providing information, there are many plain sentences in the text to be synthesized, and if the accent phrase has an accent nucleus, the change in the fundamental frequency is different from the position of the nucleus. When there is no symbol, there is a tendency to generally descend from the position of the head of the accent phrase toward the end of the phrase. However, in conversation speech between humans, there is a case where the fundamental frequency is lowered again toward the end of the accent phrase and then raised again to convey the intention of continuing the conversation.

  The movement in which the fundamental frequency rises again at some position up to the end of the phrase after the fall in the accent phrase is called “end of tone”. For example, it is a phenomenon seen in an utterance that raises the fundamental frequency again at the end of a phrase such as “~ ↑, ~ ↑” when it is desired to continue speaking for a long time. A label indicating the existence of such a re-rise is referred to as “end of phrase label”. For example, in order for a speech synthesizer to synthesize a more natural conversation speech, it is recognized whether or not an accent phrase is generated at the end of the phrase (the fundamental frequency is increased again), that is, the end-of-phrase tone. Labeling is essential.

  Conventionally, for example, for English news speech, the basic frequency is re-established at the end of a phrase by using the length and position of various units such as phrases, sentences, and paragraphs, and parts of speech corresponding to several words before and after the phrase boundary as features. The classification of whether or not to rise was performed (Non-Patent Document 1). As is apparent from Non-Patent Document 1, the construction of such a prediction model is generally a method of automatic configuration based on a large amount of data.

K. Ross and M. Ostendorf, “Prediction of abstract prosodic labels for speech synthesis”, Computer Speech and Language, vol.10, Issue 3, pp. 155-185 (1996).

  However, conversations are not always spoken with almost correct grammar like news information sentences, and even if function words such as particles are omitted, the intention is transmitted from the content words. For this reason, not only based on the part of speech as in the prior art, but also a phrase end tone processing based on the content of the utterance is required. In addition, although the end-of-phrase tone is generated near the phrase boundary, the distance from the boundary is not always constant, so it is necessary to incorporate as many words as possible before and after the phrase boundary as feature values for classification.

  However, in the prior art, as described above, for example, the phrase end tone is predicted using the feature quantity obtained from the part-of-speech information limited to a few words before and after the phrase boundary. is there.

  The present invention has been made in view of such a problem, and an object thereof is to provide an end-of-phrase tone prediction apparatus that accurately predicts an end-of-phrase tone based on a lot of information other than the part of speech.

  The phrase end tone prediction apparatus of the present invention includes a feature amount information extraction unit, a word information database, a feature amount conversion unit, a phrase end tone prediction model, and a prediction unit. The feature quantity information extraction unit receives morpheme information and accent phrase information as input, and from that information the appearance form, part of speech, reading, accent phrase accent type, accent phrase end pose required by the phrase end tone prediction model Feature information about presence / absence is extracted. The word information database stores bit strings corresponding to the appearance forms and parts of speech of all words to be handled. The feature amount conversion unit receives the feature amount information output from the feature amount information extraction unit and generates a feature amount vector corresponding to the feature amount information with reference to a bit string stored in the word information database. The phrase end tone prediction model is a prediction model that binarizes the presence / absence of the phrase end tone. The prediction unit receives the feature amount vector, and binary-classifies the feature amount vector using the phrase end tone prediction model, and assigns a phrase end tone label to the accent phrase when a phrase end tone exists.

According to the phrase endnote prediction device of the present invention, it is conventionally given manually using a prediction model that binaryly classifies the presence or absence of the phrase endnote from the feature quantity including the appearance form that is larger in number than the part of speech. The end-of-phrase tone label can be automatically assigned with high accuracy. Accordingly, when a large amount of speech is synthesized from, for example, a conversation sentence expressed in text, a large amount of input data for speech synthesis to which an accurate ending tone label is assigned can be generated.

The figure which shows the function structural example of the phrase end tone prediction apparatus 100 of this invention. The figure which shows the operation | movement flow of the phrase end tone prediction apparatus. The figure which shows an example of feature-value information. The figure which shows an example of an accent nucleus. The figure which shows the operation | movement flow of the feature-value conversion part 30. FIG. The figure which shows the structural example of the feature-value vector of 1 accent phrase. The figure which shows the example of the appearance form bit string 60. FIG. The figure which shows the example of a support vector. The figure which shows the other example of a feature-value vector. FIG. 10 is a diagram illustrating an example of a feature amount vector according to the fourth embodiment.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows a functional configuration example of the phrase end tone prediction apparatus 100 of the present invention. The operation flow is shown in FIG. The phrase end tone prediction apparatus 100 includes a feature quantity information extraction unit 10, a word information database 20, a feature quantity conversion unit 30, a phrase end tone prediction model storage unit 40, a prediction unit 50, and a control unit 60. It has. The phrase endnote prediction device 100 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

  The feature quantity information extraction unit 10 receives the morpheme information and the accent phrase information as input, and the appearance form, the part of speech, the reading, the accent phrase accent type, and the phrase end pose required by the phrase end tone prediction model from the information. Feature information about presence / absence is extracted (step S10). The word information database 20 stores bit strings corresponding to appearance forms and parts of speech of all words to be handled. Those bit strings are stored in advance.

  The feature amount conversion unit 30 receives the feature amount information output from the feature amount information extraction unit 10 and generates a feature amount vector corresponding to the feature amount information with reference to the bit string stored in the word information database 20. (Step S30).

  The predicting unit 50 receives the feature quantity vector, performs binary classification on the feature quantity vector using the phrase end tone prediction model stored in the phrase end tone prediction model storage unit 40, and when there is a phrase end tone. Then, a phrase end tone label is given to the accent phrase (step S50).

  The feature quantity information extraction process in step 10, the feature quantity conversion process in step 20, and the prediction process in step 50 are repeated until the processing for all accent phrases in the input text data is completed (no in step S60). ). The control unit 60 performs the repetitive processing and the control of the time-series operations between the functional units described above.

  As described above, according to the phrase end tone predicting apparatus 100 of the present invention, it is possible to automatically give a phrase end tone label. Hereinafter, the operation of each unit will be described in more detail with specific examples.

[Feature information extraction unit]
The feature quantity extraction unit 10 receives morpheme information and accent phrase information as inputs. The morpheme information is information such as appearance form, part of speech, and reading. The accent phrase information is information such as the position of the accent phrase boundary, the accent type of the accent phrase, and whether or not a pause is added at the end of the accent phrase. The feature quantity information extraction unit 10 receives these pieces of information as input, and extracts feature quantity information necessary for classification of accent phrases for which the end-of-phrase tone prediction is performed.

  As feature quantity information to be extracted, a series of word appearances and part-of-speech series included in an accent phrase for which the phrase end tone prediction is performed is extracted from position information of the accent phrase boundary. Subsequently, the reading series included in the accent phrase is connected, and the length of the entire accent phrase is extracted. From the accent type information of the accent phrase, the position information of the accent (accent nucleus position) in the accent phrase and information on whether or not a pause is added at the end of the accent phrase are extracted.

  FIG. 3 shows an example of feature amount information extracted from one accent phrase. From the left side of FIG. 3, a series of appearance forms, parts of speech, and readings, and feature amount information such as the position of the accent nucleus of the accent phrase and the presence or absence of a phrase end pose are arranged. FIG. 4 shows the position of the accent nucleus. The horizontal direction in FIG. 4 is the elapsed time and represents the level of the L and H frequencies. In this example, the accent kernel is the third reading from the beginning. The accent core may be a number obtained by counting readings from the end of the accent phrase.

The morpheme information and the accent phrase information are information obtained as a result of language analysis processing performed by a conventional speech synthesizer.
[Feature conversion unit]
FIG. 5 shows an operation flow of the feature amount conversion unit 30. The feature amount conversion unit 30 uses feature amount information of all words (appearance forms) constituting the accent phrase and its part of speech. Instead of creating a feature vector by limiting the number of words used in the prior art, the number of words and the corresponding part of speech are set as a feature vector.

  As a result, even if the number of words constituting the accent phrase changes, the feature quantity vector is always set with the size of the sum of the appearance form of the word and the number of parts of speech corresponding thereto. In the prior art, the number of words exceeding the upper limit of the number set to a certain value was not included in the feature amount, but this expression method can include words and parts of speech that could not be included conventionally. It becomes.

  FIG. 6 shows a configuration example of a feature vector of one accent phrase. The feature vector includes an appearance bit string 60, a part-of-speech bit string 61, an accent phrase length 62, an accent nucleus position 63, and a phrase end pose presence / absence 64. This order is not limited to this example. The order of the appearance bit string 60 and the part-of-speech bit string 61 may be reversed back and forth. The numerical sequence in FIG. 6 is an example of the feature quantity vector.

  The feature amount conversion unit 30 receives the feature amount information in units of one accent phrase extracted by the feature amount information extraction unit 10, and refers to the bit string stored in the word information database to correspond to the feature amount information. Generate a quantity vector. When the input feature quantity information is appearance form information constituting an accent phrase (yes in step S31), the feature quantity conversion unit 30 refers to the appearance form bit string stored in the word information database 20 and inputs the feature quantity information. The bit of the bit string corresponding to the appearing appearance information is set to “1” (step S32).

  FIG. 7 illustrates a part of the appearance bit string 60. Bits respectively corresponding to the appearance forms of “Out”, “Buy”, “Sell”, “Loan”, “...” Are arranged for the number of all the appearance forms handled by the phrase endnote prediction device 100. . For example, if the number of words is 60,000, an array of 60,000 bits becomes the appearance type bit string 60.

  In the case where the accent phrase is “Is it coming out”, in this example, since the appearance type “out” is located at the head of the appearance type bit string 60, the first bit of the appearance type bit string 60 is set to “1”. Is done. Processing for setting the bits of the bit string corresponding to the input appearance information to “1” until the bits of the appearance bit string 60 at the positions corresponding to all the appearances of the accent phrase are set to “1”. Is repeated. When the accent phrase is “I'm coming out”, since the number of appearance forms is 5, the bit at any position corresponding to the appearance form in 60,000 bit strings is set to “1”.

  For the part of speech, the bit at the position corresponding to the part of speech of the part of speech bit string 61 is set to “1” by the same processing as the appearance form (step S34). The part-of-speech bit string 61 is arranged by the number of parts-of-speech.

  Next, the feature quantity conversion unit 30 counts the number of readings of the feature quantity information and sets it as the accent phrase length (step S36). In the case of this example, eight accents are counted and one accent phrase is completed (yes in step S37). Therefore, after the arrangement of the appearance bit string 60 and the part-of-speech bit string 61, the accent phrase length, in this case “8” ”, The accent nucleus position of the feature value information, in this example“ 3 ”, and bit information of presence / absence of the phrase end pose is added, and the feature value conversion processing of one accent phrase is finished (step S38). Then, the number of readings counted in step S36 is reset in preparation for the processing of the next accent phrase. The processes in steps S31 to S38 described above are repeated until the processes for all accent phrases are completed. The sequence shown in FIG. 6 is an example of the feature vector of the accent phrase “Is it coming out?”. The number of “0” bits in the appearance bit string and the number of “0” and “1” bits in the part-of-speech bit string 61 are omitted. Although the example in which the feature amount conversion unit 30 counts readings has been described, reading may be replaced with any of phonemes, syllables, and mora.

[End of Tone Prediction Model Storage Unit]
The phrase end tone prediction model storage unit 40 includes a phrase end tone prediction model that is a prediction model for binary classification of presence / absence of a phrase end tone, a support vector that is a feature quantity vector that represents a feature of a classification boundary, and a support vector thereof The classification category is stored. Since the phrase end tone prediction model performs accurate prediction in a situation where there is little data, the problem is solved by modeling only the classification boundary of the classification category without modeling the distribution shape of each category. For this, for example, a classifier such as a support vector machine can be used.
In general, in the classification using the support vector machine, classification is performed using the following formula.

Here, s _k is a support vector that is a feature vector representing the features of the classification boundary. y _k is a classification category for support vector _{s k} forming a _{s k} and a pair. b is the time of learning, is a value corresponding to the sections of the classification boundary surface which is obtained at the same time when choosing a support vector s _k from data for learning.

FIG. 8 shows an example of the support vector s _k and the classification category y _k stored in the phrase end tone prediction model storage unit 40. The classification category y _k is set to +1 or -1. +1 means that there is a phrase end tone, that is, the fundamental frequency rises again at the end of the accent phrase. -1 means that there is no ending tone, that is, the fundamental frequency does not rise again at the ending.

Function K (·) as input and the support vector s _k feature vector x, is a function for calculating a non-negative value corresponding to the similarity between the two vectors. For this function K (•), a polynomial function or a Gaussian function is used. In this case the output value of the function K (·) is high, classification category y _k given in support vector s _k is appreciated. As in Equation (1), the product of y _k and K (•) is calculated and added for all support vectors s _k to determine whether or not there is a classification destination, that is, a phrase end tone.

[Predictor]
The prediction unit 50 reads the phrase end tone prediction model, the support vector _sk, and the classification category y _k stored in the phrase end tone prediction model storage unit 40, and inputs the feature amount vector converted by the feature amount conversion unit 30. Then, it is determined whether or not there is a re-increase in the fundamental frequency at the end of the accent phrase using the phrase end tone prediction model, and a phrase end tone label is assigned if there is a phrase end tone.

  Note that the feature vector may be composed of a partial combination of the elements of the feature vector described above. For example, the configuration of the feature quantity vector such as only the combination of the appearance bit string 60 and the part of speech bit string 61 or the combination of the appearance bit string 60, the accent phrase length 62, the accent nucleus position 63, and the phrase end pose presence / absence 64 is shown in FIG. It is not limited to the example shown in.

  The appearance bit string 60 and the part-of-speech bit string 61 may be elements of one feature vector. For example, the appearance form “out” and the part of speech “verb stem”, as well as “te” and “utilization ending”, “kuru” and “auxiliary verb”, “n” and “auxiliary noun”, “sound” and “determination” Only the element corresponding to each set of “Lyrics” is set to “1”.

  FIG. 9 shows a feature vector composed of the appearance form / part of speech bit string 80 as a set of the appearance form and the part of speech. The accent phrase length 62, the accent nucleus 63, and the phrase end pose presence / absence 64 are the same as the feature vector of the first embodiment shown in FIG. Each bit of the appearance form / part of speech bit string 80 includes, for example, “appearance form_output: part of speech_verb stem”, “appearance form_buy: part of speech_verb stem”,. "...".

  By handling the feature quantity vector in this way, it is possible to clarify the co-occurrence relationship between the appearance form and the part of speech. In addition, the number of dimensions of the feature vector can be reduced.

  Note that the feature quantity vector may be configured by a partial combination of elements of the feature quantity vector as in the first embodiment. For example, the configuration of the feature quantity vector such as only the combination of the appearance form / part of speech bit string 80 and the accent phrase length 62 or the combination of the appearance form / part of speech bit string 80, the accent phrase length 62, and the accent nucleus position 63 is shown in FIG. It is not limited to the example shown.

  The above-described embodiment is an example of generating a feature vector for all appearance forms of an accent phrase, but it is not necessary to target all appearance forms. It may be limited to N words from the end of the accent phrase toward the beginning. For example, the feature quantity vector may be limited to the number of N = 2 appearance forms.

  If the accent phrase is “I'm coming out” as described above, and if it is limited to two words from the end of the phrase, for example, when the feature vector is the appearance form / part of speech bit string 80 shown in Example 2, Only two bits constituting the appearance form / part-of-speech bit string 80 corresponding to the set of “n” and “auxiliary noun” and the set of “sound” and “determination” are “1”. All the elements corresponding to the combinations of the appearance form and the part of speech of other words are set to 0. The length of the other accent phrases, the position of the accent nucleus, and the presence / absence of the phrase end pose are converted and set in the same manner as in the first or second embodiment. In the case of this embodiment, since the processing amount of the feature amount conversion unit 30 is reduced, the load on the computer can be reduced. The same idea can be applied to the feature vector (FIG. 6) shown in the first embodiment.

  N words from the end of the accent phrase may be expressed for each position. In other words, as information of the i-th word from the end of the phrase, N partial vectors having elements of all combinations of appearance forms and parts of speech are connected to form a feature vector. In order to express that the number is less than N, an element indicating that there is no word is added at each i position.

An example is shown in FIG. FIG. 10 shows an example where N = 3 and there is no third word from the end of the phrase. Appearance form, part of speech bit stream 80 _1, a first partial feature quantity vectors counted from phrase end. Appearance form, part of speech bit stream ₈₀₂ is a second part feature vectors counted from phrase end. In this example, it indicates a case where the word up to two eyes exist counted from phrase end, clause 3, counting from the end of the partial characteristic amount vector (appearance form, part of speech bit stream) 80 to the end of _3, A bit “1” indicating that there is no N = third word from the end of the phrase is given.

  In this way, it is possible to create a fixed-length feature vector that limits the number of words (N) that make up the accent phrase to be determined for the presence or absence of the end-of-phrase tone. By setting the feature vector to a fixed length, a program for causing a computer to function as the end-of-pitch tone apparatus 100 of the present invention can be simplified. Moreover, there is an effect of simplifying the hardware configuration in the case where the phrase end tone device 100 of the present invention is configured by hardware.

  The above-described embodiment has been described with respect to an accent phrase unit, that is, an example of predicting the presence or absence of a phrase end tone by focusing on only one accent phrase, but information on accent phrases before and after the accent phrase to be predicted May be used to predict the presence or absence of the end-of-phrase tone to be predicted.

  According to the phrase end tone prediction apparatus 100 of the present invention described above, a large amount of text data to which an accurate phrase end tone label is assigned is used when a large amount of speech is synthesized from a conversation sentence expressed as text. Can be generated. The text data generated by the end-of-speech tone prediction device 100 can be used as input data for the speech synthesizer, and the text data is accurately given the presence or absence of the end-speech tone, so that the synthesized speech is also rich in expression. It can be voice.

Note that the processes described in the above method and apparatus are not only executed in time series according to the order of description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (1)

Using morpheme information and accent phrase information as input, feature information on the appearance form, part of speech, reading, accent phrase accent type, presence of accent phrase end pose, etc. required by the phrase end tone prediction model A feature information extracting unit to extract;
A word information database for storing bit strings corresponding to appearance forms and parts of speech of all words to be handled;
A feature amount conversion unit that generates the feature amount vector corresponding to the feature amount information with reference to the bit string stored in the word information database, using the feature amount information output by the feature amount information extraction unit;
A model boundary of presence / absence of end-of-sound tone is modeled, and an end-of- speech tone prediction model, which is a prediction model that binarizes the input feature vector into presence / absence of end- of- pitch tone,
When the feature value vector output from the feature value conversion unit is input, the feature value vector is binary-classified by the above-mentioned phrase end tone prediction model, and when there is a phrase end tone, a phrase end tone label is assigned to the accent phrase A prediction unit to
An end-of-phrase tone prediction apparatus comprising:

Images