JP6523423B2 - Speech synthesizer, speech synthesis method and program - Google Patents

Description

Embodiments of the present invention, the voice if NaruSo location, a speech if Narukata method and program.

  Speech synthesis not only allows you to select and read voices from a few prepared in advance, but also generates a new speech synthesis dictionary for the voice of a specific speaker, such as a celebrity or a familiar person, and various text contents There is a growing need to make people read. In order to meet such needs, there has been proposed a technique for automatically generating a speech synthesis dictionary from speech data of a target speaker to be generated as a dictionary. In addition, as a technology for generating a speech synthesis dictionary from a small amount of speech data of a target speaker, a model prepared in advance and representing the average characteristics of a plurality of speakers is converted to be closer to the target speaker's characteristics. There is a technique of speaker adaptation that generates a model of a target speaker.

  Conventional techniques for automatically generating a speech synthesis dictionary are mainly intended to make the speech and the speech of the target speaker as similar as possible. However, target speakers targeted for dictionary generation include not only professional narrators and voice actors, but also general speakers who have not received any training on speech production. For this reason, if the target speaker's speech skill is low, the low skill of the target speaker is faithfully reproduced, and the speech synthesis dictionary is difficult to use in some applications.

  There is also a need to generate not only the native language of the target speaker but also a speech synthesis dictionary of a foreign language by the voice of the target speaker. In order to meet this need, it is possible to generate a speech synthesis dictionary of the language from the recorded speech if speech in which the target speaker is made to read the foreign language can be recorded. However, when a speech synthesis dictionary is generated from the recorded speech of an incorrect speech or an unnatural speech with an accent as the speech of the language, the characteristics of the speech are reflected, and the speech synthesis dictionary becomes incomprehensible even by native speakers. I will.

JP, 2013-72903, A JP 2002-244689 A

An object of the present invention is to provide a speech if NaruSo location that can generate speech synthesis dictionary adjusting the speaker of the similarity in accordance with the speech skills and native degree of a target, the voice multiplexer Narukata method and program It is to provide.

Speech if NaruSo location embodiment comprises a voice analyzer, and a speaker adaptation section, and the target speaker level specifying unit, a determination unit, and a speech synthesis unit. The voice analysis unit analyzes voice data of any target speaker and generates a voice database including data representing the features of the target speaker's speech. The speaker adaptation unit performs speaker adaptation that converts a predetermined base model so as to be close to the feature of the target speaker based on the voice database, and generates a model of the target speaker. Target speaker level specifying unit, for speaker level representative of at least one of the native level of the speaker with respect to the language of the speaker's speech skills and speech synthesis dictionary, the target speaker level is the speaker the target level Accept the specification of The determination unit determines the fidelity of the speaker reproduction in the speaker adaptation according to the relationship between the designated target speaker level and the target speaker level which is the speaker level of the target speaker. Determine the values of the parameters involved. The speech synthesis unit generates a speech waveform according to the value of the parameter .

FIG. 1 is a block diagram showing a configuration example of a speech synthesis dictionary generation device according to a first embodiment. FIG. 1 is a block diagram showing a schematic configuration of a speech synthesizer. The conceptual diagram of the piecewise linear regression used by the speaker adaptation of HMM system. The figure which shows an example of the method a determination part determines the value of a parameter. FIG. 7 is a block diagram showing an example of the configuration of a speech synthesis dictionary generation device according to a second embodiment. FIG. 11 is a block diagram showing an example of the configuration of a speech synthesis dictionary generation device according to a third embodiment. The figure which shows the example of a display of GUI which designates a target speaker level. The conceptual diagram of speaker adaptation using the model learned by cluster adaptive learning. The conceptual diagram which shows the relationship between the interpolation ratio r in Formula (2), and the weight vector of a target. FIG. 16 is a block diagram showing an example of the configuration of a speech synthesis dictionary generation device according to a sixth embodiment;

First Embodiment
FIG. 1 is a block diagram showing a configuration example of the speech synthesis dictionary generation device 100 of the present embodiment. As shown in FIG. 1, the speech synthesis dictionary generation apparatus 100 according to this embodiment includes a speech analysis unit 101, a speaker adaptation unit 102, a target speaker level designation unit 103, and a target speaker level designation unit 104. And a determination unit 105. When the speech synthesis dictionary generation device 100 receives the recorded speech 10 of an arbitrary target speaker to be generated as a dictionary and the text 20 (hereinafter referred to as “recorded text”) corresponding to the read contents, A speech synthesis dictionary 30 is generated that includes a model of a target speaker modeling the speaker's voice quality and speech.

  Among the above configurations, the target speaker level designation unit 103, the target speaker level designation unit 104, and the determination unit 105 are components unique to the present embodiment, but the technique of speaker adaptation is This configuration is general to the speech synthesis dictionary generation device used.

  The speech synthesis dictionary 30 generated by the speech synthesis dictionary generation device 100 according to the present embodiment is data necessary for the speech synthesis device, and it is an acoustic model that models voice quality, and a prosody that models prosody such as intonation / rhythm. It contains models and various other information necessary for speech synthesis. The speech synthesizer is generally composed of a language processing unit 40 and a speech synthesis unit 50 as shown in FIG. 2, and when a text is input, it generates a speech waveform for that. The language processing unit 40 analyzes the input text, acquires various language information such as the reading and accent of the text, the position of the pose, and other word boundaries and parts of speech, and passes it to the speech synthesis unit 50. The speech synthesis unit 50 generates a prosody pattern such as intonation / rhythm using the prosody model included in the speech synthesis dictionary 30 based on the above information, and further uses the acoustic model included in the speech synthesis dictionary 30 for speech Generate a waveform.

  In the case of an HMM (Hidden Markov Model) -based system such as that described in Patent Document 2, the prosody model and the acoustic model included in the speech synthesis dictionary 30 are phonetic and / or acoustic models obtained by linguistic analysis of text. It is what modeled the correspondence with linguistic information and parameter sequences, such as a prosody and an acoustic. Specifically, it consists of a decision tree in which each parameter is clustered in a phonological language environment for each state, and a probability distribution of parameters assigned to each leaf node of the decision tree. As the prosody parameter, there are a pitch parameter that represents the pitch of the voice, and a duration that represents the length of the sound. Further, as the acoustic parameter, there are a spectral parameter indicating the feature of the vocal tract, an aperiodicity index indicating the degree of aperiodicity of the sound source signal, and the like. The state refers to the internal state when the time change of each parameter is modeled by HMM. Usually, each phoneme section includes 3 to 5 states because it is modeled by a 3 to 5 HMM transitioning from left to right with no backtracking. Therefore, for example, in the decision tree for the first state of the pitch parameter, the probability distribution of the pitch value of the leading section in the phoneme section is clustered in the phonological language environment, and based on the phonological language information about the target phoneme section By tracing the decision tree, it is possible to obtain the probability distribution of the pitch parameter of the leading section of the phoneme. As a probability distribution of parameters, a normal distribution is often used, and in that case, it is expressed by an average vector representing the center of the distribution and a covariance matrix representing the spread of the distribution.

  The speech synthesis unit 50 selects the probability distribution for each state of each parameter with the decision tree as described above, generates a parameter sequence having the maximum probability based on these probability distributions, and generates these parameter sequences. Generate an audio waveform based on it. In the case of a general HMM-based system, a sound source waveform is generated based on the generated pitch parameter and aperiodic index, and a vocal tract filter whose filter characteristics change with time according to the generated spectral parameter is generated on this sound source waveform. By convoluting, an audio waveform is generated.

  The speech analysis unit 101 analyzes the recorded speech 10 and the recorded text 20 input to the speech synthesis dictionary generation device 100, and generates a speech database (hereinafter referred to as a speech DB) 110. The speech DB 110 includes various types of sound and prosody data required for speaker adaptation, that is, data representing the features of the speech of the target speaker. Specifically, a time series such as a spectral parameter representing the feature of the spectral envelope, an aperiodic index representing the ratio of aperiodic components in each frequency band, a pitch parameter representing the fundamental frequency (F0) (for example, every frame) , A sequence of labels such as phonemes, time information (such as start time and end time of phonemes) and language information (such as accents and headings of words including phonemes, parts of speech, connection strength with words before and after, etc.) Information on the position and length of the pose is included in the voice DB 110. The voice DB 110 includes at least a part of these pieces of information, but may include information other than those listed here. In addition, although mel frequency cepstrum (mel cepstrum) and mel frequency line spectrum pair (mel LSP) are generally and commonly used as spectral parameters, any parameters representing the characteristics of the spectral envelope may be used. It is also good.

  In the speech analysis unit 101, in order to generate these pieces of information included in the speech DB 110, processes such as phoneme labeling, fundamental frequency extraction, spectral envelope extraction, aperiodicity index extraction, linguistic information extraction and the like are automatically performed. There are several existing methods for each of these processes, and any of them may be used, or another new method may be used. For example, in phoneme labeling, a method using an HMM is generally used. There are many fundamental frequency extraction methods such as a method using autocorrelation of speech waveform, a method using cepstrum, a method using a harmonic structure of spectrum, and the like. There are many methods for spectral envelope extraction, such as a method using pitch synchronization analysis, a method using cepstrum, and a method called STRAIGHT. For aperiodic index extraction, the speech waveform is divided into periodic components and non-periodic components by a method using autocorrelation in the speech waveform of each frequency band or a method called PSHF to obtain the power ratio for each frequency band Methods etc. exist. In the linguistic information extraction, information such as accent information, part-of-speech and connection strength between words are obtained from the result of linguistic processing such as morphological analysis.

  The speech DB 110 generated by the speech analysis unit 101 is used together with the speaker adaptation base model 120 to generate a model of the target speaker in the speaker adaptation unit 102.

  Similar to the model included in the speech synthesis dictionary 30, the speaker adaptation base model 120 includes phonological and linguistic information obtained by linguistic analysis of text, and a parameter series such as spectral parameters, pitch parameters, and aperiodicity index. Modeling the correspondence relationship between Usually, a model representing the average characteristics of these speakers is learned from a large amount of speech data of a plurality of persons, and a model covering a wide range of phonological and language environments is used as a base model 120 for speaker adaptation. For example, in the case of an HMM-based system as described in Patent Document 2, the speaker adaptation base model 120 is assigned to a decision tree obtained by clustering each parameter in a phonological language environment and each leaf node of the decision tree. It consists of the probability distribution of different parameters.

As a learning method of the speaker adaptation base model 120, as described in Patent Document 2, from the speech data of a plurality of speakers, a general model learning method of HMM speech synthesis is used. "Speaker model", or as described in reference 1 below, normalizing speaker-to-speaker feature variances using a scheme called Speaker Adaptive Training (SAT) There is a way to learn.
(Reference 1) J. Yamagishi and T. Kobayashi, "Average-Voice-Based Speech Synthesis Using HSMM-Based Speaker Adaptation and Adaptive Training", IEICE Trans. Information and Systems, Vol. No. 2, pp. 533-543 (2007-2).

  In the present embodiment, it is assumed that the speaker adaptation base model 120 is learned from speech data of a plurality of speakers who are native to the language and who have high vocal skills.

The speaker adaptation unit 102 performs speaker adaptation that converts the speaker adaptation base model 120 so as to be closer to the features of the target speaker (speaker of the recorded speech 10) using the voice DB 110, Generate a voice quality / speaking model close to. Here, the probability distribution of the speaker adaptation base model 120 is obtained using a technique such as maximum likelihood linear regression (MLLR), constrained maximum likelihood linear regression (cMLLR), or structural a posteriori established maximum linear regression (SMAPLR). By optimizing in accordance with the parameters of the speech DB 110, the speaker adaptation base model 120 approaches the characteristics of the target speaker. For example, in the case of a method using maximum likelihood linear regression, the mean vector μ _i of the probability distribution of the parameters assigned to the leaf node i in the decision tree is transformed as in the following equation (1). Here, A and W are matrices, B and ξ _i are vectors, ξ _i = [1, μ _i ^T ] ^T (T is transpose), W = [b A], and W is called a regression matrix.

  In the conversion of Equation (1), the conversion is performed after the regression matrix W is optimized so that the likelihood of the probability distribution after conversion is maximized with respect to the parameters of the model of the target speaker. In addition to the mean vector of the probability distribution, the covariance matrix may also be transformed, but details are omitted here.

  In such transformation by maximum likelihood linear regression, the probability distribution of all leaf nodes of the decision tree may be transformed by one common regression matrix, but in general, the difference in the speaker nature differs depending on the phoneme etc. Is a very rough conversion, and the speaker nature of the target speaker may not be sufficiently reproduced, and furthermore, the phonological property may also be lost. On the other hand, when there is a large amount of voice data of the target speaker, very precise speaker adaptation is possible by preparing a different regression matrix for each probability distribution of each leaf node, but speaker adaptation is used In many cases, the voice data of the target speaker is small, so the voice data of the target speaker assigned to each leaf node may be very small or not at all, and many leaf nodes can not calculate the regression matrix. It will come out.

  Therefore, usually, the probability distribution of the transformation source is clustered into a plurality of regression classes, and a transformation matrix is determined for each regression class to transform the probability distribution. Such transformation is called piecewise linear regression. The image is shown in FIG. In clustering to a regression class, as shown in FIG. 3, all leaves are usually determined based on the distance between a decision tree (usually a binary tree) of a base model for speaker adaptation 120 (usually a binary tree) clustered in a phonological language environment as shown in FIG. A binary tree is used which is the result of clustering the probability distribution of nodes by physical quantities (hereinafter, these decision trees and binary trees are called regression class trees). In these methods, a minimum threshold is set for the speech data volume of the target speaker per regression class, and the granularity of the regression class is controlled according to the speech data volume of the target speaker.

  Specifically, first, it is examined to which leaf node of the regression class tree each sample of the target speaker's parameters is assigned, and the number of samples assigned to each leaf node is calculated. If there is a leaf node for which the number of allocated samples is less than the threshold value, the leaf nodes below the parent node are merged back to the parent node. This operation is repeated until the number of samples of all leaf nodes exceeds the minimum threshold, and each leaf node finally generated becomes a regression class. As a result, when the amount of speech data of the target speaker is small, each regression class is large (that is, the number of transformation matrices is small) and coarse-grained adaptation is performed, and when the amount of speech data is large, each regression class is large (i.e., The number of transformation matrices is reduced, resulting in fine-grained adaptation.

  In the present embodiment, as described above, the speaker adaptation unit 102 obtains a transformation matrix for each regression class and performs transformation of probability distribution, such as a minimum threshold for the speech data amount of the target speaker per regression class. , It is assumed that the parameters of the regression class granularity (that is, the fidelity of the speaker reproduction in the speaker adaptation) can be externally controlled. For example, when controlling the granularity of the regression class by setting the minimum threshold to the speech data amount of the target speaker per regression class, usually, using fixed values empirically obtained for each type of prosody / sound parameter, It is often set to a relatively small value within the range of sufficient data volume that the transformation matrix can calculate. In this case, the voice quality and characteristics of the target speaker can be reproduced as faithfully as possible according to the amount of available voice data.

  On the other hand, if such a minimum threshold is set to a larger value, the regression class becomes larger, resulting in coarse-grained adaptation. In this case, a model is generated that reflects the characteristics of the speaker adaptation base model 120, although the sound quality and the manner of speech of the target speaker are generally approached. That is, by increasing the minimum threshold, it is possible to lower the fidelity of the speaker reproduction in the speaker adaptation. In the present embodiment, in the determination unit 105 to be described later, the values of such parameters are related to the speaker level of the target speaker and the target speaker level (speaker level expected for synthetic speech by the speech synthesis dictionary 30). And is input to the speaker adaptation unit 102.

  The term "speaker level" used in the present embodiment represents at least one of the speaker's speech skills and the degree of nativeness of the speaker to the language of the speech synthesis dictionary 30 to be generated. The speaker level of the target speaker is called "target speaker level", and the target speaker level is called "target speaker level". The speaker's speech skill is a numerical value or classification that represents the accuracy of the speaker's pronunciation or accent and the fluency of the speech. For example, a speaker with a very traceable speech can produce accurate and fluent speech. If it is a professional announcer, it will be represented by a number such as 100. The native degree of a speaker is a numerical value or classification indicating whether the target language is the native language for the speaker, and how much the speaking skill of the language is otherwise. For example, it is 100 in the case of the native language, 0 in the case of a language that has not even been learned. Depending on the application, the speaker level may be either one or both of the speaking skill and the native degree. In addition, an index in which the speaking skill and the native degree are combined may be set as the speaker level.

  The target speaker level specification unit 103 receives specification of a target speaker level, and passes the specified target speaker level to the determination unit 105. For example, when the user such as the target speaker performs an operation to specify the target speaker level using some user interface, the target speaker level specification unit 103 receives specification of the target speaker level by the operation of the user. To the determining unit 105. If the target speaker level can be assumed depending on the application of the speech synthesis dictionary 30 to be generated, a fixed assumed value may be set in advance as the target speaker level. In this case, the speech synthesis dictionary generation device 100 includes a storage unit for storing a preset target speaker level instead of the target speaker level designation unit 103.

  The target speaker level designation unit 104 receives designation of a target speaker level, and passes the designated target speaker level to the determination unit 105. For example, when a user such as a target speaker performs an operation to specify a target speaker level using any user interface, the target speaker level specification unit 104 receives specification of a target speaker level by this user operation. To the determining unit 105. For example, when the target speaker's speech skill or native degree is low, there is a case where he / she wants to make a voice similar to that of the target speaker, and more professional or native than the target speaker. In such a case, the user may specify a higher target speaker level.

  The determination unit 105 determines the above-mentioned speaker adaptation unit according to the relationship between the target speaker level passed from the target speaker level designation unit 104 and the target speaker level passed from the target speaker level designation unit 103. The values of parameters related to the fidelity of the speaker reproduction in the speaker adaptation according to 102 are determined.

  An example of a method in which the determination unit 105 determines the value of the parameter is shown in FIG. FIG. 4 shows a two-dimensional plane for classifying the relationship between the target speaker level and the target speaker level, the horizontal axis corresponding to the size of the target speaker level, and the vertical axis the size of the target speaker level Corresponds to The oblique broken lines in the figure indicate positions where the target speaker level and the target speaker level are equal. For example, in the determination unit 105, the relationship between the target speaker level passed from the target speaker level designation unit 104 and the target speaker level passed from the target person level designation unit 103 corresponds to the regions A to D in FIG. Determine which of the following applies. Then, when the relationship between the target speaker level and the target speaker level is in the region A, the determination unit 105 determines the value of the parameter related to the fidelity of the speaker reproduction and the fidelity of the speaker reproduction is the maximum. The default value determined in advance as the value to be Region A is a region that applies when the target speaker level is below the target speaker level or when the target speaker level is higher than the target speaker level but the difference is less than a predetermined value. Region A contains the case where the target speaker level is higher than the target speaker level but the difference is less than the predetermined value means that the parameter value is set to the default value in consideration of the speaker level uncertainty. To have a margin in the However, such a margin is not necessarily required, and only the region (lower right region than the oblique broken line in the figure) applicable when the target speaker level is equal to or lower than the target speaker level may be region A.

  In addition, when the relationship between the target speaker level and the target speaker level is in the region B, the determination unit 105 sets the value of the parameter related to the fidelity of the speaker reproduction to that of the speaker reproduction more than the default value. Decide on a value that reduces fidelity. In addition, when the relationship between the target speaker level and the target speaker level is in the region C, the determination unit 105 determines the values of the parameters related to the fidelity of the speaker reproduction, the target speaker level and the target speaker level. Further, it is determined that the fidelity of the speaker reproduction is lower than in the case where the relationship with the above applies to the region B. In addition, when the relationship between the target speaker level and the target speaker level is applicable to the region D, the determination unit 105 determines the value of the parameter related to the fidelity of the speaker reproduction, the target speaker level and the target speaker level. Further, it is determined that the fidelity of the speaker reproduction is lower than in the case where the relationship with the above applies to the region C.

  As described above, when the target speaker level is higher than the target speaker level, the determination unit 105 sets the parameter value related to the fidelity of the speaker reproduction to the fidelity of the speaker reproduction more than the default value. Is determined to be a lower value, and the value of the parameter is determined such that the greater the difference, the lower the fidelity of the speaker reproduction. At this time, among the models of the target speaker generated by speaker adaptation, the degree of change of the parameters may be changed according to the parameters used for generating the acoustic model and the parameters used for generating the prosody model.

  In many speakers, the speaker nature appears stronger in voice quality than in prosody, so the voice quality needs to be faithfully reproduced, but if even the average level of prosody is matched to the speaker, the speaker nature is to some extent It can often be reproduced. Also, for many speakers, it is relatively easy to pronounce each syllable in a sentence so that it can be correctly heard, but for prosody such as accent, intonation, and rhythm, natural and easy-to-hear reading like pronarator etc. Things are difficult unless you have a good deal of training. The same applies to reading foreign languages. For example, if a Japanese speaker who has never learned Chinese reads Chinese, each syllable is to a certain extent by reading Chinese pinyins or those converted to kana. It can be pronounced correctly, but it is almost impossible to read it in the correct tone (in the case of Mandarin Chinese, four voices). Therefore, when determining the value of the parameter related to the fidelity of the speaker reproduction, to determine the fidelity of the speaker reproduction lower than the default value, the degree of change to the default value of the parameter used to generate the acoustic model By making the degree of change to the default value of the parameter used for generating the prosody model larger than that, it becomes possible to easily generate the speech synthesis dictionary 30 which has both the reproduction of the speaker nature and the height of the speech skill. .

  For example, in the case of using the minimum threshold for the voice data volume of the target speaker per regression class described above as the parameter related to the fidelity of the speaker reproduction, the relationship between the target speaker level and the target speaker level is shown in FIG. In the case of region B, the value of the parameter used to generate the acoustic model is 10 times the default value, and the value of the parameter used to generate the prosody model is 10 times the default value. Also, if the relationship between the target speaker level and the target speaker level applies to region C in FIG. 4, the parameter value used to generate the acoustic model is 30 times the default value, and the parameter value used to generate the prosody model Let be 100 times the default value. Also, if the relationship between the target speaker level and the target speaker level applies to region D in FIG. 4, the parameter value used to generate the acoustic model is set to 100 times the default value, and the parameter value used to generate the prosody model Is considered to be 1000 times the default value.

  As described above, in the speech synthesis dictionary generation apparatus 100 of this embodiment, when a target speaker level higher than the target speaker level is specified, the fidelity of the speaker reproducibility in the speaker adaptation is automatically Overall, it is close to the speaker's voice quality and the way of speech, but the fine features are speech with the characteristics of the base model for speaker adaptation 120, that is, the speech skill and the native feature of the language A synthetic dictionary 30 is generated. As described above, according to the speech synthesis dictionary generation device 100 of the present embodiment, the speech synthesis dictionary 30 can be generated in which the similarity of the speaker characteristics is adjusted according to the target speech skill and the native degree. Even when the speaker's speech skill is low, speech synthesis with high speech skill can be realized, and even when the native degree of the target speaker is low, speech synthesis of speech close to native can be realized.

Second Embodiment
In the first embodiment, the target speaker level is specified by the user such as the target speaker, or a fixed assumed value is set in advance. However, it is very difficult to specify and set an appropriate target speaker level suitable for the actual speech skill and native degree in the recorded speech 10. Therefore, in the present embodiment, the target speaker level is estimated based on the analysis result of the voice data of the target speaker by the voice analysis unit 101, and the relationship between the designated target speaker level and the estimated target speaker level. In accordance with, determine the value of the parameter related to the fidelity of the speaker reproduction.

  FIG. 5 is a block diagram showing a configuration example of the speech synthesis dictionary generation device 200 of the present embodiment. As shown in FIG. 5, the speech synthesis dictionary generation apparatus 200 of this embodiment includes a target speaker level estimation unit 201 in place of the target speaker level designation unit 103 shown in FIG. The other configuration is the same as that of the first embodiment, and therefore, the same components as those of the first embodiment are denoted by the same reference numerals in the drawings, and the redundant description will be omitted.

The target speaker level estimation unit 201 determines the speech skill and the native degree of the target speaker based on the result of the phoneme labeling by the speech analysis unit 101 and the extracted information such as the pitch and the pose. For example, a target speaker with low speech skills tends to have a higher frequency of pauses than a speaker who can speak fluently, so this information can be used to determine the speech skill of the target speaker. In addition, there are various techniques for automatically determining the speaker's speech skill from the recorded voice, for the purpose of language learning and the like, and an example is disclosed in the following reference 2 as an example.
(Reference 2) Japanese Patent Application Laid-Open No. 2006-201491
In the technique described in this reference 2, an HMM model is used as teacher data, and an evaluation value on the pronunciation level of the speaker is calculated from probability values obtained as a result of aligning the speaker's voice using this. Any of such existing techniques may be used.

  As described above, according to the speech synthesis dictionary generation device 200 of the present embodiment, an appropriate target speaker level that matches the actual speaker level in the recorded speech 10 is automatically determined, so the designated target speech It is possible to generate a speech synthesis dictionary 30 that appropriately reflects the user level.

Third Embodiment
The target speaker level specified by the user not only affects the speech level or native degree of the speech synthesis dictionary 30 (the model of the target speaker) to be generated, but in fact it is a tradeoff of the degree of similarity of the target speaker Will be adjusted. That is, if the target speaker level higher than the target speaker's speech level or the native level is set, the similarity of the target speaker's speaker nature is somewhat sacrificed. However, in the first and second embodiments, since the user only specifies the target speaker level, it is difficult to imagine what speech synthesis dictionary 30 is finally generated. Also, although the range within which such tradeoffs can actually be adjusted is limited to some extent by the speech level and native level of the recorded speech 10, the user can not set the target speaker level without knowing this in advance. There is a need.

  Therefore, in the present embodiment, the target speaker level specified according to the input recorded speech 10 and the speaker characteristics assumed by the speech synthesis dictionary 30 (the model of the target speaker) generated as a result thereof The relationship between the degree of similarity and the specifiable range of the target speaker level are presented to the user, for example, by display on a GUI, and when the target speaker level is designated, any speech synthesis dictionary 30 is generated. Allow the user to image what

  FIG. 6 is a block diagram showing a configuration example of the speech synthesis dictionary generation device 300 of the present embodiment. As shown in FIG. 6, the speech synthesis dictionary generation apparatus 300 of this embodiment includes a target speaker level presentation / designation unit 301 in place of the target speaker level designation unit 104 shown in FIG. The other configuration is the same as that of the first and second embodiments, and therefore, the same components as those of the first and second embodiments are denoted by the same reference numerals in the drawings, and redundant description will be omitted. .

  In the speech synthesis dictionary generation apparatus 300 of this embodiment, when the recorded speech 10 is input, the target speaker level estimation unit 201 estimates the target speaker level, and the estimated target speaker level is the target speaker level. It is passed to the presentation / designation unit 301.

  Based on the target speaker level estimated by target speaker level estimation section 201, target speaker level presentation / designation section 301 sets a target speaker level range that can be designated and a target speaker level within this range. The relationship between the degree of similarity of speakeriness assumed in the speech synthesis dictionary 30 is obtained, for example, displayed on a GUI, and the user accepts an operation of designating a target speaker level using this GUI.

  An example of display by this GUI is shown in FIG. FIG. 7 (a) is a display example of the GUI when the target speaker level is estimated to be relatively high, and FIG. 7 (b) is a display example of the GUI when the target speaker level is estimated to be low. is there. In these GUIs, a slider S is provided to indicate a specifiable range of the target speaker level, and the user moves the pointer P in the slider S to specify the target speaker level. The slider S is diagonally displayed on the GUI, and the position of the pointer P in the slider S is assumed to be a target speaker level designated and a speech assumed by the generated speech synthesis dictionary 30 (model of the target speaker) It shows the relationship between the similarity of humanity. The broken circles in the figure indicate the speaker level and the similarity of the speakerness for each of the case where the speaker adaptation base model 120 is used as it is and the case where the recorded speech 10 is faithfully reproduced. It is The speaker adaptation base model 120 is located at the upper left of the figure because the speaker level is high but the voice / speaking style is completely different from that of the target speaker. On the other hand, the recording voice 10 is positioned at the right end of the figure for the target speaker itself, and the upper and lower positions change according to the height of the target speaker level. The slider S is located between two dashed circles, but in the setting that faithfully reproduces the target speaker, the similarity between the speaker level and the speaker characteristic is close to the recorded speech 10, Setting the target speaker level high indicates that the speaker adaptation is performed with coarse granularity, and the similarity of the speakerality is sacrificed to some extent. As shown in FIG. 7, the larger the difference between the speaker levels of the speaker adaptation base model 120 and the recorded speech 10, the wider the range of target speaker levels that can be set.

  The target speaker level specified by the user using the GUI illustrated in FIG. 7 is passed to the determination unit 105, and speaker adaptation is performed based on the relationship with the target speaker level passed from the target speaker level estimation unit 201. The value of the parameter relating to the speaker's fidelity at is determined by the determination unit 105. The speaker adaptation unit 102 generates the speech synthesis dictionary 30 having the speaker level and the similarity of the speakerality intended by the user by performing the speaker adaptation according to the value of the determined parameter. Can.

Fourth Embodiment
In the first to third embodiments, an example using a general speaker adaptation method in HMM speech synthesis has been described, but if it has parameters relating to the fidelity of the speaker reproduction, the first to third embodiments will be described. A speaker adaptation scheme different from that of the third embodiment may be used.

As one of the different speaker adaptation methods, there is a speaker adaptation method using a model learned by cluster adaptive training (CAT) as described in reference 3 below. In this embodiment, it is assumed that a speaker adaptation method using a model learned by this cluster adaptive learning is used.
(Reference 3) K. Yanagisawa, J. et al. Latorre, V .; Wan, M .; Gales and S. King, "Noise Robustness in HMM-TTS Speaker Adaptation" Proc. of 8th ISCA Speech Synthesis Workshop, pp. 119-124, 2013-9

  In cluster adaptive learning, a model is represented by a weighted sum of a plurality of clusters, and at the time of model learning, the model and weights of each cluster are simultaneously optimized according to the data. In the modeling of multiple speakers for speaker adaptation used in the present embodiment, as shown in FIG. 8, a decision tree modeling each cluster from a large amount of speech data including multiple speakers, and clusters Optimize the weights simultaneously. By setting the weight of the model thus created to a value optimized for each speaker used for learning, the characteristics of each speaker can be reproduced. The resulting model is hereinafter referred to as the CAT model.

  In practice, the CAT model is learned for each parameter type such as spectrum parameters and pitch parameters, as in the decision tree described in the first embodiment. The decision tree of each cluster is obtained by clustering each parameter in the phonological language environment, and the probability distribution of the target parameter (coincident with the average vector) is given to the leaf node of the cluster whose weight called bias cluster is always set to 1. A dispersion matrix is assigned, and leaf nodes of other clusters are assigned average vectors weightedly adding to the average vectors of probability distributions from the bias cluster.

  In this embodiment, the CAT model thus learned by cluster adaptive learning is used as the speaker adaptation base model 120. In the speaker adaptation in this case, it is possible to obtain a voice quality / speaking model close to the target speaker by optimizing the weights in accordance with the voice data of the target speaker. However, since this CAT model can usually represent only features in the space that can be represented by a linear sum of speaker features used for learning, for example, when the speaker used for learning is only a professional narrator, it can be used by ordinary people. Voice quality and speech may not be reproduced well. Therefore, in the present embodiment, the CAT model is learned from a plurality of speakers having various speaker levels and including various voice characteristics and speaking characteristics.

In this case, assuming that the weight vector optimized for voice data of the target speaker is W _opt , the speech synthesized with this weight W _opt is close to the target speaker, but the speaker level also reproduces the level of the target speaker It becomes a thing. On the other hand, among the speakers used for learning of the CAT model, it is assumed that the one closest to W _opt is selected from among the weight vectors optimized for the speaker with high speaker level and this is taken as W _{s (near)} The speech synthesized with this weight W _{s (near)} is relatively close to the target speaker and becomes high in speaker level. Here, W _{s (near)} is assumed to be closest to W _opt here, but it is not necessary to select by the distance of weight vector, and it is possible to select based on other information such as speaker gender and features Good.

Further, in the present embodiment, a weight vector W _target obtained by interpolating W _opt and W _{s (near)} is newly defined as in the following Expression (2), and a weight vector of a result of speaker adaptation of W _target ((2) Let it be the target weight vector).

FIG. 9 is a conceptual diagram showing the relationship between the interpolation ratio r in equation (2) and the _target weight vector W _target determined thereby. In this case, for example, when the interpolation ratio r is 1, it is set to reproduce the target speaker most faithfully, and when the interpolation ratio r is 0, the speaker level can be set to the highest. That is, this interpolation ratio r can be used as a parameter representing the fidelity of the speaker reproducibility. In the present embodiment, the determination unit 105 determines the value of the interpolation ratio r based on the relationship between the target speaker level and the target speaker level. As a result, as in the first to third embodiments, it is possible to generate the speech synthesis dictionary 30 in which the similarity of the speaker nature is adjusted according to the target speech skill and the native degree. Even when the speech skill is low, speech synthesis with high speech skill can be realized, and even when the native degree of the target speaker is low, speech synthesis of speech close to native speech can be realized.

Fifth Embodiment
In the first to fourth embodiments, the example of generating the speech synthesis dictionary 30 for HMM speech synthesis has been described, but the method of speech synthesis is not limited to HMM speech synthesis, and unit selection type speech synthesis, etc. Different speech synthesis methods may be used. For example, also in speech selection of segment selection type, there is a speaker adaptation method as disclosed in reference 4 below.
(Reference 4) Japanese Patent Application Laid-Open No. 2007-193139

  In the speaker adaptation method disclosed in reference 4, the speech segment of the base speaker is converted in accordance with the characteristics of the target speaker (target speaker). Specifically, the speech waveform of the speech segment is speech-analyzed and converted to spectral parameters, and the spectral parameters are converted to the features of the target speaker on the spectral region, and the converted spectral parameters are then converted to speech in the time domain. By converting back to the waveform, it is converted to the voice waveform of the target speaker.

  As for the conversion rule at this time, pairs of the speech unit of the base speaker and the speech unit of the target speaker are formed using the unit selection method, and these speech units are subjected to speech analysis to obtain spectral parameters It is converted into pairs, and based on these spectral parameter pairs, it is generated by modeling the conversion with regression analysis, vector quantization, and mixed Gaussian distribution (GMM). That is, as in the case of speaker adaptation in HMM speech synthesis, conversion is performed in the domain of parameters such as spectrum. In addition, some conversion methods include parameters related to the fidelity of the speaker reproduction.

  For example, among the conversion methods described in reference 4, in the method using vector quantization, the spectral parameters of the base speaker are clustered into C clusters, and the conversion matrix is generated by maximum likelihood linear regression or the like in each cluster. Generate In this case, the number C of clusters can be used as a parameter related to the fidelity of the speaker reproduction. The larger C, the higher the fidelity, and the smaller the C, the lower the fidelity. In addition, in the conversion method using GMM, the conversion rule from the base speaker to the target speaker is expressed by C Gaussian distributions. In this case, the mixing number C of the Gaussian distribution is used as the fidelity of the speaker reproduction. It can be used as a related parameter.

  In this embodiment, the cluster number C in the conversion scheme using vector quantization as described above, or the mixing number C of the Gaussian distribution in the conversion scheme using GMM is used as a parameter related to the fidelity of the speaker reproduction. Then, the determination unit 105 determines the value of the cluster number C or the value of the mixture number C of the Gaussian distribution based on the relationship between the target speaker level and the target speaker level. Thus, even when speech synthesis is performed by a method other than the HMM speech synthesis method, such as segment selection type speech synthesis, as in the first to fourth embodiments, the target speech skill and native degree The speech synthesis dictionary 30 can be generated by adjusting the similarity of the speakerality according to the case, and the speech synthesis with high speech skills is performed even when the speech skill of the target speaker is low, and the native degree of the target speaker is low However, it is possible to realize speech synthesis of speech close to native.

Sixth Embodiment
When the speaker's native degree is low, such as when generating a speech synthesis dictionary 30 of a language that can not be spoken, it is expected that speech recording in that language will be very difficult. For example, in a voice recording tool, it is difficult for Japanese speakers who do not understand Chinese to display and read Chinese text as it is. Therefore, in the present embodiment, the information of the reading of the text is converted into the reading notation of the language that the target speaker normally uses and presented to the target speaker while recording the voice and the information to be presented is the target talk Switch according to the native degree of the person.

  FIG. 10 is a block diagram showing a configuration example of the speech synthesis dictionary generation device 400 of this embodiment. As shown in FIG. 10, in addition to the configuration of the first embodiment shown in FIG. The other configuration is the same as that of the first embodiment, and therefore, the same components as those of the first embodiment are denoted by the same reference numerals in the drawings, and the redundant description will be omitted.

  When the voice recording / presenting unit 401 reads out the recorded text 20 of another language that is not the language that the target speaker normally uses, the voice recording / presentation section 401 displays the description of the recorded text 20 as the reading of the language that the target speaker normally uses. While presenting the converted display text 130 to the target speaker, the target speaker records the voice read out from the recorded text 20. For example, in the case of generating the Chinese speech synthesis dictionary 30 for Japanese, the speech recording / presentation unit 401 displays the display text 130 obtained by converting, for example, Chinese reading into katakana instead of Chinese text to be read Do. By doing this, it becomes possible for Japanese to pronounce similar to Chinese.

  At this time, the voice recording / presenting unit 401 switches the display text 130 to be presented to the target speaker in accordance with the native degree of the target speaker. That is, accents and tones can also be uttered with correct accents and tones if the speaker has learned the language. However, in the case of a speaker with a very low native degree who has never learned the language, it is very difficult to reflect it in the utterance even if the accent position and tone type are properly displayed. For example, it is almost impossible for Japanese who have not studied Chinese to correctly utter four tones that are Chinese tones.

  Therefore, the voice recording / presentation unit 401 of this embodiment switches whether to display the position of accent, the type of tone, etc., according to the native degree of the target speaker specified by the target speaker. Specifically, the voice recording / presentation unit 401 receives the native degree of the target speaker from the target speaker level designation unit 103 among the target speaker levels designated by the target speaker. Then, when the native degree of the target speaker is higher than a predetermined level, the voice recording / presentation unit 401 displays the accent position and the type of tone in addition to the reading notation. On the other hand, when the native degree of the target speaker is lower than a predetermined level, the voice recording / presenting unit 401 displays the reading notation but does not display the accent position and the tone type.

  If you do not indicate the location of the accent or the type of tone, you can not expect the correct accent or tone to be pronounced, but the target speaker does not care about the accent or tone, and thinks to concentrate on correct pronunciation. Can be expected to be correct to some extent. Therefore, when determining the value of the parameter in the determination unit 105, the parameter used for generating the acoustic model is set to a slightly higher value, while the value of the parameter used for generating the prosody model is set to be considerably lower. desirable. This increases the possibility that even a target speaker with a very low degree of nativeness can generate a speech synthesis dictionary 30 that can produce correct speech while reflecting the characteristics of the speaker.

  The target speaker level used when the determination unit 105 determines the value of the parameter is one specified by the target speaker, that is, the native speaker passed from the target speaker level designation unit 103 to the voice recording and presentation unit 401. It may be a target speaker level including a degree, or a target speaker level estimation unit 201 similar to that of the second embodiment may be separately provided, and the target speaker level estimated by the target speaker level estimation unit 201. That is, the target speaker level may be estimated using the recorded speech 10 recorded by the speech recording / presentation unit 401. Further, the value of the parameter may be determined by the determination unit 105 using both the target speaker level designated by the target speaker and the target speaker level estimated using the recorded speech 10.

  As in the present embodiment, by switching the display text 130 to be presented to the target speaker at the time of voice recording and coordinating the method of determining the value of the parameter representing the fidelity of the speaker reproducibility in the speaker adaptation. It is possible to more appropriately generate the speech synthesis dictionary 30 having a certain degree of nativeness, using the recorded speech 10 of the less native degree target speaker.

  As described above in detail with specific examples, according to the speech synthesis dictionary generation device of the embodiment, the speech synthesis in which the similarity of speaker characteristics is adjusted according to the target speech skill and the native degree A dictionary can be generated.

  Note that the speech synthesis dictionary generation device according to the above-described embodiment includes, for example, an output device (display, speaker, etc.) or an input device (keyboard) serving as a user interface to a general-purpose computer including a processor, a main storage device, an auxiliary storage device, etc. , A mouse, a touch panel, etc.) can be used. In the case of this configuration, in the speech synthesis dictionary generation device according to the embodiment, the processor installed in the computer executes a predetermined program, thereby the speech analysis unit 101, the speaker adaptation unit 102, and the target speaker level designation unit described above. The functional components such as the target speaker level designation unit 104, the determination unit 105, the target speaker level estimation unit 201, the target speaker level presentation / designation unit 301, and the voice recording / presentation unit 401 are realized. At this time, the speech synthesis dictionary generation device may be realized by installing the above program into a computer in advance, or may be stored in a storage medium such as a CD-ROM, or distributed through a network. Then, the program may be realized by appropriately installing this program on a computer. Alternatively, the above program may be executed on a server computer, and the result may be received by a client computer via a network.

  The program executed by the computer includes functional components (the speech analysis unit 101, the speaker adaptation unit 102, the target speaker level designation unit 103, the target speaker level designation) which constitute the speech synthesis dictionary generation device of the embodiment. Section 104, determination section 105, target speaker level estimation section 201, target speaker level presentation / designation section 301, voice recording / presentation section 401, etc.). As an actual hardware, for example, The processor reads out the program from the storage medium and executes the program to load the above processing units onto the main storage device and generate them on the main storage device. Note that part or all of the functional components described above can also be realized using dedicated hardware such as an ASIC or an FPGA.

  Further, various information used in the voice synthesis dictionary generation device of the embodiment is a memory built in or externally attached to the above computer, a hard disk or a recording medium such as a CD-R, a CD-RW, a DVD-RAM, a DVD-R, etc. Can be stored appropriately. For example, the speech DB 110 and the speaker adaptation base model 120 used by the speech synthesis dictionary generation device according to the embodiment can be stored by appropriately using these recording media.

  While the embodiments of the present invention have been described above, the embodiments described herein are presented as examples and are not intended to limit the scope of the invention. The novel embodiments described herein can be implemented in other various forms, and various omissions, substitutions, and modifications can be made without departing from the scope of the invention. The embodiments and the modifications thereof described herein are included in the scope and the gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

10 voice recording 20 voice recording text 30 voice synthesis dictionary 100 voice synthesis dictionary generation device 101 voice analysis unit 102 speaker adaptation unit 103 target speaker level designation unit 104 target speaker level designation unit 105 determination unit 110 voice database (voice DB)
120 Speaker Adaptive Base Model 200 Speech Synthesis Dictionary Generator 201 Target Speaker Level Estimator 300 Speech Synthesis Dictionary Generator 301 Target Speaker Level Presenting & Specifying Section 400 Speech Synthesis Dictionary Generator 401 Speech Recording & Presenting Unit

Claims (12)

Analyzing the voice data of any target speaker, the speech analysis unit for generating a speech database containing data representing a feature of the utterance of the target speaker,
A speaker adaptation unit that generates a model of the target speaker by performing speaker adaptation that converts a predetermined base model so as to be close to the characteristics of the target speaker based on the voice database;
For speaker level, which represents at least one of the native of the speaker for the language of the speaker of the speech skills and voice synthesis dictionary, the target speaker level to accept the designation of the target speaker level, which is the speaker level to target Designated section,
According to the relationship between the designated target speaker level and the target speaker level that is the speaker level of the target speaker, values of parameters related to the fidelity of the speaker reproduction in the speaker adaptation A decision unit to decide
Ruoto Koego NaruSo location and a speech synthesis unit for generating a speech waveform according to the value of the parameter.   When the designated target speaker level is higher than the target speaker level, the determination unit has lower fidelity compared to when the designated target speaker level is equal to or lower than the target speaker level. The speech synthesizer according to claim 1, wherein the value of the parameter is determined to be And a target speaker level designation unit for receiving designation of the target speaker level.
The determining portion includes a specified the target speaker level, depending on the relationship between the designated the target speaker level, according to claim 1 or 2, characterized in that to determine the value of the parameter voice if NaruSo location. The system further comprises a target speaker level estimation unit that automatically estimates the target speaker level based on at least part of data of the voice database,
The said determination part determines the value of the said parameter according to the relationship between the designated said target speaker level and the said estimated said target speaker level, The characteristic of Claim 1 or 2 characterized by the above-mentioned. voice if NaruSo location. The target speaker level designation unit determines a relationship between the target speaker level and the degree of similarity of speakerality assumed in a model of the target speaker to be generated, based on the target speaker level, and The voice according to any one of claims 1 to 4 , wherein an designating range of the target speaker level is displayed, and an operation of designating the target speaker level from the displayed range is received. if NaruSo location. The speaker adaptation section, a voice if NaruSo according to any one of claims 1 to 5, wherein the use of average voice model the speaker level models the high speaker as the base model Place. The said parameter is a parameter which defines the number of transformation matrices used for transformation of the said base model in the said speaker adaptation, The said fidelity becomes low, so that the number of the said transformation matrices is small. speech if NaruSo location according to any one of 6. The speaker adaptation unit uses, as the base model, a model represented by a weighted sum of a plurality of clusters learned by cluster adaptive learning from data of a plurality of speakers having different speaker levels, and weights the plurality of clusters. Perform the speaker adaptation by fitting a weight vector, which is a set of
The weight vector is obtained by interpolating the optimal weight vector for the target speaker and the optimal weight vector of one speaker among the plurality of speakers having the highest speaker level.
The parameters are speech if NaruSo location according to any one of claims 1 to 5, characterized in that an interpolation ratio for obtaining the weight vector. The model of the target speaker includes a prosody model and an acoustic model,
The parameters include a first parameter used to generate the prosody model and a second parameter used to generate the acoustic model,
When the determination unit determines the value of the parameter so as to lower the fidelity, the degree of change of the first parameter with respect to the default value with which the fidelity becomes higher is the change of the first parameter with respect to the default value. speech if NaruSo location according to any one of claims 2-8, characterized in that larger than change degree. It further comprises a recording unit for recording the voice data,
The recording unit records the voice data while presenting information of at least reading of a sentence to be read to the target speaker for each unit of reading;
The reading information is not the reading notation in the language to be read out, but is converted to the reading notation of the language usually used by the target speaker, and at least the native degree of the target speaker is higher than a predetermined value. when low, the voice if NaruSo location according to any one of claims 1 to 9, characterized in that does not include the symbol relating to the intonation accents and tone. Analyzing the voice data of any target speaker, the speech analysis step of generating a speech database containing data representing a feature of the utterance of the target speaker,
A speaker adaptation step of generating a model of the target speaker by performing speaker adaptation to convert a predetermined base model so as to be close to the characteristic of the target speaker based on the voice database;
For speaker level, which represents at least one of the native of the speaker for the language of the speaker of the speech skills and voice synthesis dictionary, the target speaker level to accept the designation of the target speaker level, which is the speaker level to target Designated step,
According to the relationship between the designated target speaker level and the target speaker level that is the speaker level of the target speaker, values of parameters related to the fidelity of the speaker reproduction in the speaker adaptation Decision steps to determine
Including sound Koego Narukata method and speech synthesizing step, the generating a speech waveform according to the value of the parameter. On your computer,
A voice analysis step of analyzing voice data of any target speaker to generate a voice database including data representing features of the target speaker's speech;
A speaker adaptation step of generating a model of the target speaker by performing speaker adaptation to convert a predetermined base model so as to be close to the characteristic of the target speaker based on the voice database;
For speaker level representative of at least one of the native level of the speaker with respect to the language of the speaker's speech skills and speech synthesis dictionary, the target level specified step of accepting a target speaker level specified is the speaker the target level When,
According to the relationship between the designated target speaker level and the target speaker level that is the speaker level of the target speaker, values of parameters related to the fidelity of the speaker reproduction in the speaker adaptation Decision steps to determine
Program for executing a speech synthesis step of generating a speech waveform according to the value of the parameter.

Images