JP4817250B2 - Voice quality conversion model generation device and voice quality conversion system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice quality conversion model generation device which suitably generates a voice quality conversion model capable of being adapted to voices with various properties, and a voice quality conversion system which suitably converts a voice of the voice quality of an arbitrary source speaker into a voice of the voice quality of an arbitrary target speaker by using the generated voice quality conversion model and a predetermined adapting method. <P>SOLUTION: The voice quality conversion model generation device 1 learns voice data of at least one of N source speakers and M target speakers as learning data (normal learning or adaptive learning), and generates a voice quality conversion model comprising one or two models common to at least one of the N source speakers and M target speakers. Then the voice quality conversion system adapts the generated voice quality conversion model to a voice of at least one of the arbitrary source speaker and arbitrary target speaker by using the predetermined adapting method to convert the voice of the arbitrary or specified source speaker into the voice of the voice quality of the arbitrary target speaker. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to a voice quality conversion model generation device, a voice quality conversion model generation program, and a voice quality conversion model generation for generating a voice quality conversion model for converting voice of a predetermined original speaker's voice quality into voice of a predetermined target speaker's voice quality The present invention relates to a method, a voice quality conversion system, a voice quality conversion program, and a voice quality conversion method, which convert voice of an arbitrary original speaker into voice of an arbitrary target speaker using the voice quality conversion model.

Conventionally, a voice quality conversion model that converts voice quality of one former speaker into voice quality of one target speaker is generated using voice data of the same utterance content, and this voice quality conversion model is determined in advance. The adaptation method is used to adapt to the voice of the voice quality of other original speakers and other target speakers, and using the model after the adaptation, any voice of the other original speakers is adapted to other target speakers. (For example, Non-patent Document 1).
A. Mouchtaris and JV. der Spiegel and P.M. Mucller “Non-parallel training for voice conversion by maximum licklihood constrained adaptation” ICASSP2004, vol. 1 pp. 1-4

  However, in the conventional technique of Non-Patent Document 1, the voice quality conversion model to be applied is the voice of two speakers in total, that is, the voice data of one former speaker and the voice data of one target speaker. Since it is generated from the data, the sound quality of the other person's voice converted by the model after adaptation depends only on the conversion rule between the two speakers. In other words, the voice quality after conversion may be deteriorated with respect to the voice of another person who is far from the voice data of the two speakers. In particular, when the number of voice data of others used for adaptation is small, the sound quality after conversion may be remarkably deteriorated.

  Therefore, the present invention has been made paying attention to such an unsolved problem of the conventional technology, and is suitable for generating a voice quality conversion model that can be adapted to speech of various properties. A voice quality conversion model generation apparatus, a voice quality conversion model generation program, a voice quality conversion model generation method, and a voice quality of an arbitrary original speaker by using the voice quality conversion model and a predetermined adaptation method. It is an object of the present invention to provide a voice quality conversion system, a voice quality conversion program, and a voice quality conversion method that are suitable for converting to a voice.

In order to achieve the above object, a voice quality conversion model generation device according to claim 1 according to the present invention comprises:
A voice quality conversion model generation device for generating a statistical model for voice quality conversion,
First voice data which is voice data of a predetermined N (N is an integer of 1 or more) people, and predetermined M (M is an integer of 1 or more) of the same utterance content as the first voice data initial voice conversion model generation for generating initial voice conversion model is a common statistical model based on the speaker and the M's target speaker of the N people using the second audio data is audio data of the target speaker and means,
Converting the voice quality of each of the N original speakers into the voice quality of each of the M target speakers from the voice data of the N original speakers and the voice data of the M target speakers; Specific speaker model creating means for creating a specific speaker model corresponding to each one of the speakers and each of the target speakers;
Applying the result obtained by using the specific speaker model created by the specific speaker model creating means to the eigenvoice technique to parameters constituting the initial voice quality conversion model generated by the initial voice quality conversion model generating means, A speaker control parameter, which is a weight vector for converting any of the N original speakers to any of the M target speakers, is added to the parameters constituting the initial voice quality conversion model. A voice quality conversion model generating means for generating a voice quality conversion model having the speaker control parameter,
At least one of N and M is an integer of 2 or more.

In such a configuration, the initial voice quality conversion model generation means is the same as the first voice data, which is the voice data of the predetermined N (N is an integer of 1 or more) former speakers, and the first voice data. It is possible to perform learning using a predetermined learning algorithm using, as learning data , for example, second voice data that is voice data of predetermined M (M is an integer of 1 or more) target speakers of utterance content It is. Then, by this learning, the voice of the voice quality of the N former speakers is converted to voice of the voice quality of the M target speakers (at least one of N and M is 2 or more). It is possible to generate an initial voice quality conversion model which is a statistical model common to the former speaker and the M target speakers.
Further, the specific speaker model creating means converts the voice quality of each of the N original speakers from the voice data of the N original speakers and the voice data of the M target speakers to the voice quality of each of the M target speakers. It is possible to create a specific speaker model that has a one-to-one correspondence with each of the original speaker and each of the target speakers to be converted.
Further, the voice quality conversion model generation means is a parameter constituting the initial voice quality conversion model generated by the initial voice quality conversion model generation means based on the result obtained by using the specific speaker model generated by the specific speaker model generation means for the proper voice technology. And a speaker control parameter that is a weight vector for converting any of the voices of the N original speakers into any of the voices of the M target speakers, as a parameter constituting the initial voice quality conversion model. In addition, it is possible to generate a voice quality conversion model having a speaker control parameter.

Therefore, for example, when a voice quality conversion model is generated from voice data of N (two or more) former speakers and voice data of one target speaker, one voice quality conversion model can be used for various voice qualities. The voice of the original speaker can be converted into the voice of the voice quality of the one target speaker.
Also, for example, when a voice quality conversion model is generated from the voice data of one former speaker and the voice data of M (two or more) target speakers, one voice quality conversion model is used for one original voice. The voice of the voice quality of the speaker can be converted into the voice of the voice quality of the target speaker having various voice qualities.

  In addition, for example, when a voice quality conversion model common to these speakers is generated from voice data of N (two or more) former speakers and voice data of M (two or more) target speakers Can convert voices of original speakers of various voice qualities into voices of target speakers of various voice qualities, using one or two voice quality conversion models. Here, the voice quality conversion model for converting the voice quality of the voices of the N original speakers into the voice quality of the voices of the M target speakers may be constituted by one voice quality conversion model. First voice quality conversion model for converting a person's voice into one target speaker A, and a second voice quality conversion for converting a voice of one original speaker A into voice of M target speakers You may comprise by two models with a model. In any configuration, one voice quality conversion model corresponds to at least one of N former speakers and M target speakers.

  As a result, the amount of memory required for the voice quality conversion model can be greatly reduced, so that the effect of reducing the cost for constructing a voice quality conversion system using the voice quality conversion model can be obtained. In addition, it is possible to easily construct a voice quality conversion system in an environment where memory resources are not rich (restricted). For example, in a mobile device such as a mobile phone, it is possible to easily construct a voice quality conversion system using the original memory resources without adding an internal memory or connecting an external memory. .

Further, the voice quality conversion model is used for voice quality conversion by adapting at least one of the voice quality of the voice of the original speaker to be voice quality converted and the voice quality of the target speaker of the voice quality conversion target by a predetermined adaptation method. In some cases, the voice of the voice quality of the original speakers of various voice qualities is used rather than the voice quality conversion model generated from the voice data of one of the original speakers and the voice data of the one target speaker. In addition, since the voice quality conversion model can be appropriately adapted to at least one of the voices of the target speakers having various voice qualities, the sound quality of the voice after the voice quality conversion can be improved. In particular, the higher the number of original speakers or target speakers that provide learning data, the more voice quality conversion models that reflect various characteristics can be generated. The sound quality of the converted voice can be further improved.
In addition, since it is possible to generate a voice quality conversion model by performing adaptive learning using a known Eigenvoice technology, it is possible to change the target parameters at the time of voice quality conversion by changing each speaker control parameter of the voice quality conversion model. It is possible to easily control the voice quality of the speaker. Thereby, by controlling the speaker control parameter, it is possible to easily adapt the voice quality conversion model to the voice of the voice quality of an arbitrary target speaker. Further, by using the unique voice technology, it is possible to adapt the voice of the voice quality of the new original speaker to the voice quality conversion model with a small amount of voice data (for example, data of one utterance to data of several utterances). As a result, the time required for the adaptation process can be reduced, and for example, the adaptation process and the voice quality conversion process can be performed in real time (online).

  Here, the statistical model is a GMM (Gaussian mixture model), a neural network, or the like, for example, converting voices of voice quality of N former speakers to voices of one target speaker, The voice quality of the original speaker is converted to the voice quality of M (M is an integer of 2 or more) target speakers, or the voice quality of the N original speakers is converted to the voice quality of the M target speakers. Any conversion rule can be used as long as it can express a conversion rule to be converted into a single model (function or the like). The same applies to the following invention relating to a voice quality conversion model generation device, an invention relating to a voice quality conversion model generation program, and an invention relating to a voice quality conversion model generation method.

The learning corresponds to, for example, learning using a known EM algorithm that performs maximum likelihood parameter estimation. For example, in the case of GMM, which is a statistical model using a mixed normal distribution, the mixing weight λ, the average μ, and the variance σ ² are the model parameters. This parameter is determined using learning data. The same applies to the following invention relating to a voice quality conversion model generation device, an invention relating to a voice quality conversion model generation program, and an invention relating to a voice quality conversion model generation method.

The second voice data of predetermined M people having the same utterance content as the first voice data is an utterance sentence set a, b, c (for example, 50 sentences for each set) composed of different sentences. When the voice data is included in the first voice data, for example, if the predetermined M target speakers are three persons A, B, and C (M = 3), the second voice data includes the target talk. It is necessary to include speech data of speech sentence sets a, b, and c uttered by the persons A, B, and C. For example, the utterance sentence set a, b, c for each speaker of the target speakers A, B, and C may be included, or the utterance sentence set a of the target speaker A and the utterance sentence of the target speaker B If all the same utterance contents (utterance sentence sets a, b, c) of the original speaker are included, such as the utterance sentence set c of the set b and the target speaker C, the target speakers A, B, The contents of C's speech sentence set may be anything. However, the voice data (any one of a, b, and c) of the three target speakers must be included (except for combinations where no one speaks). The same applies to the following invention relating to a voice quality conversion model generation device, an invention relating to a voice quality conversion model generation program, and an invention relating to a voice quality conversion model generation method.
The eigenvoice technique is a speaker adaptation technique using an eigenvoice space (also called eigenspace) composed of eigenvectors of a large number of speakers (preferably 100 to 200 speakers). In order to generate the eigenvoice space, for example, a specific model (specific speaker model) for each speaker is first generated by learning from speech data (learning data) of M (two or more) speakers, Based on the generated specific speaker model, a super vector is generated for each speaker. The specific speaker model is composed of an HMM or the like, and the super vector is at least a part (eg, a mixture component, a mixture coefficient, a Gaussian density, a mean vector, a covariance matrix, etc.) constituting the HMM model (eg, It is composed of a list in which parameters (average of parameters related to Gaussian distribution) are arranged in a predetermined order (an order common to all supervectors). Once the supervector is formed, the dimension of the supervector is then reduced. Specifically, a high-dimensional supervector is mapped to a lower-dimensional supervector. There are various methods such as principal component analysis method, linear discriminant analysis method, element analysis method, singularity decomposition method, and the like as methods for performing this dimension reduction. Then, eigenvectors (super-vectors after dimension reduction) for each speaker are formed by dimension reduction using any one of the above methods. A space formed from these eigenvectors becomes an eigenvoice space. For more details, see R. Kuhn, et al. “Rapid speaker adaptation in eigenvoice space.” IEEE Trans. Speech and Audio Processing, Vol. 8, No. 6, pp. 695-707, 2000. Are listed. The same applies to the following invention relating to a voice quality conversion model generation device, an invention relating to a voice quality conversion model generation program, and an invention relating to a voice quality conversion model generation method.

Furthermore, the invention according to claim 2 is the voice quality conversion model generation device according to claim 1,
M is an integer of 2 or more,
Adaptive learning means for performing adaptive learning using the first voice data of the N former speakers and the second voice data of the M target speakers;
The adaptive learning means converts an initial voice quality conversion model for converting a voice of a predetermined original speaker voice quality into a voice of a predetermined target speaker voice quality, voices of the voice quality of the N original speakers, and the M people. Voice quality conversion having a speaker control parameter capable of controlling voice quality by performing adaptive learning based on eigenvoice technology to adapt to voices of voice quality of at least the M target speakers among voices of target voice quality of It is characterized by generating a model.

  With such a configuration, an initial voice quality conversion model in which voice of a predetermined original speaker's voice quality is converted into voice of a predetermined target speaker's voice quality by the adaptive learning means is used as the voice quality of the N former speakers. , And voices of voice quality of the M target speakers, a speaker capable of controlling voice quality by performing adaptive learning based on eigenvoice technology to adapt to voices of voice quality of at least the M target speakers It is possible to generate a voice quality conversion model having sex control parameters.

  Therefore, it is possible to generate a voice quality conversion model by performing adaptive learning using a known Eigenvoice technique, so that the target parameters for voice quality conversion can be changed by changing each speaker control parameter of the voice quality conversion model. It is possible to easily control the voice quality of the speaker. Thereby, by controlling the speaker control parameter, it is possible to easily adapt the voice quality conversion model to the voice of the voice quality of an arbitrary target speaker. Further, by using the unique voice technology, it is possible to adapt the voice of the voice quality of the new original speaker to the voice quality conversion model with a small amount of voice data (for example, data of one utterance to data of several utterances). As a result, the time required for the adaptation process can be reduced, and for example, the adaptation process and the voice quality conversion process can be performed in real time (online).

  Here, the eigenvoice technique is a speaker adaptation technique using an eigenvoice space (also called eigenspace) composed of eigenvectors of a large number of speakers (preferably 100 to 200 speakers). . In order to generate the eigenvoice space, for example, a specific model (specific speaker model) for each speaker is first generated by learning from speech data (learning data) of M (two or more) speakers, Based on the generated specific speaker model, a super vector is generated for each speaker. The specific speaker model is composed of an HMM or the like, and the super vector is at least a part (eg, a mixture component, a mixture coefficient, a Gaussian density, a mean vector, a covariance matrix, etc.) constituting the HMM model (eg, It is composed of a list in which parameters (average of parameters related to Gaussian distribution) are arranged in a predetermined order (an order common to all supervectors). Once the supervector is formed, the dimension of the supervector is then reduced. Specifically, a high-dimensional supervector is mapped to a lower-dimensional supervector. There are various methods such as principal component analysis method, linear discriminant analysis method, element analysis method, singularity decomposition method, and the like as methods for performing this dimension reduction. Then, eigenvectors (super-vectors after dimension reduction) for each speaker are formed by dimension reduction using any one of the above methods. A space formed from these eigenvectors becomes an eigenvoice space. For more details, see R. Kuhn, et al. “Rapid speaker adaptation in eigenvoice space.” IEEE Trans. Speech and Audio Processing, Vol. 8, No. 6, pp. 695-707, 2000. Are listed. The same applies to the following invention relating to a voice quality conversion model generation device, an invention relating to a voice quality conversion model generation program, and an invention relating to a voice quality conversion model generation method.

On the other hand, in order to achieve the above object, a voice quality conversion model generation device according to claim 2 is provided:
A voice quality conversion model generation device for generating a statistical model for voice quality conversion,
First voice data that is voice data of a predetermined N (N is an integer of 2 or more) former speakers, and voice data of a predetermined intermediate speaker having the same utterance content as the first voice data Learning with the third voice data as learning data, and converting the voice of the voice quality of the N former speakers into the voice of the one intermediate speaker and the 1 former speaker and the 1 A first voice quality conversion model, which is one statistical model common to a human intermediate speaker, is generated, and a third voice data of the one intermediate speaker and a predetermined utterance content same as that of the third voice data are generated. Learning is performed by using the second voice data, which is voice data of M (M is an integer of 2 or more) target speakers, as learning data, and the voice of the one intermediate speaker is used as the target speaker's voice. One-to-one for each of the one intermediate speaker and the target speaker for conversion to voice-quality speech Is a response to the statistical model, characterized in that it comprises a voice conversion model generating means for generating M second voice conversion model.

  With such a configuration, the voice quality conversion model generation means learns the first voice data of the predetermined N former speakers and the third voice data of the predetermined one intermediate speaker as learning data. One statistic common to the N former speakers and the one intermediate speaker to convert the voice quality of the N former speakers to the voice of the one intermediate speaker It is possible to generate a first voice quality conversion model that is a model.

  Further, the voice quality conversion model generation means learns the third voice data of the one intermediate speaker and the second voice data of the M target speakers as learning data, and the one intermediate talk data is learned. M models which are one-to-one statistical models corresponding to the one intermediate speaker and each of the target speakers for converting the voice of the speaker into voices of the voice quality of the M target speakers A second voice quality conversion model can be generated.

  In the conventional method of generating voice quality conversion models for converting voices of voice quality of N original speakers into voices of voice quality of M target speakers, N × M voice quality conversion models are separately generated for each speaker. Must be generated. On the other hand, the above-described configuration of the present invention is configured to generate one first voice quality conversion model for converting voices of voice quality of N former speakers through a middle speaker into voices of one middle speaker. , One first voice quality conversion model, and M second voice quality conversion models for each speaker for converting the voice of one intermediate speaker into the voice of M target speakers, (M + 1) models in total. Therefore, an effect that the memory capacity necessary for storing the voice quality conversion model can be greatly reduced as compared with the conventional case.

On the other hand, in order to achieve the above object, in the voice quality conversion system according to claim 3 ,
A voice quality conversion system for converting the voice of any of the original speaker to the voice of the other voice,
Voice quality conversion model storage means for storing a voice quality conversion model having a speaker control parameter generated by the voice quality conversion model generation device according to claim 1 ;
Former speaker voice data acquisition means for acquiring voice data of any former speaker;
Target speaker voice data acquisition means for acquiring voice data of an arbitrary target speaker;
A speaker control parameter value specifying means for specifying a speaker control parameter value related to the voice of the voice quality of the M target speakers in the voice conversion model ;
Based on the audio data obtained in the previous Kimoto speaker speech data acquisition means, and the parameter values specified in the speaker information control parameter value specification unit, a voice conversion model stored in the voice conversion model storing means, Second adaptation means for creating a new voice quality conversion model by adapting the voice quality conversion model to the voice of the voice quality of the arbitrary original speaker and the specified parameter value using a predetermined adaptation method;
Based on the new voice quality conversion model created by the second adapting means and the voice data acquired by the original speaker voice data acquiring means, the voice data of the arbitrary former speaker is converted into voice data of another voice quality. characterized in that it comprises a voice conversion means for, the.

With such a configuration, when the original speaker voice data acquisition unit acquires voice data used for the adaptation process of an arbitrary original speaker, and the control parameter value is specified by the speaker control parameter value specification unit, The second adaptation means uses the voice quality conversion model stored in the voice quality conversion model storage means based on the voice data and the speaker control parameter value and the voice quality conversion model using the predetermined adaptation technique. The voice of the original speaker's voice quality and the specified parameter value are adapted. Then, when the original speaker voice data acquisition means acquires voice data of an arbitrary utterance content that is a voice quality conversion target of an arbitrary original speaker, the voice quality conversion means uses the new voice quality conversion model adapted by the second adaptation means. Is used to convert the voice data for voice quality conversion acquired by the voice data acquisition means into voice data of an arbitrary target speaker voice quality (other voice quality) given by the speaker control parameter value.

Thus, the voice quality conversion model generated by the voice quality conversion model generating apparatus according to claim 1 is converted to a voice of another voice quality by controlling the value of the speaker quality control parameter by the speaker quality control parameter value specifying means. The effect that it can adapt easily is acquired.

Also, with this configuration, when the target speaker's voice is not available , conversion to an arbitrary target speaker using the speaker control parameter is possible. When the speaker control parameter is manipulated, it can be converted into a voice of a non-existent speaker.

On the other hand, in order to achieve the above object, the voice quality conversion system according to claim 4 is:
A voice quality conversion system that converts a voice of an arbitrary utterance content of an arbitrary former speaker into a voice of a predetermined target voice quality of M (M is an integer of 2 or more),
A first voice of the voice quality of a predetermined N (N is an integer of 2 or more) original speakers, which is generated by the voice quality conversion model generation device according to claim 2 , is converted into a voice of one intermediate speaker. Voice quality conversion model and voice quality conversion model storage means for storing a second voice quality conversion model for converting the voice of the one intermediate speaker into the voice of the voice quality of the M target speakers;
Former speaker voice data acquisition means for acquiring voice data of the arbitrary former speaker;
Based on the voice data acquired by the former speaker voice data acquisition means and the first voice quality conversion model stored in the voice quality conversion model storage means, the first voice quality conversion model is converted into the arbitrary voice using a predetermined adaptive method. Adaptation means to adapt to the voice quality of the former speaker,
Using the adapted first voice quality conversion model, the voice data of the arbitrary utterance content of the arbitrary original speaker acquired by the original speaker voice data acquisition means is converted into the voice data of the voice quality of the intermediate speaker And using the second voice quality conversion model stored in the voice quality conversion model storage means, the voice quality conversion means for converting the voice data converted into the voice quality of the intermediate speaker into the voice data of the arbitrary target speaker. And.

  With such a configuration, when the original speaker voice data acquisition unit acquires voice data used for the adaptation processing of an arbitrary original speaker, the adaptation unit is stored in the voice data and the voice quality conversion model storage unit. Based on the first voice quality conversion model, the first voice quality conversion model is adapted to the voice of the voice quality of the arbitrary former speaker using a predetermined adaptation method. Then, when the original speaker voice data acquisition means acquires voice data of an arbitrary utterance content that is a voice quality conversion target of any original speaker (the voice data used for adaptation may be a voice quality conversion target), the voice quality conversion The means converts the voice data of the voice quality conversion target acquired by the original speaker voice data acquisition means into voice data of the voice quality of the one intermediate speaker using the adapted first voice quality conversion model. . Further, the voice quality conversion means converts the voice data of the voice quality of the intermediate speaker into voice data of the target speaker using the second voice quality conversion model.

Thus, the same operation and effect as the above-described operation and effect of the voice quality conversion model generation device according to claim 2 can be obtained.
Furthermore, the invention according to claim 5 is the voice quality conversion system according to claim 4 ,
The adapting means converts the first voice conversion model into the voice of the voice quality of any of the original speakers with respect to the parameters related to the voice quality of the N original speakers constituting the first voice conversion model. An adaptive parameter value to be adapted is estimated using the predetermined adaptation method, and a parameter value related to the voice of the voice quality of the N former speakers of the first voice quality conversion model is converted to the estimated adaptive parameter value. It is characterized by that.

If it is such a structure, an adaptation means will use the predetermined | prescribed adaptation method, the value of the parameter which concerns on the voice of the voice quality of N former speakers who comprise the 1st voice quality conversion model will be set as arbitrary former speakers. Can be converted to a value suitable for converting the voice of the above voice quality into the voice of the one intermediate speaker. Thus, the same operation and effect as the above-described operation and effect of the voice quality conversion model generation device according to claim 2 can be obtained.

On the other hand, in order to achieve the above object, a voice quality conversion model generation program according to claim 6 is provided:
A voice quality conversion model generation program for generating a statistical model for voice quality conversion,
First voice data which is voice data of a predetermined N (N is an integer of 1 or more) people, and predetermined M (M is an integer of 1 or more) of the same utterance content as the first voice data initial voice conversion model generation for generating an initial voice conversion model is a common statistical model based on the speaker and the M's target speaker of the N people using the second audio data is audio data of the target speaker and the step,
Converting the voice quality of each of the N original speakers into the voice quality of each of the M target speakers from the voice data of the N original speakers and the voice data of the M target speakers; A specific speaker model creating step of creating a specific speaker model corresponding to each one of the speakers and each of the target speakers;
Applying the result obtained by using the specific speaker model created in the specific speaker model creation step for eigenvoice technology to the parameters constituting the initial voice quality conversion model generated in the initial voice quality conversion model generation step, A speaker control parameter, which is a weight vector for converting any of the N original speakers to any of the M target speakers, is added to the parameters constituting the initial voice quality conversion model. it is, includes a program for executing the voice conversion model generating step of generating a voice conversion model having the speaker information control parameters to the computer,
At least one of N and M is an integer of 2 or more.

With such a configuration, when the program is read by the computer and the computer executes processing according to the read program, the same operation and effect as those of the voice quality conversion model generation device according to claim 1 can be obtained.
In order to achieve the above object, a voice quality conversion model generation method according to claim 7 comprises:
A voice conversion model generation method for generating a statistical model for voice conversion,
First voice data which is voice data of a predetermined N (N is an integer of 1 or more) people, and predetermined M (M is an integer of 1 or more) of the same utterance content as the first voice data initial voice conversion model generation for generating an initial voice conversion model is a common statistical model based on the speaker and the M's target speaker of the N people using the second audio data is audio data of the target speaker and the step,
Converting the voice quality of each of the N original speakers into the voice quality of each of the M target speakers from the voice data of the N original speakers and the voice data of the M target speakers; A specific speaker model creating step of creating a specific speaker model corresponding to each one of the speakers and each of the target speakers;
Applying the result obtained by using the specific speaker model created in the specific speaker model creation step for eigenvoice technology to the parameters constituting the initial voice quality conversion model generated in the initial voice quality conversion model generation step, A speaker control parameter, which is a weight vector for converting any of the N original speakers to any of the M target speakers, is added to the parameters constituting the initial voice quality conversion model. it is, and a voice conversion model generating step of generating a voice conversion model having the speaker information control parameter,
At least one of N and M is an integer of 2 or more.
Thus, the same operation and effect as the voice quality conversion model generation device according to claim 1 can be obtained.

On the other hand, in order to achieve the above object, the voice quality conversion server client system according to claim 8 comprises:
And the client computer and the server computer are connected via a network, the voice-conversion-client-server system that converts speech of any source speaker to the speech of the other voice,
The server computer
And voice conversion model storage means for storing the voice quality conversion model generated by the voice conversion model generating device according to claim 1,
Voice quality conversion model transmission means for transmitting the voice quality conversion model stored in the voice quality conversion model storage means to the client computer;
The client computer is
Former speaker voice data acquisition means for acquiring voice data of the arbitrary former speaker;
A speaker control parameter value specifying means for specifying a parameter value of the speaker control parameter in the voice conversion model;
And voice conversion model receiving means for receiving the voice conversion model from the previous SL server computer,
Based on the voice data acquired by the former speaker voice data acquiring means, the parameter value specified by the speaker control parameter value specifying means, and the voice quality conversion model received by the voice quality conversion model receiving means, a predetermined Adapting means for adapting the voice quality conversion model of the arbitrary speaker's voice quality and voice to the designated parameter value by using an adaptation technique to create a new voice quality conversion model ;
Based on the new voice quality conversion model created by the adapting means and the voice data acquired by the original speaker voice data acquiring means, the voice quality conversion for converting the voice data of the arbitrary former speaker into voice data of another voice quality And means.

  With such a configuration, the server computer uses the voice quality conversion model storage means to generate a predetermined N (N is an integer of 2 or more) narratives generated by the voice quality conversion model generation device according to claim 1. It is possible to store a voice quality conversion model for converting a voice of a person's voice quality into a voice of the voice quality of a target speaker of a predetermined M (M is an integer equal to or greater than 2). The voice quality conversion model stored in the conversion model storage means can be transmitted to the client computer.

Further, the client computer, by Motohanashi's voice data acquiring means can acquire speech data of the arbitrary source speaker by voice quality conversion model receiving unit, wherein the voice conversion model from the server computer Can be received, and the voice data acquisition means can acquire voice data used for adaptive processing of an arbitrary speaker, and the control parameter value can be specified by the speaker control parameter value specification means. It is possible, and the voice quality conversion model is arbitrarily determined by the adaptation means using a predetermined adaptation method based on the voice data and the speaker control parameter value and the voice quality conversion model stored in the voice quality conversion model storage means . The voice quality of the original speaker and the voice quality of the desired target speaker can be adapted.

Further, the client computer uses the adapted voice quality conversion model by the voice quality conversion means to convert the voice data of any utterance content of the arbitrary original speaker acquired by the original speaker voice data acquisition means to the arbitrary Can be converted into voice data of the target speaker.
Thus, the same operation and effect as the voice quality conversion system of claim 3 can be obtained.
In addition, since the server computer holds relatively large data such as a voice quality conversion model, the client computer on the user side needs a memory for the voice quality conversion model temporarily only during voice quality conversion. However, there is an advantage that it is not necessary to keep the voice conversion model. Furthermore, since the voice quality conversion model can be managed on the server computer side, the voice quality conversion model can be easily upgraded, and the upgraded voice quality conversion model can be easily provided to the user. Can also be obtained.

Here, the “client computer” corresponds to a desktop computer such as a PC or WS, a portable computer such as a notebook PC, a PDA or a mobile phone, or a computer game machine such as a portable game machine. The same applies to the following client computers .

As described above, according to the voice quality conversion model generation device according to claims 1 and 2 , the voice quality conversion model generation program according to claim 6, and the voice quality conversion model generation method according to claim 7 , according to the present invention. For example, the amount of memory required for the voice quality conversion model can be greatly reduced, so that the cost required for constructing a voice quality conversion system using the voice quality conversion model can be reduced. In addition, it is possible to easily construct a voice quality conversion system in an environment where memory resources are not rich (restricted).

Further, according to the voice conversion model generation apparatus according to claim 1, wherein according to the present invention, it becomes possible to generate a voice conversion model by performing an adaptive learning using known eigenvoice (Eigenvoice) technology, story By controlling the humanity control parameters, it is possible to easily adapt the voice quality conversion model to the voice of the voice quality of an arbitrary target speaker.

According to the voice quality conversion system of any one of claims 4 and 5 according to the present invention, the voice quality conversion model generated by the voice quality conversion model generation device according to claim 2 is converted into a voice quality by a predetermined adaptive method. source-speaker voice quality of the voice of the target, and of the target speaker in the voice quality of the voice of the voice conversion target. Thus use in voice conversion adapt at least one, in addition to the effect of the second aspect, 1 Rather than adaptively using a voice quality conversion model generated from the voice data of a person's original speaker and the voice data of one target speaker, the voice of the voice quality of the original speaker of various voice qualities and various voice qualities Since the voice quality conversion model can be appropriately adapted to at least one of the target speaker's voices, the sound quality of the voice after voice quality conversion can be improved.

According to the voice quality conversion system according to claim 3 of the present invention, the voice quality conversion model using the proper voice technology generated by the voice quality conversion model generation device according to claim 1 is used for voice quality conversion. In addition to the effect of the first aspect , since the time required for the adaptive processing can be reduced by using the eigenvoice technique, for example, it is possible to perform the adaptive processing and voice quality conversion processing in real time (online). It is done.

Further, according to the voice quality conversion client-server system according to claim 8 of the present invention, in addition to the effect of the voice quality conversion system according to claim 3 , it is possible to easily upgrade the voice quality conversion model. At the same time, it is possible to easily provide the user with the upgraded voice quality conversion model.

[First Embodiment]
Hereinafter, embodiments of the present invention will be described with reference to the drawings. 1 to 5 are diagrams showing a first embodiment of a voice quality conversion model generation device, a voice quality conversion model generation program, and a voice quality conversion model generation method according to the present invention.
First, the configuration of the voice quality conversion model generation device according to the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing the configuration of a voice quality conversion model generation device 1 according to the first embodiment of the present invention.

  The voice quality conversion model generation device 1 stores the voice data of an original speaker that is a voice quality conversion source speaker and voice data of a target speaker that is a voice quality conversion target speaker. A target speaker voice data storage unit 11 for storing, a voice quality conversion model generation unit 12 for generating a voice quality conversion model for converting the voice quality of the voice quality of the original speaker to that of the target speaker, and a voice quality conversion model The voice quality conversion model storage unit 13 is stored.

  The former speaker voice data storage unit 10 stores voice data (hereinafter referred to as first voice data) of utterances spoken by a large number (for example, 200 or more) former speakers for each former speaker. It has a function. Here, the first voice data is obtained by having each of the above-mentioned many former speakers utter a sentence of the utterance sentence set a to c (consisting of 50 sentences in each set), and collecting it with a microphone or the like. This is voice waveform data that has been subjected to signal processing by being input to an information processing device such as a PC (personal computer) via a device. The first voice data may be data composed of a parameter series (feature parameter series) of feature parameters (for example, cepstrum coefficient (CC), mel cepstrum coefficient (MFCC), etc.) extracted from the voice waveform data. good.

  The target speaker voice data storage unit 11 stores voice data (hereinafter, referred to as second voice data) of utterances spoken by a large number (for example, 200 or more) target speakers for each target speaker. It has a function. Here, the second voice data includes all of the utterance sentence sets a to c uttered by the large number of former speakers, and each target speaker has the utterance sentence sets a to c (each set has a different content). Sentence of a sentence composed of 50 sentences (sentences differ between sets) is input to an information processing device such as a PC via a sound collection device such as a microphone for signal processing. Voice waveform data. Further, the second sound data may be stored as a series of feature parameters previously extracted from the sound waveform data.

  The voice quality conversion model generation unit 12 stores the first voice data of N people (N ≧ 2) or one former speaker stored in the former speaker voice data storage unit 10 and the target speaker voice data storage unit 11. Using the stored second voice data of M people (M ≧ 2) or one target speaker as learning data, the voice quality of the voice quality of N people or one former speaker is set as one or M goals. It has a function of learning a conversion rule (statistical model) so as to convert the voice into the voice quality of the speaker and generating one voice quality conversion model common to all of the original speaker and the target speaker. Specifically, the types of voice quality conversion models to be generated are broadly classified as follows: (1) a voice quality conversion model (hereinafter referred to as “voice quality conversion model”) that converts the voice quality of N former speakers to the voice quality of one target speaker. (Referred to as N to 1 voice quality conversion model), (2) voice quality conversion model (hereinafter referred to as 1 to M voice quality conversion model) that converts the voice quality of one former speaker to the voice quality of M target speakers. 3) voice quality conversion models (hereinafter referred to as N vs. M voice quality conversion models) that convert the voice quality of N former speakers to the voice quality of M target speakers. .

  As another form of the N-to-M voice quality conversion model, the voice quality conversion model generation unit 12 (4) a voice quality conversion model that converts voices of the voice quality of N former speakers into voices of one intermediate speaker ( Hereinafter, it is referred to as an N-to-1 intermediate voice quality conversion model) and M corresponding to each of the M target speakers that convert the voice of one intermediate speaker into the voice of the voice quality of M target speakers. It is also possible to generate (1 + M) voice quality conversion models (hereinafter referred to as N to M intermediate voice quality conversion models) composed of the following voice quality conversion models (hereinafter referred to as 1 to M intermediate voice quality conversion models). is there. In this case, it is necessary to acquire voice data (hereinafter referred to as third voice data) of a speaker who is an intermediate speaker from the original speaker voice data storage unit 10 or the target speaker voice data storage unit 11. Further, for example, in an external speech synthesizer (TTS), synthesized speech data of an intermediate speaker may be generated and acquired from the text sentences in the utterance sentence sets a to c.

Further, the voice quality conversion model generation unit 12 has two learning modes, a normal learning mode and an adaptive learning mode, and selects a mode set as a learning condition from these modes, and learns the selected mode. Learning with the contents is performed to generate the various voice quality conversion models (1) to (4) described above.
The voice quality conversion model storage unit 13 stores the voice quality conversion models (1) to (4) generated by the voice quality conversion model generation unit 12, or stores an initial voice quality conversion model used in adaptive learning described later. It has a function.

Further, a detailed configuration of the voice quality conversion model generation unit 12 will be described with reference to FIG. Here, FIG. 2 is a block diagram showing a detailed configuration of the voice quality conversion model generation unit 12.
As shown in FIG. 2, the voice quality conversion model generation unit 12 includes a feature amount extraction unit 12a, a normal learning unit 12b, a specific speaker model generation unit 12c, and an adaptive learning unit 12d. .
The feature amount extraction unit 12a performs four or more dimensions by analyzing processes such as cepstrum analysis and linear prediction analysis from the first to third voice data acquired from the original speaker voice data storage unit 10 and the target speaker voice data storage unit 11. It has a function of extracting (for example, 40 dimensions) feature quantities (also referred to as feature parameters).

  The normal learning unit 12b operates when the normal learning mode is set as a learning condition, and the feature amount of the first to third speech data extracted by the feature amount extraction unit 12a is used as learning data to determine the original speaker's It has a function of learning a conversion rule (statistical model) so as to convert a voice of voice quality into voice of a target speaker and generating a voice quality conversion model having any one of the above configurations (1) to (4) ing. In the present embodiment, specifically, learning (estimation) using a known EM algorithm is performed, and each parameter of GMM (Gaussian mixture model) is determined. Generate.

  The specific speaker model generation unit 12c operates when the adaptive learning mode is set as a learning condition, and uses the feature amounts of the first to third speech data extracted by the feature amount extraction unit 12a as learning data. A specific speaker model in which each of the original speakers and each of the target speakers has a one-to-one correspondence by learning conversion rules so as to convert each speaker's speech into each target speaker's speech. It has the function to generate.

  The adaptive learning unit 12d operates when the adaptive learning mode is set as a learning condition, acquires an initial voice quality conversion model from the voice quality conversion model storage unit 13, and uses the initial voice quality conversion model by using a predetermined adaptation method. And has a function of adapting to the specific speaker model generated by the specific speaker model generation unit 12c. In the present embodiment, the parameters of the initial voice quality conversion model are determined from the specific speaker model using either a proper adaptation technique, eigenvoice technology or speaker normalized training (SAT). Based on the estimation result, a voice quality conversion model having any one of the configurations (1) to (4) is generated. In the present embodiment, it is possible to set a repeated learning mode as a learning condition. When the iterative learning mode is set, the adaptive learning unit 12d repeatedly performs adaptive learning on the initial voice quality conversion model until the predetermined condition is satisfied, and performs the learning after each learning as the next model. It is used as an initial voice quality conversion model in adaptive learning.

  In this embodiment, the voice quality conversion model generation device 1 includes a processor (not shown), a RAM (Random Access Memory), and a ROM (Read Only Memory) in which a dedicated program is stored. The functions of the above-described units are achieved by executing a dedicated program. In addition, the above-described units include those that perform their functions only with dedicated programs, and those that perform their functions by controlling hardware using dedicated programs.

Furthermore, the flow of voice quality conversion model generation processing in the voice quality conversion model generation apparatus 1 will be described with reference to FIG. Here, FIG. 3 is a flowchart showing voice quality conversion model generation processing in the voice quality conversion model generation apparatus 1.
As shown in FIG. 3, the voice quality conversion model generation process first proceeds to step S100, and whether or not the voice quality conversion model generation unit 12 has received a voice quality conversion model generation instruction from the user via an operation unit (not shown). If it is determined that there is a generation instruction (Yes), the process proceeds to step S102. If not (No), the determination process is repeated until there is a generation instruction.

  Here, the generation instruction includes information set as learning conditions (hereinafter referred to as setting information). The setting information includes the configuration of the voice quality conversion model (N to 1, 1 to M, N to M, intermediate speaker), information of the original speaker, the target speaker and the intermediate speaker (for example, the original speaker and the target story). Information such as a learning number (normal learning mode, adaptive learning mode (on / off of repeated learning mode, predetermined condition (repeated condition)))) and the like are included.

When the process proceeds to step S102, the voice quality conversion model generation unit 12 determines whether or not the configuration of the voice quality conversion model is N-to-1 (original speaker vs. target speaker) based on the setting information. If it is determined that there is (Yes), the process proceeds to step S104. If not (No), the process proceeds to step S114.
When the process proceeds to step S104, the voice quality conversion model generation unit 12 acquires the first voice data of the designated N former speakers from the former speaker voice data storage unit 10 based on the setting information, and is designated. The second voice data of one target speaker is acquired from the target speaker voice data storage unit 11, and the process proceeds to step S106.

In step S106, the voice quality conversion model generation unit 12 determines whether or not the normal learning mode is set based on the setting information. If it is determined that the normal learning mode is set (Yes), the process proceeds to step S108. If not (No), the process proceeds to step S112.
When the process moves to step S108, the voice quality conversion model generation unit 12 executes a voice quality conversion model generation process by normal learning, generates a voice quality conversion model corresponding to the model configuration of the setting information, and moves to step S110.

In step S110, the voice quality conversion model generation unit 12 stores the voice quality conversion model generated in step S108 or step S112 in the voice quality conversion model storage unit 13 and ends the process.
On the other hand, when the learning mode is not the normal learning mode and the process proceeds to step S112 in step S106, the voice quality conversion model generation unit 12 executes a voice quality conversion model generation process by adaptive learning and corresponds to the model configuration of the setting information. The generated voice quality conversion model is generated, and the process proceeds to step S110.

In step S102, if the configuration of the voice quality conversion model is not N-to-1 and the process proceeds to step S114, the voice quality conversion model generation unit 12 sets the voice quality conversion model to 1-M (original speaker vs. target speaker). ). If it is determined that the pair is M (Yes), the process proceeds to step S116, and if not (No), the process proceeds to step S118.
When the process proceeds to step S116, the voice quality conversion model generation unit 12 acquires the first voice data of the designated one former speaker from the former speaker voice data storage unit 10 based on the setting information, and is designated. The second voice data of the M target speakers is acquired from the target speaker voice data storage unit 11, and the process proceeds to step S106.

When the process proceeds to step S118, the voice quality conversion model generation unit 12 determines whether or not the configuration of the voice quality conversion model is based on the setting information and applies an intermediate speaker. If determined (Yes), the process proceeds to step S120, and if not (No), the process proceeds to step S122.
When the process proceeds to step S120, the voice quality conversion model generation unit 12 acquires the first voice data of the designated N former speakers from the former speaker voice data storage unit 10 based on the setting information, and is designated. The third voice data of one intermediate speaker is acquired from the original speaker voice data storage unit 10 or the target speaker voice data storage unit 11, and the second voice data of the designated M target speakers is obtained. Obtained from the target speaker voice data storage unit 11, the process proceeds to step S106.

When the process proceeds to step S122, the voice quality conversion model generation unit 12 acquires the first voice data of the designated N former speakers from the former speaker voice data storage unit 10 based on the setting information. The second voice data of the designated M target speakers is acquired from the target speaker voice data storage unit 11, and the process proceeds to step S106.
Furthermore, the flow of voice quality conversion model generation processing by normal learning in the voice quality conversion model generation unit 12 will be described with reference to FIG. Here, FIG. 4 is a flowchart showing voice quality conversion model generation processing by normal learning in the voice quality conversion model generation unit 12.

The voice quality conversion model generation process by normal learning is executed in step S108, and as shown in FIG. 4, first, the process proceeds to step S200, and the feature amount extraction unit 12a performs the original speaker voice data storage unit 10 and the target speaker. The feature amount is extracted from the first to third sound data acquired from the sound data storage unit 11, and the process proceeds to step S202.
In step S202, the normal learning unit 12b determines whether or not the voice quality conversion model setting configuration is N-to-1 and if it is determined to be N-to-1 (Yes), the process proceeds to step S204. When that is not right (No), it transfers to step S208.

In step S204, the normal learning unit 12b uses the feature values of the N former speakers and the feature amount of one target speaker extracted in step S200 as learning data, and the voice quality of the N former speakers. The conversion rule (statistical model) is learned so as to convert the voice of the voice into the voice quality of one target speaker, and the process proceeds to step S206.
In step S206, based on the learning result of step S204, the normal learning unit 12b converts the voice quality of N original speakers into the voice quality of one target speaker. One N-to-1 voice quality conversion model common to one target speaker is generated, and a series of processes is terminated and the process returns to the original process.

On the other hand, when the setting configuration of the voice quality conversion model is not N-to-1 in step S202, the normal learning unit 12b determines whether or not the setting configuration of the voice quality conversion model is 1-to-M. If it is determined to be 1-to-M (Yes), the process proceeds to step S210. If not (No), the process proceeds to step S214.
When the process proceeds to step S210, the normal learning unit 12b uses the feature value of one original speaker and the feature values of M target speakers extracted in step S200 as learning data. The conversion rule (statistical model) is learned so as to convert the voice of the voice quality of the speaker into the voice quality of the M target speakers, and the process proceeds to step S212.

In step S212, based on the learning result in step S210, the normal learning unit 12b converts the voice quality of one original speaker into the voice quality of M target speakers, One 1-to-M voice quality conversion model common to the M target speakers is generated, a series of processing is terminated, and the original processing is restored.
On the other hand, when the setting structure of the voice quality conversion model is not 1 to M in Step S208 and the process moves to Step S214, whether or not the setting structure of the voice quality conversion model is a structure that applies the intermediate speaker in the normal learning unit 12b. If it is determined that there is a configuration to apply the intermediate speaker (Yes), the process proceeds to step S216. If not (No), the process proceeds to step S224.

When the process proceeds to step S216, the normal learning unit 12b uses the feature values of the N original speakers and the feature value of one intermediate speaker extracted in step S200 as learning data. The conversion rule (statistical model) is learned so as to convert the voice of the voice quality of the speaker into the voice of one intermediate speaker, and the process proceeds to step S218.
In step S218, the normal learning unit 12b converts the voice of the voice quality of the N original speakers into the voice of one intermediate speaker based on the learning result of step S216, and the N original speakers and one person One N-to-1 intermediate voice quality conversion model common to the intermediate speakers is generated, and the process proceeds to step S220.

In step S220, the normal learning unit 12b uses the feature value of one intermediate speaker and the feature values of M target speakers extracted in step S200 as learning data, and the speech of one intermediate speaker. Then, the conversion rule (statistical model) is learned so as to be converted into the speech of each of the M target speakers, and the process proceeds to step S222.
In step S222, the normal learning unit 12b converts the speech of one intermediate speaker into the speech of each of the M target speakers based on the learning result of step S220. M corresponding one-to-M intermediate voice quality conversion models are generated, a series of processing is terminated, and the original processing is restored.

  On the other hand, when the setting configuration of the voice quality conversion model shifts to step S224 instead of the configuration in which the intermediate speaker is applied in step S214, the characteristics of the N original speakers extracted in step S200 in the normal learning unit 12b. Conversion rule (statistical model) so as to convert voices of voice quality of N original speakers into voices of voice quality of M target speakers, using learning amount and feature quantity of M target speakers as learning data Is transferred to step S226.

  In step S226, the normal learning unit 12b converts the speech quality of N original speakers into the speech quality of M target speakers based on the learning result of step S214. An N-to-M voice quality conversion model common to the M target speakers is generated, a series of processing is terminated, and the original processing is restored. Here, the N-to-M voice quality conversion model may be configured by one common voice quality conversion model, or may be configured by two combinations of the N-to-1 voice quality conversion model and the 1-to-M voice quality conversion model. good. In the case of the two combinations, an N to 1 target speaker and a 1 to M former speaker are set as a common speaker.

Furthermore, the flow of voice quality conversion model generation processing by adaptive learning in the voice quality conversion model generation unit 12 will be described with reference to FIG. Here, FIG. 5 is a flowchart showing voice quality conversion model generation processing by adaptive learning in the voice quality conversion model generation unit 12.
The voice quality conversion model generation process by adaptive learning is executed in step S112. As shown in FIG. 5, first, the process proceeds to step S300, and the specific speaker model generation unit 12c determines the voice quality conversion model specified in the setting information. Based on the configuration, one initial voice quality conversion model corresponding to the setting configuration is acquired from the voice quality conversion model storage unit 13, and the process proceeds to step S302. Specifically, if the setting configuration is N to 1, an N to 1 voice quality conversion model is acquired as an initial voice quality conversion model. To do. Note that the initial voice quality conversion model includes a statistical model such as a neural network or GMM.

In step S302, the feature quantity extraction unit 12a extracts feature quantities from the first to third voice data acquired from the original speaker voice data storage unit 10 and the target speaker voice data storage unit 11, and the process proceeds to step S304. .
In step S304, the specific speaker model generation unit 12c determines whether or not the configuration of the voice quality conversion model is N-to-1 based on the setting information, and if it is determined to be N-to-1 (Yes). The process proceeds to step S306, and if not (No), the process proceeds to step S322.

  In step S306, the specific speaker model generation unit 12c uses the feature values of the N original speakers and the feature value of one target speaker extracted in step S302 as learning data, and the N narratives. Conversion rules are learned so that each voice of a speaker is converted into a voice of one target speaker, and N specific speaker models respectively corresponding to N former speakers are generated. The process proceeds to step S308. Specifically, for each of N original speakers, learning is performed based on the acquired initial voice quality conversion model using the feature amount of each speaker and the feature amount of one target speaker. Then, a voice quality conversion model (= specific speaker model) for each speaker is generated.

  In step S308, the adaptive learning unit 12d performs an adaptation process for adapting the N specific speaker models generated in step S306 to the initial voice conversion model using a predetermined adaptation method, and the process proceeds to step S310. . Specifically, the parameters of one acquired initial voice quality conversion model are estimated using a predetermined adaptive method using all N specific speaker models.

In step S310, the adaptive learning unit 12d determines whether or not the adaptation process has been completed. If it is determined that the adaptation process has ended (Yes), the process proceeds to step S312; otherwise (No), the process ends. The adaptive process is executed until.
When the process proceeds to step S312, the adaptive learning unit 12d determines whether the iterative learning mode is set based on the setting information. If it is determined that the setting is set (Yes), the process proceeds to step S314. If not (No), the process proceeds to step S318.

  When the process proceeds to step S314, the adaptive learning unit 12d evaluates the model after adaptation, and the process proceeds to step S316. For example, when a cepstrum coefficient is used as the spectral characteristic amount of the voice of the voice quality conversion model, it is possible to use cepstral distortion as an evaluation measure of conversion accuracy. The cepstrum distortion is calculated by calculating the distortion between the cepstrum converted from the original speaker and the cepstrum of the target speaker. The cepstrum distortion is expressed by the following formula (1), and the evaluation becomes higher as the calculated value is smaller.

In the above equation (1), C _i ^x represents the cepstrum coefficient of the target speaker's voice, C _i ^y represents the cepstrum coefficient of the converted voice, and p represents the order of the cepstrum coefficient.
In step S316, the adaptive learning unit 12d determines whether or not the evaluation result in step S314 satisfies a predetermined condition. If it is determined that the evaluation is satisfied (Yes), the process proceeds to step S318. When that is not right (No), it transfers to step S320. For example, as a predetermined condition, a threshold value for the cepstrum distortion is given, and adaptive learning is repeatedly performed until the threshold value becomes equal to or less than the threshold value.

  When the process proceeds to step S318, the adaptive learning unit 12d generates one voice quality conversion model common to all the original speakers and the target speaker based on the result of the adaptive processing using a predetermined adaptation method. Then, the series of processes is terminated and the process returns to the original process. Specifically, the parameter value of the initial voice quality conversion model is converted into the parameter value estimated by the adaptive processing to generate a voice quality conversion model. The voice quality conversion model generated in this way is a model in which N former speakers, M target speakers, or both are adapted according to the configuration of the voice quality conversion model specified in the setting information. Become.

On the other hand, if the predetermined condition is not satisfied in step S316 and the process proceeds to step S320, the adaptive learning unit 12d replaces the initial voice quality conversion model with the voice quality conversion model after adaptive learning, and the process proceeds to step S304. .
In step S304, when the configuration of the voice quality conversion model specified by the setting information is not N-to-1 and the process moves to step S322, the specific speaker model generation unit 12c determines the voice quality conversion model specified by the setting information. It is determined whether or not the configuration is 1-to-M. If it is determined that the configuration is 1-to-M (Yes), the process proceeds to step S324. If not (No), the process proceeds to step S328.

  When the process proceeds to step S324, the specific speaker model generation unit 12c uses the feature values of one original speaker and the feature values of M target speakers extracted in step S300 as learning data. M specific speaker models respectively corresponding to the M target speakers are learned by converting conversion rules so as to convert voices of the voice quality of the former speaker to the respective voices of the M target speakers. And the process proceeds to step S326. Specifically, for each speaker of M target speakers, learning is performed based on the acquired initial voice quality conversion model using the feature amount of each speaker and the feature amount of one original speaker. Then, a specific speaker model for each speaker is generated.

  In step S326, the adaptive learning unit 12d performs an adaptation process for adapting the M specific speaker models generated in step S324 to the initial voice conversion model using a predetermined adaptation method, and the process proceeds to step S310. . Specifically, the parameters of one acquired initial voice quality conversion model are estimated using a predetermined adaptive method using all M specific speaker models.

  On the other hand, in step S322, when the configuration of the voice quality conversion model specified by the setting information shifts to step S328 instead of 1: M, the configuration specified by the setting information applies the intermediate speaker in the adaptive learning unit 12d. It is determined whether or not the configuration is the same, and if it is determined that the configuration is that of applying an intermediate speaker (Yes), the process proceeds to step S330, and if not (No), the process proceeds to step S334.

When the process proceeds to step S330, the specific speaker model generation unit 12c uses the feature values of the N original speakers and the feature values of one intermediate speaker extracted in step S300 as learning data. Learning conversion rules to convert each voice of a person's former speaker into one voice of an intermediate speaker, and generate N specific speaker models corresponding to each of N former speakers Then, the process proceeds to step S332.
In step S332, the adaptive learning unit 12d performs an adaptation process for adapting the N specific speaker models generated in step S330 to the initial voice quality conversion model using a predetermined adaptation method, and the process proceeds to step S310. .

  On the other hand, in step S328, when the configuration of the voice quality conversion model specified by the setting information is not the configuration in which the intermediate speaker is applied, but the process moves to step S334, the specific speaker model generation unit 12c extracts in step S300. Using the feature values of the N original speakers and the feature values of the M target speakers as learning data, the voices of the N original speakers are converted into the voices of the M target speakers, respectively. Thus, the conversion rules are learned to generate N × M specific speaker models respectively corresponding to the N original speakers and M target speakers, and the process proceeds to step S336. Specifically, for each speaker of the N original speakers and for each speaker of the M target speakers, the above acquisition is performed using the feature amounts of the original speakers and the target speakers. Learning is performed based on the initial voice quality conversion model, and a specific speaker model is generated for each set of one original speaker and one target speaker.

  In step S336, the adaptive learning unit 12d executes an adaptation process for adapting the N × M specific speaker models generated in step S334 to the initial voice quality conversion model using a predetermined adaptation method, and then proceeds to step S310. Transition. Specifically, the parameters of one acquired initial voice quality conversion model are estimated using a predetermined adaptation method using all N × M specific speaker models.

Next, the operation of this embodiment will be described.
When the voice quality conversion model generation apparatus 1 receives a voice quality conversion model generation instruction (including setting information) by an operation of an operation unit (not shown) by the user (a branch of “Yes” in step S100), the setting information Based on the above, the configuration of the voice quality conversion model to be generated is determined. Here, it is assumed that the voice quality conversion model to be generated is a 1-to-M voice quality conversion model (“Yes” branch in step S114).

  When the configuration of the model is determined, the voice quality conversion model generation unit 12 next selects voice data of the one former speaker specified by the setting information from the former speaker voice data storage unit 10 (the above-described utterance sentence set). voice data of a, b, and c), and voice data of M target speakers specified by the setting information from the target speaker voice data storage unit 11 (speech of the above-described speech sentence sets a, b, and c) Data) is acquired (step S116). That is, the voice data of one former speaker and the voice data of M target speakers are data sets having the same utterance content (hereinafter referred to as a parallel voice data set).

  When the parallel audio data set is acquired, it is determined whether the generation mode of the voice quality conversion model is the normal learning mode or the adaptive learning mode based on the setting information. Here, it is assumed that the adaptive learning mode is set ("No" branch of step S106). As a result, the adaptive learning unit 12d executes a process for generating a 1-to-M voice quality conversion model by adaptive learning using a predetermined adaptation method (step S112).

When the generation process of the voice quality conversion model by adaptive learning is started, first, an initial voice quality conversion model is acquired from the voice quality conversion model storage section 13 in the adaptive learning section 12d (step S300). This initial voice quality conversion model is composed of GMMs trained with a parallel speech data set for pre-learning of one former speaker and M target speakers.
Hereinafter, a voice quality conversion method based on GMM (for example, see T. Toda ct al., Proc. ICASSP, Vol. 1, pp. 9-12, Philadel-phia, USA, Mar. 2005.) will be described in detail. The static / dynamic feature amount X _t of the voice of the original speaker serving as the conversion source and the static / dynamic feature amount Y _t of the voice of the target speaker serving as the conversion destination associated with each frame in the time domain. Are represented by the following formulas (2) and (3), respectively.

X _t = [x _t ^T , Δx _t ^T ] ^T (2)

Y _t = [y _t ^T , Δy _t ^T ] ^T (3)

In the above formulas (2) and (3), p is the number of dimensions of the feature quantity, and T indicates transposition. In the GMM, the joint probability distribution p (X _t , Y _t | λ) of speech feature amounts X _t and Y _t is expressed by the following equation (4).

In the above equation (4), α _i is the weight of the i-th distribution, and m is the number of mixtures. X _t and Y _t each have a 2D dimension. N (x; μ _i , Σ _i ) is a normal distribution having an average vector μ _i and a covariance matrix Σ _{i in the i-th} distribution, and is represented by the following expression (5).

Next, each of the input and output feature value series vector ^{_{X = [X 1 T, ···}} , X T T] T, Y = [Y 1 T, ···, Y T T] When is ^T, static and The likelihood for the dynamic feature quantity sequence is expressed by the following equation (6).

In the frame t, the conditional probability density of the output feature quantity y _i when the input feature quantity X _t is given is modeled by the GMM shown in the following equations (7) and (8).

Here, the average vector E _i (m _i ) and the covariance matrix D (m _i ) of the conditional probability distribution m _i in the above equations (7) and (8) are expressed by the following equations (9) and (10 ).

Further, static feature amount sequence to maximize the likelihood of the above equation _{(6) y = [y 1} T, ···, y T T] ^T-conversion feature amount sequence.

y = argmaxp (Y | X, λ) subject to Y = Wy (11)

Here, W is a transformation matrix that expands a static feature quantity sequence to a static / dynamic feature quantity sequence.

GMM learning is performed by estimating conversion parameters (α _i , μ _i ^(X) , μ _i ^(Y) , Σ _i ^{(X, X)} , Σ _i ^{(Y, X)} ). The combined feature vector z of X _t and Y _t is defined as the following equation (12).

z = [X _t ^T , Y _t ^T ] ^T (12)

Accordingly, the probability distribution p (z) of z is expressed by the following equation (13) by GMM.

In the above equation (13), the covariance matrix Σ _i (z) and the average vector μ _i (z) in the i-th distribution of z are represented by the following equations (14) and (15), respectively.

Here, the conversion parameters (α _i , μ _i ^(X) , μ _i ^(Y) , Σ _i ^{(X, X)} , Σ _i ^{(Y, X)} ) are estimated using a known EM algorithm.
Here, the feature quantity X _t of the equation (2), and feature amount extracted from one source speaker speech data. Further, the feature value Y _t in the above equation (3) is set as the feature value extracted from the speech data of M target speakers. Then, the estimation by the EM algorithm using these feature amounts X _t and Y _t. Then, one GMM common to the M target speakers having the conversion parameter value determined by estimation becomes an initial voice conversion model (hereinafter also referred to as GMMλ ⁰ ) for adaptive learning.

Note that the voice quality conversion model generation processing by GMM learning (parameter estimation) using the EM algorithm described above is the same as the voice quality conversion model generation processing in the normal learning mode.
In other words, in the case of the N-to-1 voice quality conversion model, feature data extracted from parallel speech data sets of N original speakers and one target speaker in the GMM learning are used as learning data. In the case of the N-to-M voice quality conversion model, feature amounts extracted from parallel speech data sets of N original speakers and M target speakers are used as learning data. In the case of the N-to-1 intermediate voice quality conversion model, feature amounts extracted from parallel speech data sets of N former speakers and one intermediate speaker are used as learning data.

On the other hand, when the voice quality conversion model generation unit 12 acquires one voice quality conversion model common to the M target speakers generated as described above as an initial voice quality conversion model (GMMλ ⁰ ), next, the feature amount In the extraction unit 12a, feature amounts are extracted from the respective speech data of the acquired parallel speech data set of the one original speaker and the M target speakers (step S302). Here, 20-dimensional cepstrum coefficients are extracted from the voice data of the original speaker and the voice data of the target speaker by using cepstrum analysis.

Then, learning is performed on the acquired initial voice quality conversion model (GMMλ ⁰ ) using the extracted feature values of the speech data of each target speaker as learning data, and an output average of the initial voice quality conversion model (GMMλ ⁰ ) is obtained. Only the vector μ ^Y _t is updated. Thus, a specific speaker model (hereinafter also referred to as GMMλ ^s ) that is a voice quality conversion model depending on each target speaker is generated (step S324).

Here, in the adaptive learning unit 12d, a specific voice technique is used as a predetermined adaptation method used for adaptive learning.
Hereinafter, a voice quality conversion method using eigenvoice technology will be described.
The 1-to-M voice quality conversion model (hereinafter also referred to as EV-GMMλ ^EV ) generated by adaptive learning using eigenvoice technology is obtained by converting the output average vector μ _i ^(Y) in the above equation (15 ⁾ into the following equation (16 ).

μ _i ^(Y) = B _i w + b _i ⁽⁰⁾ (16)

Here, b _i ⁰ and B _i = [b _i (1),..., B _i (J)] are a matrix composed of a bias vector for the i-th distribution and J representative vectors b _i (j). Represents. Mean vector of an output speaker (target speaker) is, weight vector J dimension on the subspace spanned by _{B i w = [w (1} ), ···, w (J)] is controlled by the ^T. That is, EV-GMMλ ^EV is a free parameter w depending on each output speaker (target speaker) and parameters (α _i , μ _i ^(X) , b _i ⁰ , B, common to all output speakers ⁾ . _i , Σ _i ^{(X, Y)} ).

When each specific speaker model (GMMλ ^s ) of the M target speakers is generated, the adaptive learning unit 12d connects the output average vectors μ _i ^(Y) (s) to each target speaker. 2DM super vector SV ^s (the following equation (17)) is constructed.

SV ^s = [μ ₁ ^(Y) (s) ^T ,..., Μ _s ^(Y) (s) ^T ] ^T (17)

Then, principal component analysis is performed on each super vector of the M target speakers to determine a matrix B _i based on the bias vector b _i ⁰ and the eigenvector. Thereby, SV ^s is expressed by the following formula (18).

Here, S is the target speaker number (here, target speaker number M), and w ^s is J (<S << 2DM) principal components for the s-th target speaker. The above b _i ⁰ and B _i are adapted to GMMλ ⁰ (step S326) to obtain EV-GMMλ ^EV (“Yes” branch of step S310).

If the iterative learning mode is set in the setting information (“Yes” branch in step S312), the cepstrum distortion is calculated using the evaluation voice data and the EV-GMMλ ^EV obtained by adaptive learning. calculate. Then, the calculated value is compared with a threshold value set in advance as a predetermined condition (step S314). Here, assuming that the condition is satisfied if the calculated cepstrum distortion is equal to or less than the threshold value, the generated EV-GMMλ ^EV is used as a final one-to-M voice conversion model (“Yes” branch in step S316). ).

On the other hand, if the condition is not satisfied (“No” branch in step S316), the generated EV-GMMλ ^EV is replaced with the current initial voice conversion model (GMMλ ⁰ ), and this is replaced with a new initial voice quality. As the conversion model, the specific speaker model is generated using the acquired speech data set of the target speaker, and EV-GMMλ ^EV is generated by adaptive learning using eigenvoice technology. At this time, in the learning of the specific speaker model, the feature amount already extracted at the first learning is used. That is, until the predetermined condition is satisfied, EV-GMMλ ^EV generation processing by repeated adaptive learning is performed while replacing the generated EV-GMMλ ^EV with the initial voice quality conversion model.

  The one-to-M voice quality conversion model satisfying the predetermined condition is suitable for converting the voice of a specific speaker (one former speaker) into the voice of an unspecified speaker (any of M target speakers). It becomes a 1-to-M voice quality conversion model in which various weight vectors are set. In addition, this one-to-M voice quality conversion model unifies the phoneme space modeled by each distribution due to the restriction that the probability density distribution on the input side is common among all output speakers (M target speakers). Is planned. Thus, a super vector space in which phonological and speaker characteristics are separated is configured.

The 1-to-M voice quality conversion model generated in this way is stored in the voice quality conversion model storage unit 13 (step S110).
In the case of producing by the adaptive learning N-to-1 voice conversion model, the GMM learning using EM algorithm as described above, the feature quantity X _t of the equation (1), the original story of N's for pre-learning A static and dynamic feature amount extracted from the speech data of N former speakers in the parallel speech data set of the speaker and one target speaker, and the feature amount Y _t of the above equation (2) Static and dynamic feature amounts extracted from the speech data of one target speaker in the speech data set. A GMM generated by learning using these feature quantities as learning data is used as an initial voice quality conversion model (GMMλ ⁰ ).

Further, the N-to-1 voice quality conversion model (EV-GMMλ ^EV ) generated by adaptive learning using the eigenvoice technique is obtained by changing the input average vector μ _i ^X in the above equation (15) to the following equation (19). .

μ _i ^(X) = D _i w + d _i ⁰ (19)

Here, d _i ⁰ and D _i = [d _i (1),..., D _i (K)] are matrices composed of bias vectors for the i-th distribution and K representative vectors d _i (K). Represents. The average vector of the input speakers (former speakers) is controlled by a K-dimensional weight vector w = [w (1),..., W (K)] ^T on the subspace spanned by D _i . That is, EV-GMMλ ^EV is a free parameter w depending on each input speaker (former speaker) and parameters (α _i , μ _i ^(Y) , d _i ⁰ , D, common to all input speakers. _i , Σ _i ^{(X, Y)} ).

Then, learning is performed using the feature amount X _t of the speech data of N former speakers, and only the input average vector μ ^(X) is updated. As a result, a specific speaker model (hereinafter also referred to as GMMλ ^v ) that is a voice quality conversion model depending on each original speaker is generated (step S306).
When each specific speaker model (GMMλ ^v ) of N original speakers is generated, the adaptive learning unit 12d connects the input average vectors μ _i ^(X) (v) to each original speaker. 2DM-dimensional super vector SV ^v (the following equation (20)).

SV ^v = [μ ₁ ^(Y) (v) ^T ,..., Μ _v ^(Y) (v) ^T ] ^T (20)

Then, a principal component analysis is performed on each super vector of N former speakers, thereby determining a matrix D _i using a bias vector d _i ⁰ and an eigenvector. Thus, SV ^v is expressed as the following equation (21).

Here, V is the number of former speakers (here, the number of former speakers N), and w ^v is K (<V << 2DM) principal components for the v-th former speaker. The above d _i ⁰ and D _i are adapted to GMMλ ⁰ (step S326) to obtain an N-to-1 voice quality conversion model (EV-GMMλ ^EV ) (“Yes” branch in step S310). This N-to-1 voice quality conversion model unifies the phoneme space modeled by each distribution due to the restriction that the probability density distribution on the output side is the same for all input speakers (N former speakers). It is done. Thus, a super vector space in which phonological and speaker characteristics are separated is configured.

The N-to-1 intermediate voice quality conversion model generation processing is the same as the N-to-1 voice quality conversion model, except that the voice data of one intermediate speaker is used instead of the voice data of one target speaker. It becomes.
The N to M voice quality conversion model is a combination of the 1 to M voice quality conversion model and the N to 1 voice quality conversion model. That is, the N-to-M voice quality conversion model includes two models, one N-to-one voice quality conversion model and one 1-to-M voice quality conversion model. At this time, one target speaker of N to 1 and one former speaker of 1 to M are set as a common speaker. Note that the N-to-M voice quality conversion model can be generated as one N-to-M voice quality conversion model common to N original speakers and M target speakers.

  As described above, according to the voice quality conversion model generation device 1 of the present embodiment, any one of the N former speakers is obtained using the voice data of at least one of the N former speakers and the M target speakers. N-to-1 voice quality conversion model for converting speech to the speech of one target speaker, 1-to-M voice quality conversion model for converting speech of one former speaker to the speech of any of M target speakers, It is possible to generate an N-to-M voice quality conversion model that converts any speech of N original speakers to any speech of M target speakers. Also, the voice of any one of N former speakers is converted into the voice of one intermediate speaker, and the voice of one intermediate speaker is converted into the voice of any of M target speakers. An N to M intermediate voice quality conversion model can be generated.

  Thus, if the N-to-1 voice quality conversion model is used, the N-to-1 voice quality conversion model is adapted to the voice of an arbitrary (new) original speaker by using a predetermined adaptation method. Can be converted into the voice of one target speaker. Further, in the case of the 1-to-M voice quality conversion model, the 1-to-M voice quality conversion model is adapted to the voice of an arbitrary target speaker using a predetermined adaptation method, whereby the voice of one former speaker is It can be converted to the voice of any target speaker. Further, in the case of an N-to-M voice quality conversion model, the voice of an arbitrary former speaker can be arbitrarily changed by adapting the N-to-M voice quality conversion model to the voice of an arbitrary former speaker and an arbitrary target speaker. The target speaker's voice can be converted.

  Further, adaptation to the voice of an arbitrary speaker can be performed using a small number of voice data of the arbitrary speaker using a predetermined adaptation method. In particular, by generating a voice quality conversion model using eigenvoice technology, it is possible to control speaker characteristics with a small amount of parameters by creating a subspace in consideration of inter-speaker fluctuations, making it easier to convert the voice quality conversion model. Can be adapted to the voice of any speaker.

In the first embodiment, the voice quality conversion model generation unit 12 corresponds to the voice quality conversion model generation means according to claim 1 or 2 .
In the first embodiment, steps S104 to S112, S116, and S112 correspond to the voice quality conversion model generation step recited in claim 6 .

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to the drawings. 6 to 9 are diagrams showing a second embodiment of the voice quality conversion system, voice quality conversion program, and voice quality conversion method according to the present invention.
First, the configuration of the voice quality conversion system according to the present invention will be described with reference to FIG. FIG. 6 is a block diagram showing the configuration of the voice quality conversion system 2 according to the second embodiment of the present invention.

  As shown in FIG. 6, the voice quality conversion system 2 includes a voice quality conversion model storage unit 20 that stores a voice quality conversion model, a target speaker voice data acquisition unit 21 that acquires voice data of a target speaker, and a former speaker's voice data. An original speaker voice data acquisition unit 22 that acquires voice data, a voice quality conversion model adaptation unit 23 that adapts the voice quality conversion model to at least one voice of the original speaker and the target speaker, and a voice quality conversion model after adaptation. The voice quality conversion unit 24 converts the voice of the original speaker into the voice of the target speaker, and the speaker control parameter specifying unit 25 that specifies the speaker control parameter value. Then, a normal mode for performing voice quality conversion by adapting a voice quality conversion model using the voice data of the original speaker and the voice data of the target speaker, and parameters (speakerness control parameters) of the model unit on the target speaker side There are two modes: a parameter control mode in which a value is specified and a voice quality conversion model is adapted based on the specified value to perform voice quality conversion.

  The voice quality conversion model storage unit 20 includes the N to 1, 1 to M and N to M voice quality conversion models and the N to M intermediate voice quality conversion models generated by the voice quality conversion model generation apparatus 1 according to the first embodiment. At least one type of model is stored. Note that the voice quality conversion model may store at least one of these four types one by one, or not only one by one but may store a plurality for each condition according to the model generation conditions.

  The target speaker voice data acquisition unit 21 has a function of acquiring voice data of the target speaker used for adaptation of the voice quality conversion model. Specifically, the voice of the target speaker is input via a microphone, and signal processing is performed on the input voice signal to generate voice data. In addition, it has a function of acquiring voice data for adaptation of the target speaker from an external device or the like via a network or the like.

  The former speaker voice data acquisition unit 22 has a function of acquiring voice data of the former speaker used for adaptation of the voice quality conversion model and voice data of the voice quality conversion target of the former speaker. Specifically, the voice of the original speaker is input through a microphone, and signal processing is performed on the input voice signal to generate voice data for adaptation of the original speaker or voice quality conversion of the original speaker. It has a function. In addition, it has a function of acquiring voice data for adaptation of the original speaker and voice data for voice quality conversion of the original speaker via a network or the like from an external device or the like.

  The voice quality conversion model adaptation unit 23 acquires a voice quality conversion model to be used for voice quality conversion from the voice quality conversion model storage unit 20, and according to the configuration of the acquired voice quality conversion model, the voice data of the original speaker for adaptation and the adaptation Based on at least one of the target speaker's voice data and the designated speaker control parameter value, the acquired voice quality conversion model is converted into the voice of the original speaker and the target speaker using a predetermined adaptive method. And a voice quality conversion model after adaptation by adapting to at least one voice represented by the speaker control parameter value.

  The voice quality conversion unit 24 uses the original speaker voice data conversion voice data acquired by the original speaker voice data acquisition unit 22 and the post-adaptation voice quality conversion model generated by the voice quality conversion model adaptation unit 23 to A function for converting the voice of the voice quality of the speaker to be converted into the voice of the voice quality of the target speaker and outputting the converted voice (hereinafter referred to as converted voice) via an amplifier and a speaker (not shown). have.

  The speaker control parameter specifying unit 25 operates in the parameter control mode, and is a model on the target speaker side of the 1-to-M or N-to-M voice quality conversion model used for voice quality conversion obtained from the voice quality conversion model storage unit 20. And a function for designating an arbitrary value (speaker control parameter value) for the talker control parameter (weight vector) constituting the unit. Specifically, a value input from the user via an operation unit (not shown) can be specified as a speaker control parameter value.

  In this embodiment, the voice quality conversion system 2 includes a processor (not shown), a RAM (Random Access Memory), and a ROM (Read Only Memory) in which a dedicated program is stored. By executing the program, the above functions are performed. In addition, the above-described units include those that perform their functions only with dedicated programs, and those that perform their functions by controlling hardware using dedicated programs.

Further, a detailed configuration of the voice quality conversion model adaptation unit 23 will be described with reference to FIG. Here, FIG. 7 is a block diagram showing a detailed configuration of the voice quality conversion model adaptation unit 23.
As shown in FIG. 7, the voice quality conversion model adaptation unit 23 includes a model acquisition unit 23a, an adaptation target extraction unit 23b, a feature amount extraction unit 23c, and an adaptation unit 23d.
The model acquisition unit 23a reads out the corresponding voice quality conversion model from the voice quality conversion model storage unit 20 based on the information of the voice quality conversion model to be used included in the voice quality conversion start instruction information, and outputs it to the adaptation target extraction unit 23b. It has a function to do.

  The adaptation target extraction unit 23b has a function of extracting a model part having parameters to be adapted from the model according to the configuration of the voice quality conversion model input from the model acquisition unit 23a. Specifically, the voice quality conversion model has, for example, a 40-dimensional parameter including 20 dimensions on the original speaker side and 20 dimensions on the target speaker side. Of these 40-dimensional parameters, the target of adaptation is a parameter portion generated from the speech data of N original speakers or M target speakers. Therefore, in the case of the N-to-1 voice quality conversion model, a model part on the former speaker side composed of parameters of N former speakers is extracted as an adaptation target. Similarly, if the model is a one-to-M voice quality conversion model, the model on the M target speaker side is an N-to-M intermediate voice quality conversion model. Each model is extracted. On the other hand, in the case of the N-to-M voice quality conversion model, since both the original speaker side and the target speaker side are to be applied, these are extracted, respectively. In this way, the extracted model part is output to the adaptation part 23d.

  The feature amount extraction unit 23c is acquired from the original speaker voice data acquisition unit 22 and the target speaker voice data acquisition unit 21 in the same manner as the feature amount extraction unit 12a of the voice quality conversion model generation device 1 of the first embodiment. To extract feature values (feature parameters) from the original speaker voice data and target speaker voice data by analysis processing such as cepstrum analysis or linear prediction analysis according to the parameter characteristics of the voice quality conversion model used for voice quality conversion have. The extracted feature amount of the original speaker (hereinafter referred to as the original speaker feature amount) and the target speaker feature amount (hereinafter referred to as the target speaker feature amount) are output to the adaptation unit 23d.

  The adaptation unit 23d, for the former speaker side model unit and the target speaker side model unit input from the adaptation target extraction unit 23b, and the former speaker feature amount and target story input from the feature amount extraction unit 23c. A parameter that constitutes the model unit using a predetermined adaptation method based on a person feature, and a function for generating a post-adaptation voice quality conversion model based on the estimated parameter value.

In addition, when a parameter value is specified by the speaker control parameter specifying unit 25, the target speaker side parameter is adapted to the specified parameter value by using the specified parameter value, and post-adaptive voice quality conversion is performed. It also has a function to generate a model. The generated post-adaptation voice quality conversion model is output to the voice quality conversion unit 24.
Furthermore, the flow of voice quality conversion processing in the normal mode in the voice quality conversion system 2 will be described with reference to FIG. Here, FIG. 8 is a flowchart showing voice quality conversion processing in the normal mode in the voice quality conversion system 2.

  As shown in FIG. 8, the voice quality conversion process in the normal mode first proceeds to step S400, where the voice quality conversion model adapting unit 23 receives a voice quality conversion start instruction (start instruction information from the user via an operation unit not shown). If it is determined (Yes), the process proceeds to step S402. If not (No), the determination process is repeated until there is an instruction.

When the process proceeds to step S402, the model acquisition unit 23a acquires the voice quality conversion model having the configuration designated by the start instruction information from the voice quality conversion model storage unit 20, and uses the acquired voice quality conversion model as the adaptation target extraction unit 23b. And the process proceeds to step S404.
In step S404, the voice quality conversion model adaptation unit 23 determines whether or not the configuration of the voice quality conversion model specified by the start instruction information is an N-to-1 configuration or a configuration adapted to an intermediate speaker. If it is determined that the configuration is (Yes), the process proceeds to step S406. If not (No), the process proceeds to step S426.

When the process proceeds to step S406, the adaptation target extraction unit 23b extracts the model part on the original speaker side from the N-to-1 voice quality conversion model or the N-to-1 intermediate voice quality conversion model acquired in step S402. The model part on the former speaker side is output to the adaptation unit 23d, and the process proceeds to step S408.
In step S408, the voice quality conversion model adaptation unit 23 requests the user to acquire adaptation original speaker voice data, and the process proceeds to step S410. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S410, the original speaker voice data acquisition unit 22 determines whether the voice data of the original speaker for adaptation has been acquired (or generated from the input voice), and if it is determined that the voice data has been acquired (generated) (Yes) outputs the acquired (generated) original voice data for adaptation to the feature amount extraction unit 23c, and proceeds to step S412.
In step S412, the feature amount extraction unit 23c extracts feature amounts from the adaptation original speaker voice data input from the original speaker voice data acquisition unit 22, and the extracted original speaker feature amounts are applied to the adaptation unit 23d. And the process proceeds to step S414.

In step S414, the adaptation unit 23d estimates a parameter value of the model unit on the former speaker side using a predetermined adaptation method based on the former speaker feature amount input from the feature amount extraction unit 23c, and then proceeds to step S416. Transition.
In step S416, the adaptation unit 23d generates a post-adaptation original speaker speech model based on the parameter value estimated in step S414, outputs the generated post-adaptation voice quality conversion model to the voice quality conversion unit 24, and proceeds to step S418. Transition.

In step S418, the voice quality conversion unit 24 requests the user to acquire original speaker voice data for voice quality conversion, and the process proceeds to step S420. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S420, the original speaker voice data acquisition unit 22 determines whether or not the voice data of the original speaker for voice quality conversion has been acquired (or generated from the input voice), and is determined to have been acquired (generated). If yes (Yes), the acquired (generated) original speaker voice data for voice quality conversion is output to the voice quality converting unit 24 and the process proceeds to step S422. If not (No), until acquired (generated). The determination process is repeated (or the generation process is performed).

When the process proceeds to step S422, the voice quality conversion unit 24 converts the voice data of the voice quality of the original speaker to be converted, which is input from the voice data acquisition unit 22, into voice data of the target speaker's voice quality. Then, the process proceeds to step S424.
In step S424, the voice quality conversion unit 24 outputs the converted voice based on the voice data after the voice quality conversion, and ends the process.

  On the other hand, if the configuration of the voice quality conversion model is not an N-to-1 configuration or a configuration in which an intermediate speaker is applied in step S404, and the process proceeds to step S426, the voice quality conversion model adaptation unit 23 specifies the start instruction information. It is determined whether the configuration of the voice quality conversion model is a one-to-M configuration. If it is determined that the configuration is a one-to-M configuration (Yes), the process proceeds to step S428; otherwise (No). The process proceeds to step S438.

When the process proceeds to step S428, the adaptation target extraction unit 23b extracts the model unit on the target speaker side from the 1-to-M voice quality conversion model acquired in step S402, and extracts the model unit on the target speaker side thus extracted. Is output to the adaptation unit 23d, and the process proceeds to step S430.
In step S430, the voice quality conversion model adaptation unit 23 requests the user to acquire adaptation target speaker voice data, and the process proceeds to step S432. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S432, the target speaker voice data acquisition unit 21 determines whether or not the voice data of the target speaker for adaptation has been acquired (or generated from the input voice), and when it is determined that the target speaker voice data has been acquired (generated) (Yes) outputs the acquired (generated) target speaker voice data for adaptation to the feature amount extraction unit 23c, and proceeds to step S434.
In step S434, the feature amount extraction unit 23c extracts feature amounts from the target speaker voice data for adaptation input from the target speaker voice data acquisition unit 21, and the extracted target speaker feature amounts are applied to the adaptation unit 23d. And the process proceeds to step S436.

In step S436, the adaptation unit 23d estimates a parameter value of the model unit on the target speaker side using a predetermined adaptation method based on the target speaker feature amount input from the feature amount extraction unit 23c, and then proceeds to step S416. Transition.
In step S426, if the configuration of the voice quality conversion model is not 1 to M and the process proceeds to step S438, the adaptation target extraction unit 23b uses the N to M voice quality conversion model acquired in step S402 to obtain the original speaker. Both the model unit on the side and the model unit on the target speaker side are extracted, and the extracted model unit on the original speaker side and the model unit on the target speaker side are output to the adaptation unit 23d, and the process proceeds to step S440.

In step S440, the voice quality conversion model adaptation unit 23 requests the user to acquire the original speaker voice data and target speaker voice data for adaptation, and the process proceeds to step S442. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S442, it is determined whether or not the original speaker voice data acquisition unit 22 and the target speaker voice data acquisition unit 21 have acquired (or generated from input voice) voice data of the original speaker and target speaker for adaptation. When it is determined that each has been acquired and acquired (generated) (Yes), the acquired (generated) adaptation original voice data and target speaker audio data are output to the feature amount extraction unit 23c. The process proceeds to step S444.

In step S444, the feature amount extraction unit 23c uses the original speaker voice data for adaptation input from the source speaker voice data acquisition unit 22 and the target speaker for adaptation input from the target speaker voice data acquisition unit 21. The feature amount is extracted from the speech data, and the extracted original speaker feature amount and target speaker feature amount are output to the adaptation unit 23d, and the process proceeds to step S446.
In step S446, the adaptation unit 23d estimates a parameter value of the model unit on the former speaker side using a predetermined adaptation method based on the former speaker feature amount input from the feature amount extraction unit 23c. Based on the target speaker feature input from the extraction unit 23c, the parameter values of the model unit on the target speaker side are estimated using a predetermined adaptation method, and the process proceeds to step S416.

Further, the flow of voice quality conversion processing in the parameter control mode in the voice quality conversion system 2 will be described with reference to FIG. Here, FIG. 9 is a flowchart showing voice quality conversion processing in the parameter control mode in the voice quality conversion system 2.
As shown in FIG. 9, the voice quality conversion process in the parameter control mode first proceeds to step S500, and in the voice quality conversion model adaptation section 23, whether or not a voice quality conversion start instruction is received from the user via an operation section (not shown). If it is determined (Yes), the process proceeds to step S502. If not (No), the determination process is repeated until an instruction is given.

When the process proceeds to step S <b> 502, the model acquisition unit 23 a obtains the voice quality conversion model (either 1-to-M or N-to-M voice quality conversion model) having the configuration specified by the start instruction information from the voice quality conversion model storage unit 20. Acquire and move to step S504.
In step S504, the voice quality conversion model adaptation unit 23 determines whether or not the configuration of the voice quality conversion model specified by the start instruction information is a 1-to-M configuration, and if it is determined to be 1-to-M (Yes) ) Proceeds to step S506, otherwise (No) proceeds to step S522.

When the process proceeds to step S506, the adaptation target extraction unit 23b extracts the model unit on the target speaker side from the one-to-M voice quality conversion model acquired in step S502, and extracts the model unit on the target speaker side thus extracted. Is output to the adaptation unit 23d, and the process proceeds to step S508.
In step S508, the voice quality conversion model adaptation unit 23 requests the user to specify the speaker control parameter value of the target speaker model for adaptation, and the process proceeds to step S510. Here, the designation request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S510, the speaker control parameter specifying unit 25 determines whether or not the speaker control parameter is specified via an operation unit (not shown). If it is determined that the speaker control parameter is specified (Yes), The designated control parameter value is output to the adaptation unit 23d, and the process proceeds to step S512. If not (No), the determination process is repeated until designated.
When the process proceeds to step S512, the adaptation unit 23d uses the speaker control parameter value input from the speaker control parameter specification unit 25, and then adapts to the target speaker's voice expressed by the parameter value. A voice quality conversion model is generated, the generated post-adaptation voice quality conversion model is output to the voice quality conversion unit 24, and the process proceeds to step S514.

In step S514, the voice quality conversion unit 24 requests the user to acquire original speaker voice data for voice quality conversion, and the process proceeds to step S516. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S516, the original speaker voice data acquisition unit 22 determines whether or not the voice data of the original speaker for voice quality conversion has been acquired (or generated from the input voice), and is determined to have been acquired (generated). If yes (Yes), the acquired (generated) original speaker voice data for voice quality conversion is output to the voice quality converting unit 24, and the process proceeds to step S518. Otherwise (No), until acquired (generated). The determination process is repeated (or the generation process is performed).

When the process proceeds to step S518, the voice quality conversion unit 24 converts the voice data of the voice quality of the original speaker to be converted, which is input from the voice data acquisition unit 22, into voice data of the target speaker's voice quality. Then, the process proceeds to step S520.
In step S520, the voice quality conversion unit 24 outputs the converted voice based on the voice data after the voice quality conversion, and ends the process.

  On the other hand, if the configuration of the acquired voice quality conversion model is not 1-to-M in step S504, and the process proceeds to step S522, the adaptation target extraction unit 23b uses the original N-to-M voice quality conversion model acquired in step S502. Both the model part on the speaker side and the model part on the target speaker side are extracted, and the extracted model part on the original speaker side and the model part on the target speaker side are output to the adaptation unit 23d, and the process proceeds to step S524. To do.

In step S524, the voice quality conversion model adaptation unit 23 requests the user to acquire the original speaker voice data for adaptation, and the process proceeds to step S526. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S526, it is determined whether or not the voice data of the original speaker for adaptation has been acquired (or generated from the input voice) in the original speaker voice data acquisition unit 22, and it is determined that the voice data has been acquired (generated). (Yes) outputs the acquired (generated) original voice data for adaptation to the feature amount extraction unit 23c, and proceeds to step S528.

In step S528, the feature amount extraction unit 23c extracts feature amounts from the adaptation original speaker voice data input from the original speaker voice data acquisition unit 22, and the extracted original speaker feature amounts are applied to the adaptation unit 23d. And the process proceeds to step S530.
In step S530, the voice quality conversion model adaptation unit 23 requests the user to specify the speaker control parameter value of the target speaker side model for adaptation, and the process proceeds to step S532. Here, the designation request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S532, the speaker control parameter specifying unit 25 determines whether or not a speaker control parameter is specified via an operation unit (not shown). If it is determined that the speaker control parameter is specified (Yes), The designated control parameter value is output to the adaptation unit 23d, and the process proceeds to step S534. If not (No), the determination process is repeated until designated.
In step S534, the adaptation unit 23d estimates a parameter value of the model unit on the former speaker side using a predetermined adaptation method based on the former speaker feature amount input from the feature amount extraction unit 23c, and then proceeds to step S536. Transition.
In step S536, the adaptation unit 23d generates a post-adaptation voice quality conversion model based on the parameter value estimated in step S534 and the speaker control parameter value specified in step S532, and the generated post-adaptation voice quality The conversion model is output to the voice quality conversion unit 24 and the process proceeds to step S514.

Next, the operation of this embodiment will be described.
Here, the voice quality conversion model storage unit 20 stores the one-to-M voice quality conversion model (EV-GMMλ ^EV ) generated using the eigenvoice technology in the voice quality conversion model generation apparatus 1 in the first embodiment. Suppose that it is done. The specific operation of the voice quality conversion system 2 having this 1-to-M voice quality conversion model will be described.

First, the voice quality conversion process in the normal mode will be described.
In the voice quality conversion system 2, first, when a voice quality conversion start instruction (including start instruction information) is input by an operation of an operation section (not shown) by the user ("Yes" branch in step S400), the model acquisition section 23a Based on the start instruction information, the voice quality conversion model specified by the start instruction information is acquired from the voice quality conversion model storage unit 20 (step S402). Here, it is assumed that the 1-to-M voice quality conversion model generated using the eigenvoice technique is designated (“Yes” branch in step S426).

When the 1-to-M voice quality conversion model is acquired, the voice quality conversion model adaptation unit 23 extracts the model part on the target speaker side in the adaptation target extraction unit 23b (step S428). The model portion on the target speaker side includes an output average vector μ _i ^(Y) having b _i ⁰ and B _i determined based on the super vectors of M target speakers.
When the model part on the target speaker side is extracted (or in parallel with the extraction process), the target speaker voice data acquisition unit 21 acquires voice data of the target speaker for adaptation (step S430). Here, the utterance content of the target speaker may be any content. For example, the speech content (non-parallel) may be different from the parallel speech data set when the voice quality conversion model is generated. Further, the voice data may be about one utterance to several utterances, and it is sufficient if a very small number (for example, 1/10 or less) of voice data can be acquired for an utterance sentence set of 50 sentences at the time of model generation.

  When the voice data of the target speaker for adaptation is acquired (“Yes” branch of step S432), the feature amount extraction unit 23c extracts the feature amount from the acquired speech data (step S434). Using the extracted feature amount, an optimum weight vector in the model unit on the target speaker side is estimated (step S436). Here, since the acquired 1-to-M voice quality conversion model is generated using the eigenvoice technique, the estimated value of the weight vector can be easily calculated by the maximum likelihood estimation by the EM algorithm using the extracted feature amount as learning data. It is possible to obtain

Hereinafter, adaptive processing using the EM algorithm will be described.
In the voice quality conversion based on the eigenvoice technique, maximum likelihood conversion using the EM algorithm is performed based on EV-GMMλ ^EV in which the weight vector is appropriately set as in the conventional method.
Hereinafter, a method for determining the weight vector will be described.
In unsupervised conversion model learning, adaptation of the voice quality conversion model to a desired output speaker (target speaker) is performed by changing the weight vector w of EV-GMMλ ^EV from the target speaker's voice data only to the following equation (22). Based on maximum likelihood estimation based on this.

In the above equation (22), Y _t (tar) represents the feature vector sequence of the output speaker (target speaker). The estimated weight vector is expressed by the following equations (23) to (25).

When the adaptive unit 23d estimates the weight vector represented by the above equations (23) to (25) for the output average vector μ _i ^(Y) by the maximum likelihood estimation using the EM algorithm, the acquired 1 A weight vector of the anti-M voice quality conversion model (output average vector μ _i ^(Y) ) is converted into the estimated weight vector to generate a post-adaptation voice quality conversion model (step S416).

Once the post-adaptation voice quality conversion model has been generated, next, the original speaker voice data acquisition unit 22 requests acquisition of voice data of an arbitrary original speaker as a conversion candidate (step S418). Data is acquired ("Yes" branch of step S420).
When the voice data of an arbitrary original speaker as a conversion candidate is acquired, the voice quality conversion unit 24 uses the generated post-adaptation voice quality conversion model to convert the voice data of the original speaker's voice quality to the target speaker. Is converted into voice data of the voice quality (step S422). Based on the voice data after the voice quality conversion, the converted voice is output via an amplifier and a speaker (not shown) (step S424).

For the generated voice quality conversion model, known adaptive methods such as maximum likelihood linear regression (MLLR), maximum a posteriori probability estimation method (MAP), constrained MLLR (CMLLR), and a combination of MLLR and MAP are used. To adapt the voice quality conversion model.
Specifically, in the case of the N-to-1 voice quality conversion model, the voice quality conversion model is adapted to the voice of an arbitrary original speaker (steps S414 and S416), and the arbitrary narrative is used by using the voice quality conversion model after the adaptation. The voice data of the person's voice quality is converted into voice data of the voice quality of one target speaker.

  In the case of the N-to-M voice quality conversion model, the N-to-1 voice quality conversion model is adapted to the speech of an arbitrary original speaker, and the 1-to-M voice quality conversion model is adapted to the speech of an arbitrary target speaker (step S446). , S416). First, the voice data of the voice quality of the arbitrary original speaker is converted into the voice data of the voice quality of one target speaker using the N-to-1 voice quality conversion model after adaptation. Next, using the 1-to-M voice quality conversion model after adaptation, the voice data converted into the voice quality of the one target speaker is converted into voice data of the voice quality of the arbitrary target speaker. That is, by performing voice quality conversion in two stages, voice data of an arbitrary original speaker is converted into voice data of a voice quality of an arbitrary target speaker.

Note that in the case of the N-to-M voice quality conversion model, one N-to-M voice quality conversion model common to N original speakers and M target speakers is used as the voice of any original speaker and any target speech. It is also possible to adapt to the person's voice.
Further, the voice quality conversion system 2 can set the parameter control mode as described above for the voice quality conversion model generated using the eigenvoice technology. By setting the parameter control mode, the weight vector of the output average vector μ _i ^(Y) in the one-to-M voice quality conversion model is set as the speaker control parameter, and the parameter value is set to an arbitrary value of the user. It is possible to make the voice quality conversion system 2 function like an equalizer.

Specifically, after extracting the model part on the target speaker side from the acquired 1-to-M voice quality conversion model, the user is requested to specify the speaker control parameter (step S508). When the user manually inputs a speaker control parameter value via an operation unit (not shown), the speaker control parameter specifying unit 25 outputs (specifies) the input parameter value to the adaptation unit 23d. (Branch “Yes” in step S510). The adaptation unit 23d converts the weight vector of the output average vector μ _i ^{(Y) in} the acquired 1-to-M voice quality conversion model into the designated control parameter value, thereby generating a post-adaptation voice quality conversion model ( Step S512).

  As described above, according to the voice quality conversion system 2 of the present embodiment, N-to-1 and 1-pair generated using at least one of the speech data of N former speakers and the speech data of M target speakers. The M, N-to-M voice quality conversion model can be used for voice quality conversion by adapting the voice of a desired (arbitrary) original speaker or a desired (arbitrary) target speaker. As a result, the voice data of an arbitrary utterance content of a specific original speaker can be easily converted into voice data of the desired target speaker's voice quality, or the voice data of an arbitrary utterance content of a desired original speaker can be converted. It can be easily converted into voice data of a voice quality of a specific target speaker or a desired target speaker.

  In addition, N-to-one, one-to-M, and N-to-M voice quality generated using eigenvoice technology and using at least one of voice data of N original speakers and voice data of M target speakers The conversion model can be used for voice conversion by adapting the voice of a desired (arbitrary) original speaker or a desired (arbitrary) target speaker. As a result, the voice quality conversion model can be easily applied to the voice of the desired speaker by the EM algorithm using a small number of voice data (which may be non-parallel) of at least one of the desired original speaker and the desired target speaker. Can be made.

  In addition, a one-to-M or N-to-M voice quality conversion model generated using eigenvoice technology and using at least one of voice data of N original speakers and voice data of M target speakers The user can specify an arbitrary value for the value of the weight vector (speaker control parameter) within a predetermined range. As a result, even if the voice quality of the converted voice is finely adjusted or the target speaker's voice data cannot be obtained, the voice control parameter value is manually controlled to obtain a converted voice having a desired voice quality or a voice quality close thereto. Can be. In addition, if other speech parameters such as F0 are controlled simultaneously, the super vector space may be configured by an average vector of all speech parameters desired to be controlled, not only the spectrum.

In the second embodiment, the target-speaker speech data acquiring unit 21 corresponds to the target-speaker speech data acquisition means according to claim 3 or 4, Motohanashi's voice data obtaining unit 22, claim 3 Or the voice quality conversion model adaptation unit 23 corresponds to the adaptation unit according to claim 4 or 5 , or the second adaptation unit according to claim 3 , and the voice quality conversion unit Reference numeral 24 corresponds to the voice quality conversion means according to the third or fourth aspect .
In the second embodiment, the speaker control parameter specifying unit 25 corresponds to the speaker control parameter value specifying means described in claim 3 .

[Example 1]
Furthermore, Example 1 of the voice quality conversion system 2 of the present invention will be described with reference to FIG. Here, FIG. 10 shows evaluation data when the adaptive method is used for the one-to-M voice conversion model generated by the method of the present invention and when the adaptive method is used for the voice conversion model generated by the conventional method. It is a figure which shows the mel cepstrum distortion between the conversion audio | voice and the natural voice of an output speaker in FIG.

Using one male as an input speaker, voice quality conversion using the 1-to-M voice quality conversion model of the present invention was evaluated.
In the evaluation, a total of 160 males and 80 females in the JNAS data were used as pre-learning output speakers. An output speaker for evaluation was used. Each pre-learning speaker data is one of subsets A to G consisting of 50 sentences in the phoneme balance 503 sentences, and the evaluation speaker data is a subset J consisting of 53 sentences. As input speaker data, subsets A to G were used for prior learning, and subset J was used for evaluation.

At the time of learning the voice quality conversion model for the speaker for evaluation, 1 to 32 sentences out of 53 sentences were used, and the remaining 21 sentences were used for evaluation. In the conventional voice quality conversion model learning, the voice quality conversion model learning (that is, supervised learning) is performed using parallel speech data with the input speaker. On the other hand, in the learning of the one-to-M voice quality conversion model using the eigenvoice technique which is the method of the present invention, the EV-GMM is previously learned and the weight vector is estimated using only the data of the speaker for evaluation. (Ie unsupervised learning). The number of mixtures in EV-GMM was 512, and the number of representative vectors was 1591. The number of mixtures in the voice conversion model of the conventional method was determined afterwards so that the mel cepstrum distortion in the evaluation data was minimized for each speaker and each learning data amount. In both the 1-to-M voice quality conversion model (EV-GMMλ ^EV ) of the present invention and the voice quality conversion model of the conventional method, only the diagonal component is used for the covariance matrix and the mutual covariance matrix. The analysis conditions are the same as in the literature (T. Toda et al., Proc. ICASSP, Vol. 1, pp. 912, Philadel-phia, USA, Mar. 2005.).

Here, in the graph shown in FIG. 10, the vertical axis represents the cepstrum distortion [dB], and the horizontal axis represents the number of learning data used for adaptation. Also, the dotted line in FIG. 10 shows the cepstrum distortion of the voice quality conversion result by the conventional method (Conventional VC (Voice Conversion)), and the solid line in FIG. 10 shows the cepstrum of the voice quality conversion result by the method of the present invention (EVC (Eigenvoice Conversion)). Shows distortion. As shown in FIG. 10, when the learning data used for adaptation is small, the cepstrum distortion of the voice quality conversion result by the EV-GMMλ ^EV of the present invention is significantly lower than that of the conventional method. This is because EV-GMMλ ^EV can obtain a higher-accuracy post-adaptation voice quality conversion model by utilizing the information of the output speaker for pre-learning.

In the conventional method, the cepstrum distortion is greatly reduced as the amount of learning data used for adaptation increases. This is because it becomes possible to accurately model the joint probability density with a larger mixture distribution. On the other hand, in EV-GMMλ ^EV , the decrease in cepstrum distortion seen when learning data is further increased from two sentences is small.
This is because the number of dimensions of the weight vector is a free parameter of the ^EV-GMMλ EV is constant, on the contrary, it means that the weight vector if there are about two sentences training data can be sufficiently estimated. In the case of using the learning data of several tens of sentences, ^EV-GMMλ EV is inferior in conversion accuracy with conventional methods point, there is a great advantage of allowing unsupervised learning.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to the drawings. FIGS. 11-16 is a figure which shows 3rd Embodiment of the voice quality conversion client server system based on this invention.
First, the configuration of a voice quality conversion client / server system according to the present invention will be described with reference to FIG. FIG. 11 is a block diagram showing a schematic configuration of the voice quality conversion client server system 3 according to the third embodiment of the present invention.

  As shown in FIG. 11, the voice quality conversion client server system 3 uses a server computer 30 that transmits a voice quality conversion model in response to an acquisition request from the client computer 31 and a voice quality conversion using the voice quality conversion model received from the server computer 30. And a network 32 that connects the server computer 30 and the client computer 31 so as to be capable of data communication with each other. In FIG. 11, three client computers 31 are connected to the network 32. However, the present invention is not limited to this configuration, and the number of connected client computers 31 may be less than three. There may be more than three.

Next, a detailed configuration of the server computer 30 will be described with reference to FIG. Here, FIG. 12 is a block diagram showing a detailed configuration of the server computer 30. Note that the same components as those in the voice quality conversion system 2 in the second embodiment are denoted by the same reference numerals and description thereof is omitted.
As illustrated in FIG. 12, the server computer 30 includes a data communication unit 30 a, a voice quality conversion model transmission unit 30 b, and a voice quality conversion model storage unit 20.

The data communication unit 30a receives a voice quality conversion model acquisition request from the client computer 31 via the network 32, or acquires a voice quality conversion model via the network 32 according to a command from the voice quality conversion model transmission unit 30b. A function of transmitting to the requesting client computer 31 is provided.
In response to the acquisition request from the client computer 31 received by the data communication unit 30a, the voice quality conversion model transmission unit 30b reads the voice quality conversion model indicated by the acquisition request from the voice quality conversion model storage unit 20, and reads the voice quality conversion thus read out A function of transmitting the model to the client computer 31 via the data communication unit 30a is provided.

  Here, the server computer 30 includes a processor (not shown), a RAM (Random Access Memory), and a ROM (Read Only Memory) in which a dedicated program is stored, and the processor executes the dedicated program. Fulfills the functions of the above-mentioned parts. In addition, the above-described units include those that perform their functions only with dedicated programs, and those that perform their functions by controlling hardware using dedicated programs.

Further, a detailed configuration of the client computer 31 will be described with reference to FIG. Here, FIG. 13 is a block diagram showing a detailed configuration of the client computer 31. Note that the same components as those in the voice quality conversion system 2 in the second embodiment are denoted by the same reference numerals and description thereof is omitted.
As shown in FIG. 13, the client computer 31 includes a data communication unit 31a, a voice quality conversion model reception unit 31b, a target speaker voice data acquisition unit 21, a former speaker voice data acquisition unit 22, and a voice quality conversion model adaptation. The configuration includes a unit 23, a voice quality conversion unit 24, and a speaker control parameter designation unit 25. In other words, the data communication unit 31a and the voice quality conversion model receiving unit 31b are added to the voice quality conversion system 2 in the second embodiment, and the voice quality conversion model storage unit 20 is further removed.

The data communication unit 31a transmits a voice quality conversion model acquisition request to the server computer 30 via the network 32 in response to a command from the voice quality conversion model reception unit 31b, or transmits the voice quality conversion model from the server computer 30 to the network. 32 is provided through a function of receiving data via the terminal 32.
The voice quality conversion model transmission unit 30b sends a request for acquiring a voice quality conversion model having a designated configuration to the server computer 30 via the data communication unit 30a in response to a voice quality conversion start instruction via an operation unit (not shown) from the user. The voice quality conversion model from the server computer 30 received by the data communication unit 31a is transmitted to the model acquisition unit 23a.

Here, similarly to the voice quality conversion system 2 in the second embodiment, the client computer 31 is configured to be able to set the normal mode and the parameter control mode as the voice quality conversion mode.
The client computer 31 includes a processor (not shown), a RAM (Random Access Memory), and a ROM (Read Only Memory) in which a dedicated program is stored, and the processor executes the dedicated program. It fulfills the functions of the above parts. In addition, the above-described units include those that perform their functions only with dedicated programs, and those that perform their functions by controlling hardware using dedicated programs.

Furthermore, the flow of the voice quality conversion model transmission process which is one of the operation processes of the server computer 30 will be described with reference to FIG. Here, FIG. 14 is a flowchart showing the voice quality conversion model transmission processing of the server computer 30.
As shown in FIG. 14, the voice quality conversion model transmission process first proceeds to step S600, and the voice quality conversion model transmission unit 30b receives a voice quality conversion model acquisition request from the client computer 31 via the data communication unit 30a. If it is determined that it has been received (Yes), the process proceeds to step S602. If not (No), the determination process is repeated until reception.

  When the process proceeds to step S602, the voice quality conversion model transmission unit 30b reads the voice quality conversion model having the configuration indicated by the acquisition request acquired in step S600 from the voice quality conversion model storage unit 20, and proceeds to step S604. Here, the acquisition request includes information (such as an IP address) of the client computer that is the acquisition request, information that specifies the configuration of the voice quality conversion model, and the like.

In step S604, the voice quality conversion model read in step S602 is transmitted to the acquisition request source client computer 31 via the data communication unit 30a, and the process proceeds to step S600.
Further, the flow of voice quality conversion processing in the normal mode in the client computer 31 will be described with reference to FIG. Here, FIG. 15 is a flowchart showing voice quality conversion processing in the normal mode in the client computer 31.

  As shown in FIG. 15, the voice quality conversion process in the normal mode first proceeds to step S700, where the voice quality conversion model reception unit 31b receives a voice quality conversion start instruction (start instruction information from the user via an operation unit not shown). (Yes), the process proceeds to step S702. If not (No), the determination process is repeated until an instruction is received.

  When the process proceeds to step S702, the voice quality conversion model receiving unit 31b generates a request for acquiring a voice quality conversion model having the configuration specified by the start instruction information, and the server computer 30 transmits the acquisition request via the data communication unit 30a. And the process proceeds to step S704. Here, the client computer 31 has information such as the IP address of the server computer 30 in advance.

In step S704, the voice quality conversion model reception unit 31b determines whether or not the voice quality conversion model is received from the server computer 30, and if it is determined that the voice quality conversion model has been received (Yes), the received voice quality conversion model is acquired as a model. The data is transmitted to the unit 23a and the process proceeds to step S706. If not (No), the determination process is repeated until reception.
In step S706, the voice quality conversion model adaptation unit 23 determines whether or not the configuration of the voice quality conversion model specified by the start instruction information is an N-to-1 configuration or a configuration in which an intermediate speaker is adapted. If it is determined that the configuration is (Yes), the process proceeds to step S708; otherwise (No), the process proceeds to step S728.

When the process proceeds to step S708, the adaptation target extracting unit 23b extracts the model part on the original speaker side from the N-to-1 voice quality conversion model or the N-to-1 intermediate voice quality conversion model received in step S704, and the extraction is performed. The model part on the former speaker side is output to the adaptation unit 23d, and the process proceeds to step S710.
In step S710, the voice quality conversion model adaptation unit 23 requests the user to acquire adaptation original speaker voice data, and the process proceeds to step S712. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S712, the original speaker voice data acquisition unit 22 determines whether or not the voice data of the original speaker for adaptation has been acquired (or generated from the input voice), and if it is determined that the voice data has been acquired (generated) (Yes) outputs the acquired (generated) original voice data for adaptation to the feature amount extraction unit 23c, and proceeds to step S714.
In step S714, the feature amount extraction unit 23c extracts feature amounts from the adaptation original speaker voice data input from the original speaker voice data acquisition unit 22, and the extracted former speaker feature amounts are applied to the adaptation unit 23d. And the process proceeds to step S716.

In step S716, the adaptation unit 23d estimates a parameter value of the model unit on the former speaker side using a predetermined adaptation method based on the former speaker feature amount input from the feature amount extraction unit 23c, and then proceeds to step S718. Transition.
In step S718, the adaptation unit 23d generates a post-adaptation original speaker voice model based on the parameter value estimated in step S716, and outputs the generated post-adaptation voice quality conversion model to the voice quality conversion unit 24, and then proceeds to step S720. Transition.

In step S720, the voice quality conversion unit 24 requests the user to acquire original speaker voice data for voice quality conversion, and the process proceeds to step S722. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S722, the original speaker voice data acquisition unit 22 determines whether or not the voice data of the original speaker for voice quality conversion has been acquired (or generated from the input voice), and is determined to have been acquired (generated). If yes (Yes), the acquired (generated) original speaker voice data for voice quality conversion is output to the voice quality converting unit 24 and the process proceeds to step S724. If not (No), until the acquired (generated) voice data is converted. The determination process is repeated (or the generation process is performed).

When the process proceeds to step S724, the voice quality conversion unit 24 converts the voice data of the voice quality of the original speaker that is the voice quality conversion target input from the voice data conversion unit 22 into voice data of the target speaker's voice quality. Then, the process proceeds to step S726.
In step S726, the voice quality conversion unit 24 outputs the converted voice based on the voice data after the voice quality conversion, and ends the process.

  On the other hand, if the configuration of the voice quality conversion model is not an N-to-1 configuration or a configuration in which an intermediate speaker is applied in step S706, and the process proceeds to step S728, the voice quality conversion model adaptation unit 23 specifies the start instruction information. It is determined whether the configuration of the voice quality conversion model is a 1-to-M configuration. If it is determined that the configuration is a 1-to-M configuration (Yes), the process proceeds to step S730; otherwise (No). The process proceeds to step S740.

When the process proceeds to step S730, the adaptation target extraction unit 23b extracts the model unit on the target speaker side from the one-to-M voice quality conversion model received in step S704, and extracts the model unit on the target speaker side thus extracted. Is output to the adaptation unit 23d, and the process proceeds to step S732.
In step S732, the voice quality conversion model adaptation unit 23 requests the user to acquire adaptation target speaker voice data, and the process proceeds to step S734. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S734, the target speaker voice data acquisition unit 21 determines whether or not the voice data of the target speaker for adaptation has been acquired (or generated from the input voice). (Yes) outputs the acquired (generated) target speaker voice data for adaptation to the feature amount extraction unit 23c, and proceeds to step S736.
In step S736, the feature amount extraction unit 23c extracts feature amounts from the target speaker voice data for adaptation input from the target speaker voice data acquisition unit 21, and uses the extracted target speaker feature amounts as the adaptation unit 23d. And the process proceeds to step S738.

In step S738, the adaptation unit 23d estimates a parameter value of the model unit on the target speaker side using a predetermined adaptation method based on the target speaker feature amount input from the feature amount extraction unit 23c, and then proceeds to step S718. Transition.
In step S728, if the configuration of the voice quality conversion model is not 1 to M and the process proceeds to step S740, the adaptation target extraction unit 23b uses the N to M voice quality conversion model acquired in step S706 to obtain the original speaker. Both the model unit on the side and the model unit on the target speaker side are extracted, and the extracted model unit on the original speaker side and the model unit on the target speaker side are output to the adaptation unit 23d, and the process proceeds to step S742.

In step S742, the voice quality conversion model adaptation unit 23 requests the user to acquire the original speaker voice data and target speaker voice data for adaptation, and the process proceeds to step S744. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S744, whether or not the original speaker voice data acquisition unit 22 and the target speaker voice data acquisition unit 21 have acquired (or generated from input voice) voice data of the original speaker and target speaker for adaptation. When it is determined that each has been acquired and acquired (generated) (Yes), the acquired (generated) adaptation original voice data and target speaker audio data are output to the feature amount extraction unit 23c. The process moves to step S746.

In step S746, the feature amount extraction unit 23c uses the original speaker voice data for adaptation input from the source speaker voice data acquisition unit 22 and the target speaker for adaptation input from the target speaker voice data acquisition unit 21. A feature amount is extracted from the speech data, and the extracted original speaker feature amount and target speaker feature amount are output to the adaptation unit 23d, and the process proceeds to step S748.
In step S748, the adaptation unit 23d estimates the parameter value of the model unit on the former speaker side using a predetermined adaptation method based on the former speaker feature amount input from the feature amount extraction unit 23c, and the feature amount. Based on the target speaker feature input from the extraction unit 23c, the parameter values of the model unit on the target speaker side are estimated using a predetermined adaptation method, and the process proceeds to step S718.

Furthermore, the flow of voice quality conversion processing in the parameter control mode in the client computer 31 will be described with reference to FIG. Here, FIG. 16 is a flowchart showing voice quality conversion processing in the parameter control mode in the client computer 31.
As shown in FIG. 16, the voice quality conversion process in the parameter control mode first proceeds to step S800, and the voice quality conversion model reception unit 31b has received a voice quality conversion start instruction from the user via an operation unit (not shown). If it is determined (Yes), the process proceeds to step S802. If not (No), the determination process is repeated until an instruction is given.

  When the process proceeds to step S802, the voice quality conversion model receiving unit 31b generates a voice quality conversion model acquisition request having the configuration specified by the start instruction information, and the server computer 30 transmits the acquisition request via the data communication unit 30a. And the process proceeds to step S804. Here, the client computer 31 has information such as the IP address of the server computer 30 in advance.

In step S804, the voice quality conversion model reception unit 31b determines whether or not the voice quality conversion model is received from the server computer 30. If it is determined that the voice quality conversion model is received (Yes), the received voice quality conversion model is acquired as a model. The data is transmitted to the unit 23a and the process proceeds to step S806. If not (No), the determination process is repeated until reception.
In step S806, the voice conversion model adaptation unit 23 determines whether or not the configuration of the voice conversion model specified by the start instruction information is a one-to-M configuration, and if it is determined to be one-to-M (Yes) ) Goes to step S808, otherwise (No), goes to step S824.

When the process proceeds to step S808, the adaptation target extraction unit 23b extracts the model part on the target speaker side from the one-to-M voice quality conversion model received in step S804, and extracts the model part on the target speaker side thus extracted. Is output to the adaptation unit 23d, and the process proceeds to step S810.
In step S810, the voice quality conversion model adaptation unit 23 requests the user to specify the speaker control parameter value of the target speaker side model for adaptation, and the process proceeds to step S812. Here, the designation request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S812, the speaker control parameter specifying unit 25 determines whether or not the speaker control parameter is specified via an operation unit (not shown). If it is determined that the speaker control parameter is specified (Yes), The designated control parameter value is output to the adaptation unit 23d, and the process proceeds to step S814. If not (No), the determination process is repeated until designated.
When the process proceeds to step S814, the adaptation unit 23d performs adaptation after adapting to the speech of the target speaker expressed by the parameter value based on the speaker control parameter value input from the speaker control parameter designating unit 25. A voice quality conversion model is generated, the generated post-adaptation voice quality conversion model is output to the voice quality conversion unit 24, and the process proceeds to step S816.

In step S816, the voice quality conversion unit 24 requests the user to acquire original speaker voice data for voice quality conversion, and the process proceeds to step S818. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S818, the original speaker voice data acquisition unit 22 determines whether or not the voice data of the original speaker for voice quality conversion has been acquired (or generated from the input voice), and is determined to have been acquired (generated). If yes (Yes), the acquired (generated) original speaker voice data for voice quality conversion is output to the voice quality converting unit 24 and the process proceeds to step S820. If not (No), until acquired (generated). The determination process is repeated (or the generation process is performed).

When the process proceeds to step S820, the voice quality conversion unit 24 converts the voice data of the voice quality of the target speaker to be converted, which is input from the voice data acquisition unit 22, into voice data of the target speaker's voice quality. Then, the process proceeds to step S822.
In step S822, the voice quality conversion unit 24 outputs the converted voice based on the voice data after the voice quality conversion, and ends the process.

  On the other hand, in step S806, when the configuration of the acquired voice quality conversion model is not 1 to M and the process proceeds to step S824, the adaptation target extraction unit 23b uses the original N to M voice quality conversion model received in step S804. Both the model part on the speaker side and the model part on the target speaker side are extracted, and the extracted model part on the original speaker side and the model part on the target speaker side are output to the adaptation unit 23d, and the process proceeds to step S826. To do.

In step S826, the voice quality conversion model adaptation unit 23 requests the user to acquire the original speaker voice data for adaptation, and the process proceeds to step S828. Here, the acquisition request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).
In step S828, the original speaker voice data acquisition unit 22 determines whether or not the voice data of the original speaker for adaptation has been acquired (or generated from the input voice), and if it is determined that the voice data has been acquired (generated) (Yes) outputs the acquired (generated) original voice data for adaptation to the feature amount extraction unit 23c, and proceeds to step S830.

In step S830, the feature amount extraction unit 23c extracts feature amounts from the adaptation original speaker voice data input from the original speaker voice data acquisition unit 22, and the extracted original speaker feature amounts are applied to the adaptation unit 23d. And the process proceeds to step S832.
In step S832, the voice quality conversion model adaptation unit 23 requests the user to specify the speaker control parameter value of the target speaker model for adaptation, and the process proceeds to step S834. Here, the designation request is made by displaying a message on a display unit (not shown) or outputting a voice message from a voice output unit (not shown).

In step S834, the speaker control parameter specifying unit 25 determines whether or not a speaker control parameter has been specified via an operation unit (not shown). If it is determined that the speaker control parameter has been specified (Yes). The designated control parameter value is output to the adaptation unit 23d, and the process proceeds to step S836. If not (No), the determination process is repeated until designated.
In step S836, the adaptation unit 23d estimates a parameter value of the model unit on the former speaker side using a predetermined adaptation method based on the former speaker feature amount input from the feature amount extraction unit 23c, and then proceeds to step S838. Transition.
In step S838, the adaptation unit 23d generates a post-adaptation voice quality conversion model based on the parameter value estimated in step S836 and the speaker control parameter value specified in step S834, and the generated post-adaptation voice quality The conversion model is output to the voice quality conversion unit 24, and the process proceeds to step S816.

Next, the operation of this embodiment will be described.
First, the voice quality conversion process in the normal mode will be described.
First, when a voice quality conversion start instruction (including start instruction information) is input by a user's operation of an operation unit (not shown) by the user (“Yes” branch of step S700), the client computer 31 receives a voice quality conversion model reception unit 31b. , A request for obtaining a voice quality conversion model having a configuration designated by the start instruction information is generated based on the start instruction information and the information of the server computer 30. Then, the generated acquisition request is transmitted to the server computer 30 via the data communication unit 31a (step S702).

  On the other hand, when the server computer 30 receives a voice quality conversion model acquisition request from the client computer 31 via the data communication unit 30a in the voice quality conversion model transmission unit 30b ("Yes" branch in step S600), the server computer 30 receives the received request. The voice quality conversion model corresponding to the acquired acquisition request is read from the voice quality conversion model storage unit 20 (step S602). Then, the read voice quality conversion model is transmitted to the client computer 31 that is the acquisition request source via the data communication unit 30a (step S604).

When the client computer 31 receives the voice quality conversion model from the server computer 30 (“Yes” branch of step S704), the client computer 31 transmits the received voice quality conversion model to the model acquisition unit 23a. The model acquisition unit 23a outputs the voice quality conversion model from the voice quality conversion model reception unit 31b to the adaptation target extraction unit 23b.
The series of operation processes is the same in the parameter control mode.
Further, since the subsequent voice quality conversion model adaptation processing and voice quality conversion processing are the same as those of the voice quality conversion system 2 in the second embodiment, description thereof will be omitted.

  As described above, according to the voice quality conversion client server system 3 of the present embodiment, N-to-one generated using at least one of the voice data of N former speakers and the voice data of M target speakers. The 1-to-M and N-to-M voice quality conversion models can be used for voice quality conversion by adapting to the speech of a desired (arbitrary) original speaker or a desired (arbitrary) target speaker. As a result, the voice data of an arbitrary utterance content of a specific original speaker can be easily converted into voice data of the desired target speaker's voice quality, or the voice data of an arbitrary utterance content of a desired original speaker can be converted. It can be easily converted into voice data of a voice quality of a specific target speaker or a desired target speaker.

  In addition, N-to-one, one-to-M, and N-to-M voice quality generated using eigenvoice technology and using at least one of voice data of N original speakers and voice data of M target speakers The conversion model can be used for voice conversion by adapting the voice of a desired (arbitrary) original speaker or a desired (arbitrary) target speaker. As a result, the voice quality conversion model can be easily applied to the voice of the desired speaker by the EM algorithm using a small number of voice data (which may be non-parallel) of at least one of the desired original speaker and the desired target speaker. Can be made.

  In addition, a one-to-M or N-to-M voice quality conversion model generated using eigenvoice technology and using at least one of voice data of N original speakers and voice data of M target speakers The user can specify an arbitrary value for the value of the weight vector (speaker control parameter) within a predetermined range. As a result, even if the voice quality of the converted voice is finely adjusted or the target speaker's voice data cannot be obtained, the voice control parameter value is manually controlled to obtain a converted voice having a desired voice quality or a voice quality close thereto. Can be. In addition, if other speech parameters such as F0 are controlled simultaneously, the super vector space may be configured by an average vector of all speech parameters desired to be controlled, not only the spectrum.

  Further, the voice quality conversion model can be held by the server computer 30 and the voice quality conversion model can be transmitted to the client computer 31 in response to an acquisition request from the client computer 31. Thereby, since it is not necessary to hold the voice quality conversion model on the client computer 31 side, a memory for holding it can be made unnecessary, so that the use efficiency of the memory can be improved.

Also, since the voice quality conversion model can be held and managed on the server computer 30 side, it is possible to easily upgrade the voice quality conversion model, and to convert the voice quality conversion model after the upgrade. It can be provided to the client computer 31 easily.
In the third embodiment, the voice quality conversion model transmission processing by the data communication unit 30a and the voice quality conversion model transmission unit 30b corresponds to the voice quality conversion model transmission unit according to claim 8 , and the voice quality conversion model storage unit 20 includes: , corresponding to the voice conversion model storage means according to claim 8.

Further, in the above-mentioned third embodiment, source-speaker speech data acquiring unit 22 corresponds to the source-speaker speech data acquisition means according to claim 8, voice conversion model adaptation unit 23 according to claim 8 Corresponding to the adaptation means, the voice quality conversion unit 24 corresponds to the voice quality conversion means according to the eighth aspect .

  In the third embodiment, the client computer 31 obtains a voice quality conversion model from the server computer 30, and adapts the acquired voice quality conversion model and uses the voice quality conversion model after adaptation. Although the configuration is such that the change process is performed, the present invention is not limited to this, and other configurations such as a configuration in which the server computer 30 performs an adaptive process of the voice quality conversion model or performs a voice quality conversion process in addition to the adaptive process may be employed.

  For example, in the case where the server computer 30 is configured to perform adaptive processing and voice quality conversion processing, the client computer 31 uses the target speaker voice data acquisition unit 21 and the original speaker voice data acquisition unit 22 to perform the target speaker. It is only necessary to acquire the voice data and the voice data of the former speaker and transmit them to the server computer 30, and it becomes unnecessary to acquire the voice quality conversion model from the server computer 30. This eliminates the need to perform processing using the voice quality conversion model on the client computer 31 side, so that it is possible to perform voice quality conversion processing even when the client computer 31 is not rich in memory resources, such as a portable device. Become.

It is a block diagram which shows the structure of the voice quality conversion model production | generation apparatus 1 which concerns on the 1st Embodiment of this invention. 3 is a block diagram showing a detailed configuration of a voice quality conversion model generation unit 12. FIG. It is a flowchart which shows the voice quality conversion model production | generation process in the voice quality conversion model production | generation apparatus 1. FIG. It is a flowchart which shows the voice quality conversion model production | generation process by the normal learning in the voice quality conversion model production | generation part 12. FIG. It is a flowchart which shows the voice quality conversion model production | generation process by the adaptive learning in the voice quality conversion model production | generation part. It is a block diagram which shows the structure of the voice quality conversion system 2 which concerns on the 2nd Embodiment of this invention. 4 is a block diagram showing a detailed configuration of a voice quality conversion model adaptation unit 23. FIG. It is a flowchart which shows the voice quality conversion process at the time of the normal mode in the voice quality conversion system. It is a flowchart which shows the voice quality conversion process at the time of the parameter control mode in the voice quality conversion system. It is a figure which shows the mel cepstrum distortion between the conversion audio | voice in the evaluation data of the method of this invention, and a conventional method, and the natural voice of an output speaker. It is a block diagram which shows schematic structure of the voice quality conversion client server system 3 which concerns on the 3rd Embodiment of this invention. 2 is a block diagram showing a detailed configuration of a server computer 30. FIG. 2 is a block diagram showing a detailed configuration of a client computer 31. FIG. It is a flowchart which shows the voice quality conversion model transmission process of the server computer. 4 is a flowchart showing voice quality conversion processing in a normal mode in a client computer 31. 4 is a flowchart showing voice quality conversion processing in a parameter control mode in a client computer 31.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Voice quality conversion model production | generation apparatus 2 Voice quality conversion system 3 Voice quality conversion client server system 10 Original speaker audio | voice data storage part 11 Target speaker audio | voice data storage part 12 Voice quality conversion model production | generation part 13 Voice quality conversion model storage part 12a, 23c Feature-value extraction Unit 12b normal learning unit 12c specific speaker model generation unit 12d adaptive learning unit 20 voice quality conversion model storage unit 21 target speaker voice data acquisition unit 22 former speaker voice data acquisition unit 23 voice quality conversion model adaptation unit 24 voice quality conversion unit 25 Humanity control parameter designation unit 23a Model acquisition unit 23b Adaptation target extraction unit 23d Adaptation unit 30a, 31a Data communication unit 30b Voice quality conversion model transmission unit 31b Voice quality conversion model reception unit

Claims (8)

A voice quality conversion model generation device for generating a statistical model for voice quality conversion,
First voice data which is voice data of a predetermined N (N is an integer of 1 or more) people, and predetermined M (M is an integer of 1 or more) of the same utterance content as the first voice data initial voice conversion model generation for generating initial voice conversion model is a common statistical model based on the speaker and the M's target speaker of the N people using the second audio data is audio data of the target speaker and means,
Converting the voice quality of each of the N original speakers into the voice quality of each of the M target speakers from the voice data of the N original speakers and the voice data of the M target speakers; Specific speaker model creating means for creating a specific speaker model corresponding to each one of the speakers and each of the target speakers;
Applying the result obtained by using the specific speaker model created by the specific speaker model creating means to the eigenvoice technique to parameters constituting the initial voice quality conversion model generated by the initial voice quality conversion model generating means, A speaker control parameter, which is a weight vector for converting any of the N original speakers to any of the M target speakers, is added to the parameters constituting the initial voice quality conversion model. A voice quality conversion model generating means for generating a voice quality conversion model having the speaker control parameter,
A voice quality conversion model generation apparatus, wherein at least one of N and M is an integer of 2 or more. A voice quality conversion model generation device for generating a statistical model for voice quality conversion,
First voice data that is voice data of a predetermined N (N is an integer of 2 or more) former speakers, and voice data of a predetermined intermediate speaker having the same utterance content as the first voice data Learning with the third voice data as learning data, and converting the voice of the voice quality of the N former speakers into the voice of the one intermediate speaker and the 1 former speaker and the 1 A first voice quality conversion model, which is one statistical model common to a human intermediate speaker, is generated, and a third voice data of the one intermediate speaker and a predetermined utterance content same as that of the third voice data are generated. Learning is performed by using the second voice data, which is voice data of M (M is an integer of 2 or more) target speakers, as learning data, and the voice of the one intermediate speaker is used as the target speaker's voice. One-to-one for each of the one intermediate speaker and the target speaker for conversion to voice-quality speech Is a response to the statistical model, voice conversion model generating device characterized by comprising a voice conversion model generating means for generating M second voice conversion model. A voice quality conversion system for converting the voice of any of the original speaker to the voice of the other voice,
Voice quality conversion model storage means for storing a voice quality conversion model having a speaker control parameter generated by the voice quality conversion model generation device according to claim 1 ;
Former speaker voice data acquisition means for acquiring voice data of any former speaker;
Target speaker voice data acquisition means for acquiring voice data of an arbitrary target speaker;
A speaker control parameter value specifying means for specifying a speaker control parameter value related to the voice of the voice quality of the M target speakers in the voice conversion model ;
Based on the audio data obtained in the previous Kimoto speaker speech data acquisition means, and the parameter values specified in the speaker information control parameter value specification unit, a voice conversion model stored in the voice conversion model storing means, Second adaptation means for creating a new voice quality conversion model by adapting the voice quality conversion model to the voice of the voice quality of the arbitrary original speaker and the specified parameter value using a predetermined adaptation method;
Based on the new voice quality conversion model created by the second adapting means and the voice data acquired by the original speaker voice data acquiring means, the voice data of the arbitrary former speaker is converted into voice data of another voice quality. A voice quality conversion system. A voice quality conversion system that converts a voice of an arbitrary utterance content of an arbitrary former speaker into a voice of a predetermined target voice quality of M (M is an integer of 2 or more),
A first voice of the voice quality of a predetermined N (N is an integer of 2 or more) original speakers, which is generated by the voice quality conversion model generation device according to claim 2, is converted into a voice of one intermediate speaker. Voice quality conversion model and voice quality conversion model storage means for storing a second voice quality conversion model for converting the voice of the one intermediate speaker into the voice of the voice quality of the M target speakers;
Former speaker voice data acquisition means for acquiring voice data of the arbitrary former speaker;
Based on the voice data acquired by the former speaker voice data acquisition means and the first voice quality conversion model stored in the voice quality conversion model storage means, the first voice quality conversion model is converted into the arbitrary voice using a predetermined adaptive method. Adaptation means to adapt to the voice quality of the former speaker,
Using the adapted first voice quality conversion model, the voice data of the arbitrary utterance content of the arbitrary original speaker acquired by the original speaker voice data acquisition means is converted into the voice data of the voice quality of the intermediate speaker And using the second voice quality conversion model stored in the voice quality conversion model storage means, the voice quality conversion means for converting the voice data converted into the voice quality of the intermediate speaker into the voice data of the arbitrary target speaker. A voice quality conversion system comprising: The adapting means converts the first voice conversion model into the voice of the voice quality of any of the original speakers with respect to the parameters related to the voice quality of the N original speakers constituting the first voice conversion model. An adaptive parameter value to be adapted is estimated using the predetermined adaptation method, and a parameter value related to the voice of the voice quality of the N former speakers of the first voice quality conversion model is converted to the estimated adaptive parameter value. The voice quality conversion system according to claim 4 . A voice quality conversion model generation program for generating a statistical model for voice quality conversion,
First voice data which is voice data of a predetermined N (N is an integer of 1 or more) people, and predetermined M (M is an integer of 1 or more) of the same utterance content as the first voice data initial voice conversion model generation for generating an initial voice conversion model is a common statistical model based on the speaker and the M's target speaker of the N people using the second audio data is audio data of the target speaker Steps ,
Converting the voice quality of each of the N original speakers into the voice quality of each of the M target speakers from the voice data of the N original speakers and the voice data of the M target speakers; A specific speaker model creating step of creating a specific speaker model corresponding to each one of the speakers and each of the target speakers;
Applying the result obtained by using the specific speaker model created in the specific speaker model creation step for eigenvoice technology to the parameters constituting the initial voice quality conversion model generated in the initial voice quality conversion model generation step, A speaker control parameter, which is a weight vector for converting any of the N original speakers to any of the M target speakers, is added to the parameters constituting the initial voice quality conversion model. Including a program for causing a computer to execute a voice quality conversion model generation step of generating a voice quality conversion model having the speaker control parameter .
A voice quality conversion model generation program characterized in that at least one of N and M is an integer of 2 or more. A voice conversion model generation method for generating a statistical model for voice conversion,
First voice data which is voice data of a predetermined N (N is an integer of 1 or more) people, and predetermined M (M is an integer of 1 or more) of the same utterance content as the first voice data initial voice conversion model generation for generating an initial voice conversion model is a common statistical model based on the speaker and the M's target speaker of the N people using the second audio data is audio data of the target speaker and the step,
Converting the voice quality of each of the N original speakers into the voice quality of each of the M target speakers from the voice data of the N original speakers and the voice data of the M target speakers; A specific speaker model creating step of creating a specific speaker model corresponding to each one of the speakers and each of the target speakers;
Applying the result obtained by using the specific speaker model created in the specific speaker model creation step for eigenvoice technology to the parameters constituting the initial voice quality conversion model generated in the initial voice quality conversion model generation step, A speaker control parameter, which is a weight vector for converting any of the N original speakers to any of the M target speakers, is added to the parameters constituting the initial voice quality conversion model. A voice quality conversion model generating step for generating a voice quality conversion model having the speaker control parameter;
Including
A method for generating a voice quality conversion model, wherein at least one of N and M is an integer of 2 or more. And the client computer and the server computer are connected via a network, the voice-conversion-client-server system that converts speech of any source speaker to the speech of the other voice,
The server computer
And voice conversion model storage means for storing the voice quality conversion model generated by the voice conversion model generating device according to claim 1,
Voice quality conversion model transmission means for transmitting the voice quality conversion model stored in the voice quality conversion model storage means to the client computer;
The client computer is
Former speaker voice data acquisition means for acquiring voice data of the arbitrary former speaker;
A speaker control parameter value specifying means for specifying a parameter value of the speaker control parameter in the voice conversion model;
And voice conversion model receiving means for receiving the voice conversion model from the previous SL server computer,
Based on the voice data acquired by the former speaker voice data acquiring means, the parameter value specified by the speaker control parameter value specifying means, and the voice quality conversion model received by the voice quality conversion model receiving means, a predetermined Adapting means for adapting the voice quality conversion model of the arbitrary speaker's voice quality and voice to the designated parameter value by using an adaptation technique to create a new voice quality conversion model ;
Based on the new voice quality conversion model created by the adapting means and the voice data acquired by the original speaker voice data acquiring means, the voice quality conversion for converting the voice data of the arbitrary former speaker into voice data of another voice quality And a voice quality conversion client / server system comprising: means.

Images