JP2017003622A - Vocal quality conversion method and vocal quality conversion device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vocal quality conversion method with which it is possible to convert the vocal quality of an input speaker to the vocal quality of a discretionary speaker without using parallel data.SOLUTION: The vocal quality conversion method includes: a step S1 for estimating a weight not dependent on a speaker as an independent weight among coupled weights between a visible element layer and a hidden element layer constituting a Restricted Boltzmann Machine (RBM) that is a probability model; a step S2 for estimating, while the independent weight is fixed, each of a weight dependent on an input speaker and a weight dependent on a target speaker as a dependent weight among coupled weights; a step S3 for estimating a hidden element layer on the basis of the voice of an input speaker inputted to the visible element layer and the dependent weight of the input speaker; and a step S4 for estimating the voice of the target speaker outputted as the visible element layer on the basis of the hidden element layer and the dependent weight of the target speaker.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a method and apparatus for converting voice quality of speech of an input speaker into voice quality of a person other than the input speaker.

  In recent years, voice quality conversion technology has been actively researched in the field of audio signal processing. This voice quality conversion technique is a technique for converting only information related to speaker characteristics into information of an output speaker (that is, a target speaker) while preserving phonological information of an input speaker's voice. As the background, it is possible to improve the speech recognition accuracy of noisy environments and emotional speech, assist speech for persons with disabilities who are difficult to speak, and apply to various other tasks.

  Conventional voice quality conversion methods have been widely researched on approaches based on statistical methods. Among them, a technique using GMM (Gaussian Mixture Model) is most widely used, and various improvements have been made. As an approach other than GMM, a voice quality conversion method (see Non-Patent Document 1) using NMF (Non-negative matrix factorization) has recently been proposed and attracts attention as a method with less oversmoothness.

  A voice quality conversion technique using an RBM (Restricted Boltzmann Machine) composed of two layers of a visible layer and a hidden layer is also disclosed (see Non-Patent Document 2).

R. Takashima, T. Takiguchi and Y. Ariki: “Exemplar-based voice conversion in noisy environment”, SLT, pp. 313-317 (2012) Wataru Nakaga, Tetsuya Takiguchi, Yasuo Ariki “Voice conversion using speaker-dependent Recurrent Temporal Restricted Boltzmann Machine” Acoustical Society of Japan Proceedings (September 2012)

  However, the techniques described in Non-Patent Document 1 and Non-Patent Document 2 have a problem that parallel data is required between an input speaker and a specific output speaker (target speaker).

  That is, all of these techniques require parallel data (speech pairs with the same utterance content of the input speaker and output speaker) when learning the model, and various restrictions are imposed on the creation of parallel data. First, since there is a restriction that the utterance data of the input speaker and the output speaker must have the same utterance content, the degree of freedom of the learning data set that can be selected (or created) is low. Second, since it is necessary to synchronize both voices in units of frames, alignment is performed using dynamic programming or the like, but there is no guarantee that the frames are completely synchronized. Therefore, there is a problem that conversion is added to the voice when the alignment is expanded or contracted. Also, existing conversion models cannot be used for speaker pairs who are not learning. That is, it cannot be converted into the voice quality of an arbitrary speaker.

  Therefore, the present invention has been made in view of such problems, and provides a voice quality conversion method and apparatus capable of converting the voice quality of an input speaker into the voice quality of an arbitrary speaker without using parallel data. For the purpose.

  In order to achieve the above object, a voice quality conversion method according to the present invention is a voice quality conversion method for converting the voice quality of an input speaker's voice to the voice quality of a target speaker, and is a probability model RBM (Restricted Boltzmann Machine). A first step of estimating a speaker-independent weight as a non-dependent weight among the coupling weights between two different layer elements of the visible element layer and the hidden element layer that constitutes the non-independent weight; and In a fixed state, a second step of estimating a weight depending on the input speaker and a weight depending on the target speaker out of the coupling weights as a dependency weight, respectively, is input to the visible element layer Third step of estimating a hidden element layer based on the voice of the input speaker and the dependency weight of the input speaker, the hidden element layer, and the dependency of the target speaker And a fourth step of estimating the target speaker's voice output as the visible element layer based on the weight. For example, the joint weight is a vector composed of elements indicating 0 or 1 for each of S (S is an integer of 2 or more) speakers, the independent weight, and the dependent weight of each of the S speakers. It is expressed by the calculation using and.

  Thereby, an extended RBM is used. That is, the coupling weight between two different layer elements of the visible element layer and the hidden element layer constituting the RBM is a weight independent of the speaker (independent weight) and a weight dependent on the speaker (dependent weight). A separate adaptive RBM is used. Therefore, if the independent weight is fixed, the speaker property can be easily controlled using the dependent weight. As a result, the voice quality of the input speaker can be converted without using parallel data. Further, even if the voices of the input speaker and the target speaker are small, the dependency weights of the input speaker and the target speaker can be appropriately estimated in the second step. As a result, since the dependency weight of any target speaker can be easily estimated, if the independent weight estimated in the first step is used, the voice quality of the input speaker can be changed to the voice quality of any speaker. Can be converted.

  For example, in the first step, the independent weight is estimated based on voices of different utterance contents uttered by a plurality of speakers, and in the second step, the input speaker and the target story are estimated. The dependence weight of the input speaker and the dependence weight of the target speaker may be estimated on the basis of voices of different utterance contents uttered from each of the speakers.

  Thereby, since parallel data is not used in the first step and the second step, the voice quality can be appropriately converted without being restricted by the utterance content.

  Note that these comprehensive or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, and the system, method, integrated circuit, and computer program. And any combination of recording media.

  The voice quality conversion method of the present invention can convert the voice quality of an input speaker into the voice quality of an arbitrary speaker without using parallel data.

FIG. 1 is a diagram showing a graph structure of RBM. FIG. 2 is a diagram illustrating a graph structure of the adaptive RBM in the embodiment. FIG. 3 is a block diagram illustrating a configuration of the voice quality conversion apparatus according to the embodiment. FIG. 4 is a flowchart showing the processing operation of the voice quality conversion apparatus according to the embodiment. FIG. 5 is a diagram illustrating a result of voice quality conversion by the voice quality conversion method according to the embodiment. FIG. 6 is a diagram illustrating parameters actually estimated by the voice quality conversion method according to the embodiment. FIG. 7 is a diagram illustrating an example in which a female speaker's voice is converted into a male speaker's voice by the voice quality conversion method according to the embodiment.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  It should be noted that each of the embodiments described below shows a comprehensive or specific example. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements.

(Overview)
First, the outline of the voice quality conversion method in the present embodiment will be described.

  The voice quality conversion method according to the present embodiment uses an input speaker-output talk using an adaptive RBM (adaptive reduced Boltzmann machine; ARBM), which is an extended model of RBM (Restricted Boltzmann Machine), which is one of probabilistic models. This is a voice quality conversion method that requires not only parallel data between speakers but also parallel data between reference speakers.

  This adaptive RBM is a probabilistic model that extracts potential features while separating into speaker-independent information and speaker-dependent information from speech data in which a plurality of speakers are mixed. This model is represented by an undirected graph composed of a visible element layer and a hidden element layer, and there is no connection between the same layer elements, and there exists a connection between different layer elements with strength (weight) depending on the speaker. Furthermore, this weight is expressed by a speaker-dependent term and a speaker-independent term, and each of them is estimated simultaneously by unsupervised learning using speech data in which a plurality of speakers are mixed (not necessarily parallel). . As a result, latent features (hidden elements) can be obtained while separating into speaker-dependent weights and speaker-independent weights. When performing voice quality conversion to an arbitrary speaker, first, speaker-dependent weights and speaker-independent weights are simultaneously estimated using data of a plurality of speakers (reference speakers) as described above. Next, a new speaker-dependent weight is estimated while fixing the speaker-independent weight by using (a small amount) of data of the speaker (input speaker) to be converted. The speaker-dependent weight of the conversion destination speaker (output speaker) is similarly estimated. Then, latent features are estimated from the speech to be converted (the speech of the input speaker) using the speaker-dependent weight and speaker-independent weight of the input speaker, and then the speaker-dependent weight of the output speaker and the speaker The converted speech is obtained by inversely estimating the acoustic feature vector using the independent weight.

  Since many conventional voice quality conversion methods such as GMM and NMF are based on linear conversion, the conversion accuracy is limited. That is, since the human vocal tract shape is nonlinear, it is considered that nonlinear-based modeling is more appropriate than linear-based in order to more accurately capture the characteristics of voice quality included in the speech signal. The voice quality conversion method according to the present embodiment also uses a conversion formula based on a nonlinear function, and can perform voice quality conversion with high accuracy.

(RBM)
Next, the RBM that is the basis of the adaptive RBM in the present embodiment will be described.

  FIG. 1 is a diagram showing a graph structure of RBM.

The RBM is a two-layer network having a special structure, and is an undirected graphical model that represents a random variable distribution of a visible layer (visible element layer) and a hidden layer (hidden element layer) as shown in FIG. Originally, RBM was proposed as a model for inputting binary data, but a model (Gaussian-Bernoulli RBM; GBRBM) for inputting continuous values later was devised. However, since this model has a problem that learning becomes unstable due to the influence of the dispersion term, an improved version of GBRBM (Improved GBRBM; ImpGBBRBM) has been proposed. In this ImpGBBRBM, continuous-value visible elements
And binary hidden elements
Simultaneous probability of
Is expressed as follows.

here,
Represents the L2 norm, and the superscript represents element division.

Indicates the weight matrix between the visible layer and the hidden layer, the deviation of the visible element, the bias of the visible element, and the bias of the hidden element, all of which are parameters to be estimated.

In RBM, there is no connection between visible elements or hidden elements. That is, each visible element and hidden element are conditionally independent from each other. Therefore, each conditional probability
Is expressed by the following simple function.

here,
When
Is
Represents the j-th column vector and the i-th row vector. Also,
Is the elemental sigmoid function
Represents a normal distribution with mean and variance σ ² .

Each RBM parameter
N observation data
Log likelihood of this random variable
Is estimated to maximize. When this logarithmic likelihood is partially differentiated by each parameter,
Is obtained. However, <•> _data and <•> _model represent expected values of observation data and model data, respectively. However, since the latter expected value is generally difficult to calculate, the reconstructed data expected value <·> _recon obtained by the equations (4) and (5) is used _instead (CD: Contrastive Divergence method). In ImpGBBRBM, the variance is constrained to a non-negative value to stabilize learning.
Replace with Thus, the gradient for z _i is calculated as follows:

Each parameter is repeatedly updated from the equations (6), (7), and (8) using the stochastic gradient method (initial values are set at random). That is,
As shown, each parameter of the RBM is updated. Here, γ _θ represents a learning rate.

(Application to adaptive RBM and voice quality conversion)
In the present embodiment, an adaptive RBM (Adaptive restricted Boltzmann machine; ARBM) is defined as an extended model of the above RBM and applied to a voice quality conversion task.

(Definition of adaptive RBM)
FIG. 2 is a diagram illustrating a graph structure of the adaptive RBM.

As shown in FIG. 2, the adaptive RBM is not only a visible element and a hidden element seen in a normal RBM, but also an identification element.
It has become a model with (
Is the number of identification elements). For example, in voice quality conversion, input
Indicates that is the utterance of speaker k,
It becomes. In this model, there is an identification element between the visible element and the hidden element.
There is a connection weight controlled by. This joint weight
In the present embodiment, this is defined as follows.

However,
Are speaker-dependent weights, and an unspecified weight matrix
3rd-order tensor parameters for specifying (adapting)
It is. Also,
Is the third-floor tensor that expanded mode d
Matrices and vectors
Represents an operator that takes the inner product of. For voice conversion,
Is the connection weight by unspecified speakers, that is, speaker-independent weights,
Represents the adaptation matrix and bias matrix of speaker k (where
Is the 3rd floor tensor
Represents the k-th matrix of mode 3).

In the adaptive RBM, it is defined by equation (11)
Using the visible element
, Hidden element
, Identification element
Simultaneous probability of
Is defined as follows.

With these definitions, the conditional probability
Can be calculated as follows:

Adaptive RBM parameters
Is N pieces of learning data
Use the log likelihood
Is estimated to maximize. This log likelihood is
Elements of
Are the partial derivatives of
Can be calculated. Other parameters
Are obtained in the same manner as in equations (7), (9), and (8), respectively. Since the CD method can also be applied to the adaptive RBM, the parameters can be efficiently estimated by calculating the second term _{« model} of each partial differential value as the reconstructed value _{« recon} of the observation data. be able to.

(Voice quality conversion using adaptive RBM)
The voice quality conversion apparatus according to the present embodiment converts the voice quality of the input speaker's voice into the voice quality of an arbitrary output speaker (target speaker) using the above-described adaptive RBM.

  FIG. 3 is a block diagram showing the configuration of the voice quality conversion apparatus according to the present embodiment.

  The voice quality conversion apparatus 10 in the present embodiment is an apparatus that converts the voice quality of an input speaker's voice into the voice quality of a target speaker, and includes an independent weight estimation unit 11, a dependency weight estimation unit 12, and a hidden element layer. An estimation unit 13 and a visible element layer estimation unit 14 are provided.

As described above, the independent weight estimation unit 11 uses the N pieces of learning data and uses the adaptive RBM parameters.
Is estimated to maximize the log likelihood. That is, the non-dependent weight estimation unit 11 is a combination weight between two different layer elements of the visible element layer and the hidden element layer constituting the RBM.
Of which speaker-independent weights
Are estimated as speaker-independent weights. Here, as shown in the equation (11), the above-mentioned connection weights are a vector composed of elements indicating 0 or 1 for each of S (S is an integer of 2 or more) speakers, and speaker-independent weights. And a speaker-dependent weight of each of the S speakers. The N pieces of learning data may be voices of different utterance contents uttered by N speakers (reference speakers), that is, non-parallel data.

The dependency weight estimation unit 12 is a weight that depends on the input speaker among the connection weights in a state where the speaker-independent weight described above is fixed in the RBM.
And weight depending on the target speaker
Are estimated as speaker-dependent weights. Specifically, the dependence weight estimation unit 12 estimates the speaker dependence weight of the input speaker based on the voice of the input speaker, and estimates the speaker dependence weight of the target speaker based on the voice of the target speaker. To do. At this time, the dependency weight estimation unit 12 determines whether the input speaker and the target speaker are based on voices of different utterance contents uttered from the input speaker and the target speaker, that is, based on non-parallel data. Each speaker-dependent weight may be estimated.

The hidden element layer estimation unit 13 performs the hidden element layer based on the voice of the input speaker input to the visible element layer and the speaker-dependent weight of the input speaker described above.
Is estimated.

The visible element layer estimation unit 14 outputs the target speaker's voice output as the visible element layer based on the hidden element layer and the speaker-dependent weight of the target speaker.
Is estimated.

  FIG. 4 is a flowchart showing the processing operation of the voice quality conversion apparatus 10 in the present embodiment.

First, as shown in FIG. 4, the independent weight estimation unit 11 of the voice quality conversion apparatus 10 first uses each parameter (speech) of multiple (S persons) reference speakers to set each parameter of the adaptive RBM.
Are simultaneously estimated (step S1).

Next, the dependency weight estimation unit 12
Speaker-independent parameters (speaker-independent weights) are fixed, and the speaker-dependent weights of the input speaker and target speaker are used using adaptive data that is the speech of the input speaker and target speaker.
Is estimated from Equations (18) and (19) (step S2).

And the hidden element layer estimation part 13 is the frame acoustic feature-value of the audio | voice by which an input speaker is converted.
From this, the latent feature quantity (hidden element layer) is estimated as in the following equation (step S3).

However,
Is the first
A vector in which only the element is 1 and the other is 0 is assumed. At the same time, the variable
The length of
Extend to
Along mode 3
Shall be added. Rewriting equation (20),
, A term that does not depend on the speaker
Thus, the latent feature amount is estimated using the connection weight adapted to the input speaker. Also, equation (21) becomes once the learning of adaptive RBM is over
Is a variable
So that
Suggests that it is a latent feature that does not depend on the speaker. That is, speaker nature
Only controlled by
Is considered to represent information close to the phoneme independent of the speaker. Therefore, if you want to obtain speech with the speaker characteristics of the output speaker (target speaker), phoneme information
From
The acoustic feature value may be restored using That is, the visible element layer estimation unit 14 calculates the frame feature value of the conversion destination of the output speaker.
Is calculated as follows (step S4).

This is based on phonological information obtained from the speech of the input speaker, and generates the acoustic features of the output speaker using a basis that adapts speaker-independent weights to the output speaker (target speaker). It represents that. Also, as shown in equations (21) and (22), the acoustic features of the input speaker
Output speaker's acoustic features
When converting to
Therefore, it can be said that the voice quality conversion method in this embodiment is a voice quality conversion based on a nonlinear conversion.

In addition, when learning adaptive RBM using real speech data, there are cases where there are many speakers but there are few utterance data. in this case,
There are enough data to estimate the
Since the amount of data for estimating is small, it becomes a cause of erroneous estimation or overlearning. Therefore, in an evaluation experiment described later according to the present embodiment,
A diagonal matrix,
The number of parameters is suppressed by approximating with a matrix in which each column is equal.

(Evaluation experiment)
An evaluation experiment of the voice quality conversion method according to the present embodiment will be described in detail below with reference to FIGS.

(Experimental conditions)
In this experiment, the accuracy of the voice quality conversion method using the adaptive RBM according to the present embodiment was examined using TIMIT, which is a corpus including voices from a plurality of English-speaking speakers. TIMIT is described in detail in the document “JS Garofolo, LD Consortium, et al .:“ TIMIT: acoustic-phonetic continuous speech corpus ”, Linguistic Data Consortium (1993)”.

  From this corpus, 38 speakers (14 women and 24 men) were selected as reference speakers in order to estimate speaker-independent parameters (speaker-independent weights). From each speaker, utterance data of five sentences is used for learning (the total number of frames used for learning is approximately 270,000). In order to evaluate the voice quality conversion method in this embodiment, a pair of input speakers and output speakers (a total of 28 pairs) is created using the voices of four women and four men. The performance of voice quality conversion was compared. At this time, an evaluation by SDIR (spectral distortion improvement ratio) is performed using parallel data of input / output speakers (created by dynamic programming from two sentence data not included in the learning data based on the same utterance content). ing. As the acoustic feature amount, 32-dimensional MFCC (Mel-Frequency Cepstrum Coefficients) calculated from the STRAIGHT spectrum was used. Regarding the STRAIGHT spectrum, the literature “H. Kawahara, M. Morise, T. Takahashi, R. Nisimura, T. Irino and H. Banno:“ TANDEM-STRAIGHT: A temporally stable power spectral representation for periodic signals and applications to It is described in detail in "interference-free spectrum, F0, and aperiodicity estimation", ICASSP, pp. 3933-3936 (2008).

  The learning rate, batch size, and number of repetitions in the adaptive RBM were set to 0: 005, 50, and 500, respectively. Comparison was performed by changing the number of hidden elements to 128, 192, 256, and 512.

(Experimental results and discussion)
FIG. 5 is a diagram showing a result of voice quality conversion by the voice quality conversion method in the present embodiment. For example, in female-to-female, the voices of four females for evaluation are converted into the other three females, respectively, and the average of SDIR of all frames is taken. “Avg.” Is an average value of all combinations. From FIG. 5, it can be seen that the conversion accuracy improves if the number of hidden elements increases except for a part. Comparing the results of the number of hidden elements of 512 and 256, 512 is superior in conversion to male (female-to-male, male-to-male). However, the conversion to female (female-to-female, male-to-female) decreases the accuracy, and as a result, the total average SDIR value is lowered. This is probably because the model is overlearning as the number of parameters increases (the number of male and female speakers is 24:14, and in the model with 512 hidden elements, the converted speech is on the male side. The over-learning can be seen from the fact that they are responding strongly.)

FIG. 6 is a diagram showing parameters actually estimated by the voice quality conversion method according to the present embodiment. (A), (b) and (c) in FIG.
A part of

Is approximated as a diagonal matrix
Shows a matrix in which the diagonal components of are arranged for each speaker as a column vector,
Similarly, column vectors arranged for each speaker are shown. In FIGS. 6B and 6C, the left 14 column vector corresponds to a female speaker, and the right 24 column vector corresponds to a male speaker. As can be seen from FIG.
Each of the column vectors is a vector having a high similarity between the same sexes and a low similarity between the opposite sexes. This is consistent with the intuition that, when listening to speech and recognizing speaker differences, the difference in gender is felt greater than individual differences.

FIG. 7 is a diagram illustrating an example in which a female speaker's voice (FCJF0 in the corpus) is converted into a male speaker's voice (MWAR0) by the voice quality conversion method according to the present embodiment. In this example, the MFCC is calculated from the logarithmic spectrum at a certain time of FCJF0 (dotted line in FIG. 7A), and the adaptive RBM of FCJF0
After that, the acoustic feature value converted using the adaptive parameter of MWAR0 was restored to a logarithmic spectrum (solid line in FIG. 7B). As reference,
The spectrum restored by the adaptive parameter of FCJF0 after estimation (solid line in FIG. 7A) and the spectrum of target MWAR0 (dotted line in FIG. 7B) are shown. From FIG. 7, not only the spectrum reconstructed from the FCJF0 voice to the FCJF0 voice but also the voice spectrum converted to another speaker MWAR0, the spectrum peak in a band (low frequency) of less than about 3.5 kHz. It can be seen that the characteristics of the speaker are captured, such as the frequency (formant) of the speaker roughly matches the target. All of the bands (high frequency range) of about 3.5 kHz or more are greatly different from the target, but because the spectrum is restored from the MFCC, information in the high range is lost.

  Thus, in this embodiment, parallel data can be converted from FCJF0 to MWAR0 even though no parallel data is used during learning and a conversion model from FCJF0 to MWAR0 is not learned.

(Summary)
As described above, the voice quality conversion method according to the present embodiment includes steps S1 to S4 as shown in FIG. In step S1, a speaker-independent weight is estimated as an independent weight (the above-mentioned speaker-independent weight) among the coupling weights between two different layer elements of the visible element layer and the hidden element layer constituting the RBM. . In step S2, with the non-dependent weight fixed in the RBM, the weight depending on the input speaker and the weight depending on the target speaker among the combined weights are respectively determined as the dependency weights (the above-mentioned speaker dependent weights, or As an adaptation parameter). In step S3, the hidden element layer is estimated based on the voice of the input speaker input to the visible element layer and the dependency weight of the input speaker. In step S4, the target speaker's voice output as the visible element layer is estimated based on the hidden element layer and the dependency weight of the target speaker. Further, in the present embodiment, the connection weight is a vector composed of elements indicating 0 or 1 for each of S (S is an integer of 2 or more) speakers, an independent weight, and each story of S people. It is expressed by the calculation using the dependence weight of the person.

  Thus, in the present embodiment, an adaptive RBM in which the coupling weight between two different layer elements of the visible element layer and the hidden element layer constituting the RBM is separated into an independent weight and a dependent weight is used. Therefore, if the independent weight is fixed, the speaker property can be easily controlled using the dependent weight. As a result, the voice quality of the input speaker can be converted without using parallel data. Further, even if the voices of the input speaker and the target speaker are small, it is possible to appropriately estimate the dependency weights of the input speaker and the target speaker in step S2. As a result, since it is possible to easily estimate the dependency weight of any target speaker, the voice quality of the input speaker is converted to the voice quality of an arbitrary speaker if the non-dependent weight estimated in step S1 is used. be able to.

  In other words, in the present embodiment, the RBM that extracts the potential feature amount is expanded and modeled by separating the term depending on the speaker (dependent weight) and the term not dependent (independent weight). Voice quality conversion that does not require parallel data during learning and can be adapted to any speaker can be performed.

Note that the extended model of RBM (adaptive RBM) in this embodiment can be applied not only to voice quality conversion but also to various tasks such as voice emotion assignment and object recognition. In this model, the identification element
Can be applied to speaker recognition, for example. It is possible to control the speaker characteristics by separating each of the phonological information and the speaker information from the mixed speech.

  In the above embodiment, whether each component such as the independent weight estimation unit 11, the dependency weight estimation unit 12, the hidden element layer estimation unit 13, and the visible element layer estimation unit 14 is configured with dedicated hardware. It may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the voice quality conversion apparatus 10 and the like of each of the above embodiments is a program that causes a computer to execute each step included in the flowchart shown in FIG. Further, the voice quality conversion apparatus 10 in the above embodiment may be realized by a computer having a processor, a memory and an input / output port, or a logic circuit. Further, each hidden element in the above embodiment is, for example, 0 or 1, and is considered to indicate the presence or absence of a phoneme or phoneme in speech corresponding to the hidden element.

  Although the voice quality conversion method according to one or more aspects has been described based on the embodiment, the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to this embodiment, and the form constructed | assembled combining the component in different embodiment are also contained in the scope of the present invention. Also good.

  The voice quality conversion method according to the present invention has an effect that the voice quality of an input speaker can be converted into the voice quality of an arbitrary speaker without using parallel data. For example, a voice changer, a speech support device, an amusement device, etc. It can be applied to the voice quality conversion device.

DESCRIPTION OF SYMBOLS 10 Voice quality conversion apparatus 11 Independent weight estimation part 12 Dependence weight estimation part 13 Hidden element layer estimation part 14 Visible element layer estimation part

Claims (4)

A voice quality conversion method for converting voice quality of an input speaker's voice to voice quality of a target speaker,
First step of estimating a speaker-independent weight as an independent weight among coupling weights between two different layer elements of a visible element layer and a hidden element layer constituting a RBM (Restricted Boltzmann Machine) that is a probabilistic model When,
A second step of estimating a weight depending on the input speaker and a weight depending on the target speaker among the combined weights, with the independent weight fixed in the RBM; ,
A third step of estimating a hidden element layer based on the voice of the input speaker input to the visible element layer and the dependency weight of the input speaker;
A voice quality conversion method comprising: a fourth step of estimating speech of the target speaker output as a visible element layer based on the hidden element layer and the dependency weight of the target speaker. The joint weight is a vector composed of elements indicating 0 or 1 for each of S (S is an integer of 2 or more) speakers, the independent weight, and the dependent weight of each of the S speakers. The voice quality conversion method according to claim 1, which is represented by a calculation used. In the first step, the independent weight is estimated based on speech of different utterance contents uttered by a plurality of speakers,
In the second step, the dependency weight of the input speaker and the dependency weight of the target speaker are determined based on the speech of different utterance contents uttered from each of the input speaker and the target speaker. The voice quality conversion method according to claim 1 or 2, wherein the estimation is performed. A voice quality conversion device that converts the voice quality of an input speaker's voice to the voice quality of a target speaker,
Independent weight estimation that estimates the speaker-independent weight as the independent weight among the coupling weights between the two different layer elements of the visible element layer and the hidden element layer constituting the RBM (Restricted Boltzmann Machine) which is a probabilistic model And
A dependency weight estimating unit that estimates a weight depending on the input speaker and a weight depending on the target speaker among the connection weights, with the independent weight fixed in the RBM; ,
A hidden element estimation unit that estimates a hidden element layer based on the voice of the input speaker input to the visible element layer and the dependency weight of the input speaker;
A voice quality conversion device comprising: a hidden element layer; and a visible element layer estimation unit that estimates a voice of the target speaker output as a visible element layer based on the dependency weight of the target speaker.