JP2023171025A - Training device, training method, and training program - Google Patents

Abstract

To generate a synthetic speech of a discretionary utterance skill of a discretionary speaker without changing speaker individuality.SOLUTION: A training device that trains a speech synthesizing model acquires the utterance skill of a speaker from speech data of a plurality of speakers. Then, the training device generates speech data and utterance information (pseudo-speech data and pseudo-utterance information), in which the utterance skill is lowered for each speaker, from the speech data of and utterance information on the plurality of speakers. Then, using training data that includes the generated pseudo-speech data and pseudo-utterance information, the training device trains the speech synthesizing model that outputs for inputted text the speech feature quantity of a synthesized speech of a designated speaker and utterance skill.SELECTED DRAWING: Figure 6

Description

The present invention relates to a learning device, a learning method, and a learning program for learning a speech synthesis model.

Conventionally, in the field of speech synthesis, a technology has been proposed that performs speech synthesis using a deep neural network (DNN) (see Non-Patent Document 1). In this speech synthesis technology, a technology has also been proposed in which voice features (eg, spectrum, pitch) of multiple speakers are modeled using one DNN (see Patent Document 1). According to this technique, high-quality synthesized speech can be generated even if the amount of speech data per speaker is small.

Patent No. 6622505

Heiga Zen, et al., "STATISTICAL PARAMETRIC SPEECH SYNTHESIS USING DEEP NEURAL NETWORKS", Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013. Ozuru Takuya, et al. "Are you professional?: Analysis of prosodic features between a newscaster and amateur speakers through partial substitution by DNN-TTS", Proc. 10th International Conference on Speech Prosody 2020. 2020. Masuko et al., “Speech synthesis based on HMM using dynamic features”, IEICE, vol. J79-D-II, no. 12, pp. 2184-2190, Dec. 1996. Imai et al., “Mel-logarithmic spectral approximation (MLSA) filter for speech synthesis,” IEICE Transactions A Vol.J66-A No.2 pp.122-129, Feb. 1983.

However, the speaking skill (ease of hearing the voice) of the synthesized voice generated by the above technology depends on the quality of the recorded voice. Therefore, if the speech skill of the recorded voice is low, the speech skill of the generated synthetic voice will also be low.

Here, in order to generate synthesized speech of an arbitrary speech skill of a speaker desired by the user, a method of inputting an identifier indicating the speaker of the synthesized speech and the speech skill to the DNN may be considered.

However, this method only assigns one speaking skill to one speaker when training the DNN, so if the user changes the speaking skill input into the DNN, the speaker characteristics of the synthesized speech will also change. There's a problem.

Therefore, an object of the present invention is to generate synthesized speech of any speech skill of any speaker without changing the speaker characteristics.

In order to solve the above-mentioned problems, the present invention includes an acquisition unit that acquires voice information of a plurality of speakers and speech skills of each of the speakers, and a method that converts the speech skill to a lower level for each of the voice information of one speaker. a generation unit that generates voice information of the plurality of speakers, the speech information of the plurality of speakers, the speech skills of each of the speakers, and the speech information of the speaker whose speech skill has been lowered as learning data; On the other hand, the present invention is characterized by comprising a learning section that performs learning of a speech synthesis model that outputs speech feature amounts of synthesized speech of a specified speaker and speech skill.

According to the present invention, synthesized speech of any speaking skill of any speaker can be generated without changing the speaker characteristics.

FIG. 1 is a diagram for explaining an overview of a speech synthesis model learned by the learning device of this embodiment. FIG. 2 is a diagram illustrating an example of phoneme segmentation information. FIG. 3 is a diagram showing an example of phoneme segmentation information to which a speaker identifier and a speaking skill are added. FIG. 4 is a diagram showing an example of phoneme segmentation information to which a speaker identifier and a speaking skill are added. FIG. 5 is a diagram showing an example of the configuration of the learning device. FIG. 6 is a diagram for explaining an example of a procedure in which the learning device learns a speech synthesis model. FIG. 7 is a diagram for explaining an example of a procedure for generating synthesized speech using a learned speech synthesis model. FIG. 8 is a flowchart illustrating an example of a procedure in which the learning device learns a speech synthesis model. FIG. 9 is a flowchart illustrating an example of a procedure in which the learning device generates synthesized speech using the learned speech synthesis model. FIG. 10 is a diagram showing an example of the configuration of a computer that executes a learning program.

Hereinafter, modes for carrying out the present invention (embodiments) will be described with reference to the drawings. The present invention is not limited to this embodiment.

[overview]
First, an overview of the learning device of this embodiment will be explained using FIG. The learning device performs learning of a speech synthesis model that generates synthesized speech of input text.

For example, as shown in Figure 1, this speech synthesis model accepts input of the pronunciation/accent of the text for which synthetic speech is to be generated, the speaker's identifier, and the speech skill. This is a model that outputs the voice features of the synthesized speech uttered. This speech synthesis model is realized by, for example, a DNN (deep neural network).

The learning device learns the above-mentioned speech synthesis model using the audio data, the identifier of the speaker of the audio data, and the speaking skill of the speaker as learning data.

Here, the learning device generates data (pseudo voice data, pseudo speech skill, pseudo speech information in FIG. 2) with a lower speech skill from the learning data. The learning device then learns the speech synthesis model using learning data including the generated data.

Thereby, the learning device can learn a plurality of speaking skills for one speaker when learning the speech synthesis model. As a result, even if the trained speech synthesis model accepts input of any speech skill from any speaker, it is possible to output acoustic features of synthesized speech whose speaker characteristics do not change as much as possible.

[Audio data]
Next, the speech data and utterance information used for learning the speech synthesis model will be explained. First, audio data will be explained.

The speech data used to train the speech synthesis model is composed of speech data from N (two or more) speakers. This audio data includes audio features obtained by signal processing of the audio signal. The audio feature amount is, for example, a pitch parameter (fundamental frequency, etc.), a spectral parameter (cepstrum, mel cepstrum, etc.) of the audio signal, and the like.

Furthermore, it is desirable that the voice data used for learning the voice synthesis model include voice data of speakers with various speaking skills. For example, this audio data includes audio data of speakers with high speaking skills (e.g., announcers, voice actors, etc.) and audio data of speakers with low speaking skills (e.g., speakers other than the above-mentioned announcers, voice actors, etc.). It is desirable that both are included.

[Utterance information]
Next, utterance information used for learning the speech synthesis model will be explained. The utterance information is, for example, information indicating the pronunciation of each utterance constituting the audio data. This speech information includes, for example, phoneme segmentation information.

For example, as shown in FIG. 2, the phoneme segmentation information is information indicating the start time and end time of each phoneme that constitutes an utterance. The start time and end time are elapsed times from the start point of the utterance to 0 (seconds).

The utterance information may include accent information (accent type, accent phrase length), part of speech information, etc. in addition to phoneme segmentation information.

Furthermore, the learning device assigns to the speech information an identifier (speaker identifier) of a speaker of the speech information and a speech skill of the speaker. For example, as shown in FIGS. 3 and 4, the learning device adds an identifier of the speaker of the utterance and information on the utterance skill to the phoneme segmentation information of the utterance. Note that the speech skill information is expressed, for example, by a one-dimensional vector of 0: low speech skill to 1: high speech skill. The method for imparting speech skills will be described later.

[Configuration example]
Next, a configuration example of the learning device will be described using FIG. 5. The learning device 10 includes, for example, an input/output section 11, a storage section 12, and a control section 13.

The input/output unit 11 is an interface that controls input and output of various data. For example, the input/output unit 11 receives input of learning data (voice data, utterance information of the voice data, speaker identifier of the voice data) of the voice synthesis model. The input/output unit 11 also receives inputs such as the pronunciation/accent of the text for which synthetic speech is to be generated, the speaker's identifier, and the speaking skill. Further, the input/output unit 11 outputs synthesized speech etc. generated by the control unit 13.

The storage unit 12 is realized by a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.

The storage unit 12 stores, for example, a processing program for realizing the functions of the learning device 10, learning data used for learning a speech synthesis model (voice data, utterance information of the voice data, speaker identifier of the voice data, etc.). ) is memorized.

Further, when the learning device 10 acquires the speaking skills of the speakers of the audio data included in the learning data, the speaking skills of each speaker are added to the learning data in the storage unit 12. Further, when the speech synthesis model is learned by the learning device 10, the parameters of the speech synthesis model obtained by the learning are stored in the storage unit 12.

The control unit 13 is realized using, for example, a CPU (Central Processing Unit), and executes a processing program stored in the storage unit 12. As a result, the control unit 13 operates as shown in FIG. It functions as a generation section 138 and a synthesized speech generation section 139.

[Speech Skills Acquisition Department]
The utterance skill acquisition unit 131 acquires the utterance skill of the speaker of the audio data included in the learning data (see FIG. 6). For example, when the speech skill acquisition unit 131 receives an input of a subjective evaluation of the speech skills of a speaker of audio data from a user (evaluator) via the input/output unit 11, the speech skill acquisition unit 131 determines the speech skills of the speaker based on the evaluation result. Grant.

(1) For example, the evaluator evaluates the speaking skills of N speakers included in the audio data using the paired comparison method. In the paired comparison method, for all combinations of speakers ( _N C ₂ pairs: if N = 10 people, ₁₀ C ₂ =45 pairs), it is evaluated which speaker's speaking skill is higher.

(2) Evaluation scales include, for example, AB evaluation, which selects which speaker has higher speaking skills among a combination of speakers to be evaluated (two speakers), and score evaluation using comparative category scaling (CCR). Grant can be used. The score is, for example, -2: The latter speaker has higher speaking skills, -1: The latter speaker has slightly higher speaking skills, 0: Both speaking skills are the same, 1: The former speaker has slightly higher speaking skills. , 2: The former's speaking skill is higher, etc.

(3) Also, for example, a plurality of evaluators evaluate the same combination of speakers, and the speech skill acquisition unit 131 takes the average value of the evaluation results as the speech skill between the speakers. Here, in order to ensure the reliability of the evaluation, it is desirable that the number of evaluators be 10 or more for one combination of speakers.

(4) The speech skill acquisition unit 131 obtains the score between speakers using the above method as a speech skill. Furthermore, the speech skill acquisition unit 131 may convert the scores between speakers obtained by the above method into a one-dimensional scale using Scheffe's pairwise comparison method or the like. The speech skills of each speaker acquired by the speech skill acquisition section 131 are stored in the storage section 12.

[Acquisition part]
The acquisition unit 132 acquires voice information of a plurality of speakers as learning data (voice data, utterance information of the voice data, speaker identifier of the voice data) and the voice information of each speaker acquired by the speech skill acquisition unit 131. Acquire speaking skills.

[Generation part]
The generation unit 133 generates voice information in which the speaking skill is lowered for each voice information of one speaker. For example, the generation unit 133 generates voice data (pseudo voice data) with lowered speech skills, speech information (pseudo speech information), and pseudo speech skills from the speech information acquired by the acquisition unit 132 (see FIG. 6). ).

For example, the generation unit 133 generates pseudo voice data and pseudo speech information converted from speech data and speech information of speaker B with speech skill=1 to speech skill=0.8, and pseudo speech data and pseudo speech information converted to speech skill=0.5. Speech data and pseudo speech information, and pseudo speech data and pseudo speech information converted to speech skill=0.2 are generated.

Then, the generation unit 133 stores the generated pseudo voice data, pseudo speech information, and pseudo speech skill in the storage unit 12.

For example, in order to generate pseudo voice data, pseudo speech information, and pseudo speech skills, the generation unit 133 performs (1) conversion of voice parameters of voice data included in the voice information, (2) speech information ( (phoneme segmentation information) or a combination thereof.

[Conversion of pitch parameters]
An example of a processing procedure when the generation unit 133 generates pseudo speech data and pseudo speech skills by converting the audio parameters of the audio data will be described.

The generation unit 133 generates pseudo voice data by, for example, changing the distribution of pitch parameters within utterances in the voice data of the learning data. For example, the generation unit 133 generates the pitch parameter (F0 _ij (t), t=1,...,T: number of frames of audio parameter) of the j-th utterance of speaker i in the audio data of the learning data. A pseudo pitch parameter F0' _ij of pitch parameters is generated using the average μ _ij . For example, the generation unit 133 converts each pitch parameter using the following equation (1) so that the standard deviation of each pitch parameter in the utterance becomes small.

F0' _ij (t)=(F0 _ij (t)-μ _ij )×α+μ _ij (α: conversion weight (0≦α≦1))…Equation (1)

Here, α is the transformation weight. When α=1, the generation unit 133 does not convert the pitch parameter. When α=0, the pitch parameter is the same as the average μ _ij .

Further, the generation unit 133 obtains a pseudo speech skill S′ _i by converting the speech skill S _i of the speaker i at this time based on the following equation (2).

S' _i =α×Si...Equation (2)

Note that when the generation unit 133 converts the pitch parameter, it does not convert the speech information, so the original speech information is used as pseudo speech information.

[Conversion of speech information (phoneme segmentation information)]
An example of a processing procedure when the generation unit 133 generates pseudo speech information and pseudo speech skills by converting speech information (phoneme segmentation information) will be described.

For example, the generation unit 133 generates the p-th phoneme segmentation information (d _ij (p), p=1,...,P: the number of phonemes included in the phoneme segmentation information) of the j-th utterance of speaker i in the utterance information. Pseudo utterance information d' _ij (p) is generated by performing conversion to extend or contract the duration length.

Here, the generation unit 133 can generate a nonlinear Perform a conversion. The converted speech information (pseudo speech information) d' _ij (p) is expressed by the following equation (3).

d' _ij (p)=β×d _ij (p)(β: conversion weight (0≦β≦2))...Equation (3)

In equation (3), β is a transformation weight. In equation (3), when β=1, the generation unit 133 does not convert the p-th phoneme. When β=0, the generation unit 133 performs conversion to set the duration length of the p-th phoneme to zero. In other words, the generation unit 133 deletes the p-th phoneme.

Furthermore, the generation unit 133 obtains a pseudo speech skill S'i by converting the speech skill Si of the speaker i based on the following equation (4) using the above β.

S' _i =(1-|1-β|)×S _i ...Equation (4)

In this way, the generation unit 133 converts the distribution of pitch parameters in the audio data and converts the duration of phonemes in the utterance information (phoneme segmentation information) to expand or contract. This is because the distribution of pitch parameters and the duration of phonemes in speech have a great influence on the evaluation of speech skills. The generation unit 133 can generate unnatural speech (≈speech with low speaking skill) that differs from natural speech by converting the pitch parameter distribution, phoneme duration, etc. of the speech. Thereby, the generation unit 133 can generate voices with low speaking skill even though they are uttered by the same speaker.

Note that, when generating voice information obtained by converting a speech skill into a lower value, it is preferable that the generation unit 133 generates voice information of a plurality of speech skills based on voice information of one speaker. For example, the generation unit 133 generates, based on the voice information of a speaker (speech skill=0.8), voice information of the speaker's speaking skill=0.5 and voice information of the speaker's speaking skill=0.2. In this way, the generation unit 133 generates the audio information of multiple speaking skills based on the audio information of one speaker, so that the audio information of various speaking skills for the same speaker is added to the learning data. That will happen. As a result, the learning device 10 is able to learn a speech synthesis model capable of outputting speech features of synthesized speech of various speech skills for a specified speaker.

[Vector representation conversion unit]
The vector expression conversion unit 134 converts the speech information, the speaker identifier, and the speech skill into an expression (numerical expression) that can be used in a speech synthesis model. For example, the vector expression conversion unit 134 converts speech information into a language vector, a speaker identifier into a speaker vector, and a speech skill into a speech skill vector.

For example, when the vector expression conversion unit 134 converts speech information into a language vector, it uses a technique described in Non-Patent Document 1 for converting speech information into numerical vectors (language vectors) indicating phonemes, accents, etc.

Further, when the vector expression conversion unit 134 converts a speaker identifier into a speaker vector, a one-hot expression is used. The number of dimensions of this one-hot expression vector is the number of speakers included in the audio data = N. In this speaker vector, the value of the dimension corresponding to the speaker identifier is 1, and the value of the other dimensions is 0.

Furthermore, when the vector expression conversion unit 134 converts the speech skill into a speech skill vector, the speech skill obtained by the speech skill acquisition unit 131 may be used as is, or it may be dimensionally compressed by some method. Good too. For dimension reduction, for example, differences in speaking skills between speakers may be used as a scale value using Scheffe's paired comparison method, or results of principal component analysis (PCA) may be used.

For example, as shown in FIG. 6, the vector representation conversion unit 134 converts the speaker's speaking skill and pseudo speaking skill included in the learning data into a speaking skill vector. Furthermore, the vector expression conversion unit 134 converts the utterance information and pseudo utterance information included in the learning data into language vectors. Further, the vector expression conversion unit 134 converts the speaker identifier included in the learning data into a speaker vector.

[Study Department]
The learning unit 135 learns a speech synthesis model using the learning data. For example, the learning unit 135 uses, as learning data, voice information of a plurality of speakers and voice information (pseudo voice data, pseudo speech skills, pseudo speech information) obtained by converting the speech skills of each of the plurality of speakers to a lower level. to learn a speech synthesis model.

For example, the learning unit 135 obtains the language vector, speaker vector, and utterance skill vector for each utterance information of the learning data converted by the vector expression conversion unit 134. Further, the learning unit 135 acquires the audio feature amount of the audio data corresponding to each piece of utterance information. Then, the learning unit 135 uses the acquired language vector, speaker vector, and speech skill vector of each piece of speech information and the acoustic feature amount corresponding to each piece of speech information to calculate the language vector, speaker vector, and speech skill vector. A speech synthesis model for estimating speech features is learned (see FIG. 6).

As the learning algorithm for the speech synthesis model, for example, the learning algorithm described in Patent Document 1, Non-Patent Document 1, etc. is used. The learning unit 135 stores the parameters of the learned speech synthesis model in the storage unit 12.

Note that the neural networks used in the speech synthesis model include, for example, normal Multilayer perceptron (MLP), Recurrent Neural Network (RNN), RNN-LSTM (Long Short Term Memory), Convolutional Neural Network (CNN), etc. It is also a neural network that combines these.

The speech synthesis model uses the identifier and speaking skill of the speaker targeted for the synthesized speech as input data for generating the synthesized speech. Therefore, the learning unit 135 performs learning of the speech synthesis model by separating the characteristics of the speaker and the characteristics of the speaking skill. Thereby, the learning device 10 can learn a speech synthesis model that generates synthesized speech of any speaking skill by any speaker.

[Input reception section]
The input receiving unit 136 receives, via the input/output unit 11, the identifier, speech skill, text, etc. of the speaker for whom synthetic speech is to be generated, specified by the user.

[Text analysis department]
The text analysis unit 137 analyzes the text input from the input reception unit 136. For example, the text analysis unit 137 analyzes the input text and generates information indicating the pronunciation, accent, etc. of the text.

[Speech feature generator]
The speech feature generation unit 138 uses the speech synthesis model learned by the learning unit 135 to generate speech features of the synthesized speech of the speaker and speech skill specified by the user from the text for which the synthesized speech is to be generated. do.

For example, as shown in FIG. 7, the vector expression conversion unit 134 first converts the analysis result of the text generated by the text analysis unit 137 into a language vector. Further, the vector representation conversion unit 134 converts the speaker identifier received by the input reception unit 136 into a speaker vector, and also converts the speech skill received by the input reception unit 136 into a speech skill vector.

Then, the speech feature generation unit 138 inputs the above-mentioned speaker vector, speech skill vector, and language vector to the learned speech synthesis model, and obtains the speech feature of the synthesized speech output from the speech synthesis model. This voice feature amount is, for example, a pitch parameter used to generate synthesized voice.

[Synthetic speech generation unit]
The synthesized speech generation section 139 generates synthetic speech using the speech features generated by the speech feature generation section 138. The synthesized speech generation unit 139 uses, for example, the Maximum Likelihood Parameter Generation (MLPG) algorithm (see Non-Patent Document 2) to obtain a speech parameter sequence in which the speech feature amount is smoothed in the temporal direction. Then, the synthesized speech generation unit 139 generates a speech waveform using this speech parameter series. For example, the technique described in Non-Patent Document 3 is used to generate the audio waveform. Then, the synthesized speech generation section 139 generates synthesized speech using the generated speech waveform and outputs it via the input/output section 11.

[Example of processing procedure]
Next, an example of a processing procedure executed by the learning device 10 will be described. First, an example of a procedure in which the learning device 10 learns a speech synthesis model will be described using FIG. 8.

The learning device 10 acquires speech information (speech data, utterance information, speaker identifier) for learning a speech synthesis model (S1). Then, the acquired audio information is stored in the storage unit 12. Next, the speech skill acquisition unit 131 obtains the speech skill of the audio data obtained in S1 (S2). Then, the acquired speech skill is stored in the storage unit 12.

After S2, the generation unit 133 generates voice information (pseudo voice data, pseudo speech skill, pseudo speech information) in which the speech skill is lowered for each voice information of one speaker (S3). After that, the acquisition unit 132 acquires the voice information with a lowered speaking skill generated in S3 and the learning voice information (S4).

After S4, the vector representation conversion unit 134 converts the audio information acquired by the acquisition unit 132 in S4 into a vector expressed with a value that can be used in the voice synthesis model (S5). For example, the vector expression conversion unit 134 converts the speech skill included in the speech information into a speech skill vector, and converts the speech information into a language vector, regarding the speech information with a lower speech skill acquired by the acquisition unit 132 in S4. and converts the speaker identifier into a speaker vector. Furthermore, regarding the learning audio information acquired by the acquisition unit 132 in S4, the vector representation conversion unit 134 converts the utterance skill included in the audio information into a utterance skill vector, and converts the utterance information into a language vector. , convert the speaker identifier into a speaker vector.

After S5, the learning unit 135 learns a speech synthesis model using the speech data included in the learning speech information and the speech skill vector, language vector, and speaker vector converted in S5 (S6). .

By performing the above processing, the learning device 10 can learn a plurality of speaking skills for one speaker when learning a speech synthesis model.

Next, an example of a procedure for generating synthesized speech using the learned speech synthesis model will be described with reference to FIG.

First, the input receiving unit 136 of the learning device 10 receives input from the user of a speaker identifier for which synthetic speech is to be generated, a speaking skill, and a text (S11). Thereafter, the text analysis unit 137 analyzes the text input in S11. Further, the vector representation conversion unit 134 converts the information input in S11 into a vector (S12). For example, the vector expression conversion unit 134 converts a speaker identifier into a speaker vector, converts a speech skill into a speech skill vector, and converts a text analysis result into a language vector.

Thereafter, the speech feature generation unit 138 generates speech features of the synthesized speech using the speaker vector, speech skill vector, and language vector converted in S12 and the learned speech synthesis model (S13). . That is, the voice feature generation unit 138 generates the voice feature of the synthesized speech by inputting the speaker vector, speech skill vector, and language vector converted in S12 to the learned voice synthesis model.

Thereafter, the synthesized speech generation unit 139 generates synthesized speech using the speech features generated in S13 and outputs it (S14).

By the learning device 10 performing the above processing, it is possible to output synthesized speech of the speaker and speaking skill specified by the user.

[System configuration, etc.]
Further, each component of each part shown in the drawings is functionally conceptual, and does not necessarily need to be physically configured as shown in the drawings. In other words, the specific form of distributing and integrating each device is not limited to what is shown in the diagram, and all or part of the devices can be functionally or physically distributed or integrated in arbitrary units depending on various loads, usage conditions, etc. Can be integrated and configured. Furthermore, all or any part of each processing function performed by each device may be realized by a CPU and a program executed by the CPU, or may be realized as hardware using wired logic.

Further, among the processes described in the embodiments described above, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed manually. All or part of this can also be performed automatically using known methods. In addition, information including processing procedures, control procedures, specific names, and various data and parameters shown in the above documents and drawings may be changed arbitrarily, unless otherwise specified.

[program]
The learning device 10 described above can be implemented by installing a program (learning program) in a desired computer as packaged software or online software. For example, by causing the information processing device to execute the above program, the information processing device can be made to function as the learning device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHSs (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

FIG. 10 is a diagram showing an example of a computer that executes a learning program. Computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These parts are connected by a bus 1080.

The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores, for example, a boot program such as BIOS (Basic Input Output System). Hard disk drive interface 1030 is connected to hard disk drive 1031. Disk drive interface 1040 is connected to disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. Serial port interface 1050 is connected to, for example, mouse 1110 and keyboard 1120. Video adapter 1060 is connected to display 1130, for example.

The hard disk drive 1031 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process executed by the learning device 10 described above is implemented as a program module 1093 in which computer-executable code is written. Program module 1093 is stored in hard disk drive 1031, for example. For example, a program module 1093 for executing processing similar to the functional configuration of the learning device 10 is stored in the hard disk drive 1031. Note that the hard disk drive 1031 may be replaced by an SSD (Solid State Drive).

Further, data used in the processing of the embodiment described above is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1031. Then, the CPU 1020 reads out the program module 1093 and program data 1094 stored in the memory 1010 and the hard disk drive 1031 to the RAM 1012 as necessary and executes them.

Note that the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1031, but may be stored in a removable storage medium, for example, and read by the CPU 1020 via the disk drive 1041 or the like. Alternatively, program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from another computer via the network interface 1070.

10 Learning device 11 Input/output section 12 Storage section 13 Control section 131 Speech skill acquisition section 132 Acquisition section 133 Generation section 134 Vector expression conversion section 135 Learning section 136 Input reception section 137 Text analysis section 138 Speech feature generation section 139 Synthetic speech generation Department

Claims (8)

an acquisition unit that acquires voice information of a plurality of speakers and speech skills of each of the speakers;
a generation unit that generates voice information with a low speaking skill converted for each of the voice information of one speaker;
Using the voice information of the plurality of speakers, the speaking skill of each of the speakers, and the voice information of the speaker whose speaking skill has been converted to a lower value as learning data, a specified speaker and speaking skill is determined for the input text. A learning device comprising: a learning unit that learns a speech synthesis model that outputs speech features of synthesized speech. The voice information of the speaker is
The learning device according to claim 1, further comprising at least one of a pitch parameter and phoneme segmentation information of the utterance of the speaker. When the voice information of the speaker includes pitch parameters for each frame of utterance,
The generation unit is
The learning device according to claim 2, wherein the speech information is generated in which the speech skill is lowered by changing the distribution of pitch parameters within the speech. When the voice information of the speaker includes phoneme segmentation information of the utterance of the speaker,
The generation unit is
3. The speech information according to claim 2, characterized in that the speech information in which the speech skill is lowered is generated by expanding or contracting the duration of each phoneme constituting the utterance, which is shown in the phoneme segmentation information of the utterance of the speaker. The learning device described. The method further includes a voice feature generating unit that uses the learned voice synthesis model to generate voice features for generating synthesized speech of a speaker and speaking skill specified by the user, for the input text. The learning device according to claim 1, characterized in that: The learning device according to claim 5, further comprising: a synthetic speech generation unit that generates synthetic speech using the generated speech feature amount. A learning method performed by a learning device, comprising:
acquiring voice information of a plurality of speakers and speech skills of each of the speakers;
a step of generating voice information with a lower speaking skill converted for each of the voice information of one speaker;
Using the voice information of the plurality of speakers, the utterance skills of each of the speakers, and the voice information of the speaker whose utterance skills have been converted to a lower level as learning data, a specified speaker and utterance are generated for the input text. A learning method comprising the steps of: training a speech synthesis model that outputs speech features of synthesized speech of the skill. acquiring voice information of a plurality of speakers and speech skills of each of the speakers;
a step of generating voice information with a lower speaking skill converted for each of the voice information of one speaker;
Using the voice information of the plurality of speakers, the utterance skills of each of the speakers, and the voice information of the speaker whose utterance skills have been converted to a lower level as learning data, a specified speaker and utterance are generated for the input text. The process of learning a speech synthesis model that outputs the speech features of the synthesized speech of the skill, and the learning program for the computer to execute.

Images