JP2016151736A - Speech processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and precisely perform a style conversion on a speech.SOLUTION: A speech processing device acquires a first acoustic feature quantity generation value as an acoustic feature quantity of each of frames of a time series based upon a language feature quantity of a document and a statistical model associated with an acoustic feature quantity generated as to a first style, and obtains a second acoustic feature quantity generation value as an acoustic feature quantity of each of frames of the time series based upon the language feature quantity of the document and a statistical model associated with an acoustic feature quantity generated as to a second style. The speech processing device makes frames of the first acoustic feature quantity generation value and the second acoustic feature quantity generation value correspond to each other, and generates processing information with the difference between the first acoustic feature quantity generation value and the second acoustic feature quantity generation value for each of the frames which are made to correspond. The speech processing device makes an acoustic feature quantity of speech data generated by reading the document aloud and frames of the first acoustic feature quantity generation value correspond to each other, and processes acoustic feature quantities of the respective frames of the speech data based upon processing information generated from the first acoustic feature quantity generation value of a corresponding frame.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a voice processing device and a program.

  In recent years, a method of synthesizing speech from text using a statistical model has been developed (see, for example, Non-Patent Document 1). Research based on this method has been actively promoted, and methods for synthesizing styled speech have also been proposed (for example, see Non-Patent Documents 2, 3, and 4).

  On the other hand, methods for converting acoustic feature quantities such as the pitch and spectrum of speech have been proposed (see, for example, Patent Documents 1 and 2 and Non-Patent Document 5). Furthermore, for example, a method for controlling emotions in a voice style has been proposed. One of the methods is as follows. That is, by analyzing acoustically what was read out the same sentence by emotionless utterance and emotional utterance, the pitch, power, and speech speed were obtained, and the analysis results were manually observed and compared. Create a simple transformation rule. Then, the created conversion rule is applied to another emotionless utterance to give emotion (see, for example, Non-Patent Document 6).

Other methods for controlling emotion in the audio style include the following. That is, in advance, an acoustic analysis is performed on the same sentence read out by an emotional utterance and an utterance with an emotion to obtain a time series of acoustic feature quantities such as a pitch and a spectrum, and a correspondence between them is obtained. Then, based on the obtained correspondence, learn a neural network that takes the time series of emotional features of emotionless speech and the type of emotion as input, and outputs the time series of acoustic features with emotion To do. A time series of acoustic feature quantities is obtained by inputting a time series of acoustic feature quantities obtained by acoustic analysis of an emotionless utterance of an arbitrary sentence and a desired emotion type to the learned neural network. The generation | occurrence | production which provided the desired emotion is obtained by synthesize | combining audio | voice based on the time series of the obtained acoustic feature-value (for example, refer patent document 3).
As another method, there is a technique of learning a spectral change between a calm voice and an emotion voice for each vowel, and giving the learned vowel spectrum change to a quiet voice of an arbitrary utterance (see, for example, Non-Patent Document 7). .

Japanese Patent No. 2612867 Japanese Patent No. 2612869 JP-A-7-72900

Keiichi Tokuda, 4 others, "Speech parameter generation algorithms for HMM-based speech synthesis", IEEE, in Proc. ICASSP, 2000 Makoto Tachibana, 3 others, "Speech synthesis with various emotional expressions and speaking styles by style interpolation and morphing", IEICE, IEICE transactions on information and systems, E88-D (11), p.2484- 2491, 2005 Takashi Nose, 3 others, "A style Control Technique for HMM-Based Expressive Speech Synthesis", IEICE, IEICE transactions on information and systems, E90-D (9), p.1406-1413, 2007 Year Yamato Otani, 5 others, "Study on how to give emotions to arbitrary speakers based on addition model in HMM speech synthesis", Proceedings of the Acoustical Society of Japan, 2-7-2, p.233-236, 2014 Hideki Kawahara, 2 others, "Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds", Speech Communication, 27 (3), 1999 Yoshinori Kitahara, 1 other, "Prosodic Control to Express Emotions for Man-Machine Speech Interaction", IEICE, IEICE transactions on fundamentals of electronics, Communications and Computer Sciences, Vol.E75-A, No.2 , P.155-163, 1992 Atsushi Imai and 2 others, “Examination of emotional speech processing method based on spectrum change of vowels”, The Institute of Image Information and Television Engineers, Annual Conference of the Institute of Image Information and Television Engineers 2014, 2014

The techniques of Non-Patent Documents 1 to 4 are all methods for synthesizing speech from text, and do not convert speech.
Patent Documents 1 and 2 and Non-Patent Document 5 relate to a basic technique for converting acoustic feature quantities such as pitch and spectrum, and in order to convert sound into a desired style, the target value is set to some method. Must be given in.
Further, since the technology of Non-Patent Document 6 controls speech processing by simple rules generated manually, it is difficult to sufficiently control the acoustic feature quantity that changes in a complicated manner over time. In addition, the technique of Non-Patent Document 6 performs only control related to prosody such as pitch, power, and speech speed, and cannot control the spectrum.
Further, the technique of Patent Document 3 uses a neural network for learning parameters related to emotion, and learning requires enormous learning data and learning time.
The technique of Non-Patent Document 7 is to process a spectrum for vowels that have a great influence on hearing and convert quiet speech to emotional speech, but for consonants to process emotional speech. Absent.

  The present invention has been made in consideration of such circumstances, and provides an audio processing apparatus and a program that can perform audio style conversion easily and accurately.

One aspect of the present invention is generated from a language analysis unit that acquires a language feature of a sentence indicated by text data, the language feature acquired by the language analysis unit, and speech data of an utterance of a first style Based on a statistical model related to acoustic features, a first acoustic feature generator that generates acoustic features in units of time-series frames, the language feature obtained by the language analyzer, and a second style Based on a statistical model related to acoustic features generated from speech data of speech, a second acoustic feature generating unit that generates acoustic features in units of frames in time series, and the first acoustic feature generating unit generates Based on the similarity between the first acoustic feature value generation value that is the acoustic feature value and the second acoustic feature value generation value that is the acoustic feature value generated by the second acoustic feature value generation unit. One acoustic feature generation And the frame of the second acoustic feature value generation value are associated with each other, and the processing information is determined by the difference between the first acoustic feature value generation value and the second acoustic feature value generation value for each of the associated frames. A processing information generation unit that generates, an acoustic analysis unit that acquires acoustic features in frame units in time series from the audio data of the sentence indicated by the text data, and the acoustic feature amount acquired by the acoustic analysis unit, Based on the similarity with the first acoustic feature value generation value, the acoustic feature value frame and the first acoustic feature value generation value frame are associated with each other, and the acoustic feature value of each frame is associated with the corresponding frame. And a voice processing unit that processes based on the processing information generated by the processing information generation unit using the first acoustic feature value generation value.
According to the present invention, the speech processing apparatus uses a time-series frame-based acoustic feature amount based on the language feature amount of the text of the original speech and the statistical model related to the acoustic feature amount generated for the first style. A certain first acoustic feature value generation value is obtained. Furthermore, the speech processing apparatus performs second sound, which is a time-series frame-based acoustic feature amount, based on the linguistic feature amount of the text of the original speech and the statistical model related to the acoustic feature amount generated for the second style. A feature value generation value is obtained. The speech processing device associates the first acoustic feature value generation value frame and the second acoustic feature value generation value frame with the similarity of values, and sets the first acoustic feature value generation value and the first sound feature value for each associated frame. Processing information is generated based on the difference between the two acoustic feature value generation values. The sound processing device acquires sound feature quantities in time-series frames from the sound data of the original sound, and based on the similarity of values between the sound feature amount frame of the original sound and the first sound feature value generation value frame To associate. The sound processing device processes the acoustic feature amount of each frame of the original speech based on the processing information generated using the first acoustic feature amount generation value of the corresponding frame.
As a result, the speech processing apparatus easily converts the style of the original speech while maintaining the phonological and naturalness of the original speech well.

One aspect of the present invention is the speech processing device described above, wherein the acoustic feature amount includes at least one of information regarding a pitch and information regarding a frequency spectrum.
According to the present invention, the sound processing device changes the style by processing one or both of the pitch and the frequency spectrum of the original sound.
As a result, the speech processing apparatus can change the pitch by changing the pitch of the original speech and changing the intonation and accent while maintaining the phonological and naturalness of the original speech. Further, the speech processing apparatus can change the voice quality and change the style while changing the frequency spectrum of the original speech and maintaining the phonological and naturalness of the original speech well. Alternatively, the voice processing device changes the pitch and frequency spectrum of the original voice, changes the intonation, accent, and voice quality while maintaining the phonological and natural nature of the original voice, thereby changing the style of the original voice. Can be converted.

One aspect of the present invention is the speech processing apparatus described above, wherein the style of the speech data of the sentence indicated by the text data is the first style.
According to this invention, the speech processing apparatus uses the statistical model generated from the utterance of the same style as the original speech and the statistical model generated from the utterance of the desired style, respectively, from the text of the original speech. One acoustic feature value generation value and a second acoustic feature value generation value are generated, and processing information is generated based on a difference between them.
Thereby, the voice processing apparatus can convert the original voice into a desired style with high accuracy.

One aspect of the present invention is the speech processing apparatus described above, wherein the speech processing unit synthesizes speech data based on the processed acoustic feature amount.
According to the present invention, the speech processing apparatus synthesizes speech from the acoustic feature amount processed for style conversion.
Thereby, the sound processing apparatus can output the sound generated by converting the style of the original sound.

  According to one aspect of the present invention, a computer analyzes a language analysis unit that acquires a language feature of a sentence indicated by text data, the language feature acquired by the language analysis unit, and speech data of an utterance of a first style. Based on the generated statistical model for the acoustic feature amount, a first acoustic feature amount generation unit that generates an acoustic feature amount in time-series frames, the language feature amount acquired by the language analysis unit, and a second Second acoustic feature generating means for generating acoustic features in units of frames in time series based on a statistical model relating to acoustic features generated from speech data of utterances of the style, and the first acoustic feature generation Based on the similarity between the first acoustic feature value generation value that is the acoustic feature value generated by the means and the second acoustic feature value generation value that is the sound feature value generated by the second acoustic feature value generation means. The first acoustic feature value generation value and the second acoustic feature value generation value frame are associated with each other, and the first acoustic feature value generation value and the second sound are associated with each of the associated frames. Processing information generation means for generating processing information based on a difference from a feature value generation value, acoustic analysis means for acquiring acoustic features in time-series frames from voice data of the sentence indicated by the text data, and the acoustic analysis Based on the similarity between the acoustic feature value acquired by the means and the first acoustic feature value generation value, the frame of the acoustic feature value and the frame of the first acoustic feature value generation value are associated with each other. Audio processing unit for processing the acoustic feature amount based on the processing information generated by the processing information generation unit using the first acoustic feature amount generation value of the corresponding frame. Is a program for functioning as a voice processing device.

  According to the present invention, voice style conversion can be performed easily and accurately.

It is a functional block diagram of the audio processing apparatus by one Embodiment of this invention. It is a figure for demonstrating the acoustic feature-value which the audio processing apparatus by the embodiment uses. It is a figure which shows the acoustic feature-value which the audio processing apparatus by the embodiment uses. It is a figure which shows the language feature-value which the audio processing apparatus by the embodiment uses. It is a processing flow which shows the learning process in the audio processing apparatus by the embodiment. It is a processing flow which shows the audio processing process in the audio processing apparatus by the embodiment. It is a figure for demonstrating the acoustic feature-value acquisition process from the text in the audio processing apparatus by the embodiment. It is a figure for demonstrating the process information generation process in the audio processing apparatus by the embodiment. It is a figure for demonstrating the process of the input audio | voice in the audio processing apparatus by the embodiment. It is a figure which shows the input audio | voice and style audio | voice which carried out style conversion using the audio processing apparatus by the embodiment. It is a figure which shows the item of the subjective evaluation experiment performed about the audio processing apparatus by the embodiment. It is a figure which shows the determination emotion with respect to the learning data used in order to produce | generate the statistical model used in the subjective evaluation experiment performed about the audio | voice processing apparatus by the embodiment. It is a figure which shows the evaluation result of the subjective evaluation experiment done about the audio | voice processing apparatus by the embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The sound processing apparatus of this embodiment temporarily records the input sound, converts the acoustic feature amount, and outputs the sound again as a different style of sound. Styles include, for example, emotions such as anger and joy, and utterance expressions such as news-like, polite, zaizai, formal, and casual. The speech processing apparatus according to the present embodiment uses a statistical model related to acoustic features that has been created in advance for each of the input speech style and the desired style, and from the input speech text, the input speech style and desired For each of the styles, an acoustic feature amount in units of frames is generated. The speech processing apparatus according to the present embodiment associates the acoustic feature amount generated for the input speech style with the acoustic feature amount frame generated for the desired style, and the acoustic feature amount of the input speech style according to the association. And a difference value between the acoustic feature quantity of the desired style. The speech processing apparatus according to the present embodiment associates an acoustic feature amount in units of frames obtained from the input speech with an acoustic feature amount in units of frames generated for the input speech style from the input speech text. The speech processing apparatus according to the present embodiment changes the acoustic feature amount of the input speech by adding the difference value calculated using the acoustic feature amount of the corresponding frame to the acoustic feature amount of each frame of the input speech. Outputs audio that reflects. Thereby, the speech processing apparatus according to the present embodiment enables style conversion while maintaining the phonological and naturalness of the original speech well.

  FIG. 1 is a functional block diagram showing a configuration of a sound processing apparatus 1 according to an embodiment of the present invention, and only functional blocks related to the present embodiment are extracted and shown. The voice processing device 1 is realized by one or a plurality of computer devices. When the sound processing device 1 is realized by a plurality of computer devices, which functional unit is realized by which computer device can be arbitrarily determined. One functional unit may be realized by a plurality of computer devices. As shown in FIG. 1, the voice processing device 1 includes a learning unit 2, a storage unit 3, and a voice processing unit 4.

  The learning unit 2 includes a first speech storage unit 21, a first acoustic analysis unit 22, a first learning language analysis unit 23, a first statistical model learning unit 24, a second speech storage unit 25, and a second An acoustic analysis unit 26, a second learning language analysis unit 27, and a second statistical model learning unit 28 are provided. The storage unit 3 includes a first statistical model storage unit 31 and a second statistical model storage unit 32.

The first voice storage unit 21 stores first learning voice data. The first learning voice data is voice data for learning when the text indicated by the first learning text data is read out by the pre-conversion style (first style). The pre-conversion style is a style of input voice data input to the voice processing unit 4, and this input voice data is voice data to be subjected to style conversion. When there are a plurality of pre-conversion styles, information indicating the type of style (for example, “seduce”) is added to the first learning audio data.
The first acoustic analysis unit 22 reads the first learning voice data from the first voice storage unit 21, and acquires the time-series acoustic feature quantity from the read first learning voice data.
The first learning language analysis unit 23 acquires the language feature amount of the sentence indicated by the first learning text data.

  The first statistical model learning unit 24 includes the acoustic feature amount acquired from the first learning speech data by the first acoustic analysis unit 22 and the language feature acquired by the first learning language analysis unit 23 from the first learning text data. The statistical model of the style before conversion is generated using the quantity, and the generated statistical model is written in the first statistical model storage unit 31. When there are a plurality of types of pre-conversion styles, the first statistical model learning unit 24 generates a statistical model for each type. For example, the first statistical model learning unit 24 uses the acoustic feature amount of the first learning speech data to which the label “Silence” is assigned and the language feature amount of the first learning text data, and the style is “ Generate a "quiet" statistical model. The first statistical model learning unit 24 adds information indicating the style type to the statistical model generated for each style type, and writes the information to the first statistical model storage unit 31.

The second voice storage unit 25 stores second learning voice data. The second learning voice data is voice data for learning when the sentence indicated by the second learning text data is read out by the converted style (second style). Note that the second learning text data may be the same as or different from the first learning text data. The post-conversion style is the style of audio data desired to be obtained as a result of processing the audio data in the audio processing unit 4. When there are a plurality of converted styles, information indicating the type of style (for example, “anger”, “surprise”, “joy”) is added to the second learning audio data.
The second acoustic analysis unit 26 reads the second learning voice data from the second voice storage unit 25, and acquires the time-series acoustic feature amount from the read second learning voice data.
The second learning language analysis unit 27 acquires the language feature amount of the sentence indicated by the second learning text data by the same processing as the first learning language analysis unit 23.

  The second statistical model learning unit 28 includes the acoustic feature amount acquired from the second learning speech data by the second acoustic analysis unit 26 and the language feature acquired by the second learning language analysis unit 27 from the second learning text data. The converted statistical model is generated using the quantity, and the generated statistical model is written in the second statistical model storage unit 32. When there are a plurality of types of styles after conversion, the second statistical model learning unit 28 generates a statistical model for each type. For example, the second statistical model learning unit 28 uses the acoustic feature amount of the second learning speech data to which the “anger” label is assigned and the language feature amount of the second learning text data, and the style is “ A statistical model of anger is generated. The second statistical model learning unit 28 adds information indicating the style type to the statistical model generated for each style type and writes the information to the second statistical model storage unit 32.

  The statistical model generated by the first statistical model learning unit 24 and the statistical model generated by the second statistical model learning unit 28 are statistical models related to acoustic features. As the statistical model, for example, an acoustic model using a three-state HMM (Hidden Markov Model) can be used. This acoustic model is created for each unit in which phonemes reflecting language feature quantities are classified by clustering using an appropriate decision tree (hereinafter referred to as “phoneme clustering unit”).

  The speech processing unit 4 includes a speech processing language analysis unit 41 (language analysis unit), a first statistical model selection unit 42, a first acoustic feature quantity generation unit 43, a second statistical model selection unit 44, and a second An acoustic feature amount generation unit 45, a processing information generation unit 46, a voice processing acoustic analysis unit 47 (acoustic analysis unit), and a voice processing unit 48 are configured.

The speech processing language analysis unit 41 acquires the language feature amount of the sentence indicated by the input speech text data by the same processing as the first learning language analysis unit 23 and the second learning language analysis unit 27. The input voice text data is text data of a sentence indicating the utterance content of the input voice data.
The first statistical model selection unit 42 reads a statistical model corresponding to the style indicated by the pre-conversion style data from the first statistical model storage unit 31. The pre-conversion style data indicates the style of the input audio data.
The first acoustic feature value generation unit 43 uses the statistical model read by the first statistical model selection unit 42 and the language feature value output from the speech processing language analysis unit 41 to perform time-series frame-based acoustics. Generate feature values. The generated acoustic feature quantity is referred to as a first acoustic feature quantity generation value.

The second statistical model selection unit 44 reads a statistical model corresponding to the style indicated by the converted style data from the second statistical model storage unit 32. The post-conversion style data indicates the style of audio data desired to be obtained as a result of processing the input audio data.
The second acoustic feature value generation unit 45 uses the statistical model read by the second statistical model selection unit 44 and the language feature value output from the speech processing language analysis unit 41 to perform time-series frame-based acoustics. Generate feature values. The generated acoustic feature amount is described as a second acoustic feature amount generation value.

  The processing information generation unit 46 includes a first corresponding frame detection unit 461 and a processing information calculation unit 462. The first corresponding frame detection unit 461 includes a first acoustic feature amount generation value generated by the first acoustic feature amount generation unit 43 and a second acoustic feature amount generation value generated by the second acoustic feature amount generation unit 45. Corresponding in units of frames based on the similarity of values. The processing information calculation unit 462 creates processing information of the acoustic feature amount for each corresponding frame based on the difference between the first acoustic feature amount generation value and the second acoustic feature amount generation value.

The sound processing acoustic analysis unit 47 acquires the acoustic feature amount of the input sound data.
The voice processing unit 48 includes a second corresponding frame detection unit 481, a processing information addition unit 482, and a voice synthesis unit 483. The second corresponding frame detection unit 481 converts the first acoustic feature value generation value generated by the first acoustic feature value generation unit 43 and the acoustic feature value acquired by the sound processing acoustic analysis unit 47 into similarity in value. Based on the frame basis. The processing information adding unit 482 uses the acoustic feature amount of each frame acquired by the sound processing acoustic analysis unit 47 as the acoustic feature amount generated by the processing information generation unit 46 using the first acoustic feature amount generation value of the corresponding frame. Machining based on the machining information. The voice synthesizing unit 483 synthesizes the audio data of the acoustic feature amount obtained by the processing in the processing information adding unit 482 and outputs it as output voice data.

  Note that the learning process of the first statistical model and the second statistical model in the learning unit 2, the first statistical model and the second statistical model generated by the learning process, the language analysis process in the speech processing language analysis unit 41, the first acoustic An existing speech synthesis technique such as HTS (HMM-based speech synthesis system) can be used for the acoustic feature amount generation processing in the feature amount generation unit 43 and the second acoustic feature amount generation unit 45.

  FIG. 2 is a diagram for explaining acoustic feature amounts used in the present embodiment. In the figure, a speech waveform and a phoneme notation are shown in association with each other. From the speech waveform, a pitch (fundamental frequency) and a frequency spectrum (hereinafter referred to as “spectrum”) are obtained for each frame. Any conventional technique can be used as a method for acquiring a pitch and a frequency spectrum from a speech waveform. In the present embodiment, the frame length is 25 ms (milliseconds) and the frame shift is 5 ms.

  FIG. 3 is a diagram showing acoustic feature amounts used in the present embodiment. The acoustic feature amount shown in the figure is 153-dimensional information including static characteristics and dynamic characteristics, and is generally used in, for example, Non-Patent Document 1 and conventional techniques including HTS. The static characteristic of a certain frame is 51-dimensional information including a one-dimensional pitch and a 50-dimensional spectral coefficient obtained from the speech waveform of the frame. The acoustic feature quantity of the dynamic characteristic includes information on the primary difference (51 dimensions) of the static characteristics and the secondary difference (51 dimensions) of the static characteristics. The primary difference of the static characteristics of a certain frame is the difference between the static characteristics of that frame and the static characteristics of adjacent frames. The static difference secondary difference (51 dimension) of a frame is a difference between the primary difference of the frame and the primary difference of an adjacent frame.

  FIG. 4 is a diagram showing language feature amounts used in the present embodiment. From kanji-kana mixed sentences, accent phrase delimiters, exhalation paragraph delimiters, accent information, and part-of-speech information are obtained by morphological analysis. Further, the kanji-kana mixed text is converted into a phoneme notation, and then converted into a phoneme accent notation together with accent information obtained by morphological analysis. From the phoneme accent notation and the part-of-speech information obtained by morphological analysis, a context-dependent phoneme notation used as a language feature amount is obtained. This context-dependent phoneme notation is a language feature that is generally used in the prior art including HTS, for example.

  The context-dependent phoneme notation includes phoneme information, accent information, part-of-speech information, accent phrase information, exhalation paragraph information, and syllable number information of each time-series phoneme indicated by a single phoneme notation. The phoneme information indicates a sequence of five phonemes centering on the current phoneme. The accent information indicates the position in the accent phrase by mora. The part-of-speech information indicates the part-of-speech of the current word and the preceding and following words. The accent phrase information indicates the accent type of the current accent phrase, the preceding and following accent phrases, and the position of the current accent phrase. The exhalation paragraph information indicates the number of accent phrases and the number of mora in the current exhalation paragraph and the preceding and following exhalation paragraphs, and the position of the current exhalation paragraph. The syllable number information indicates the number of exhalation paragraphs, accent phrases, and mora.

Next, the operation of the voice processing device 1 will be described. Hereinafter, a case where the pre-conversion style is “calm” and the post-conversion style is “anger” will be described as an example.
FIG. 5 is a diagram illustrating a processing flow of pre-learning by the voice processing device 1.
First, the first voice storage unit 21 stores first learning voice data that is learning voice data with a style of “seduce”, and the second voice storage unit 25 has a style of “anger”. Second learning voice data, which is learning voice data, is stored. The first learning voice data and the second learning voice data are voice data when the same person reads out the sentence in the styles of “seduce” and “anger”, respectively. Sentences read in the “seduce” style and read-out sentences in the “anger” style may be the same or different. The acoustic feature quantity of each phoneme is affected by the phonemes before and after the phoneme. Therefore, it is desirable to use utterances of phoneme balance sentences including various phoneme arrangements in a well-balanced manner in the first learning voice data and the second learning voice data. For example, the phoneme balance 503 sentence proposed in the following references 1 and 2 can be used as a sentence to be read.

(Reference 1) Kenichi Tsuji, Takao Watanabe, Nao Kuwabara, “Designing a sentence set for speech database”, Sound lecture (Spring), p. 89-90, March 1988 (reference 2) Yoshinori Osaka, Noriyoshi Uraya, “ATR Speech / Language Database”, Acoustic Journal, Vol. 48, No. 12, p. 878-882, 1992

  The first acoustic analysis unit 22 reads out the first learning voice data to which the information of the style “calm” is added from the first voice storage unit 21. The first acoustic analysis unit 22 acquires the acoustic feature amount of each sentence frame from the speech waveform indicated by the read first learning speech data (step S110). The first learning language analysis unit 23 acquires a context-dependent phoneme notation from each sentence of the utterance content of the first learning speech data indicated by the first learning text data, and sets it as a language feature amount (step S120). The context-dependent phoneme notation obtained from the reading sentence indicated by the first learning text data may differ from the speech waveform when the sentence is actually read out, the position of the pose, the accent break, the accent position, and the like. Therefore, the context-dependent phoneme notation acquired by the first learning language analysis unit 23 is manually checked and corrected.

  The first statistical model learning unit 24 generates a statistical model by statistical model machine learning using the acoustic feature obtained in step S110 and the language feature obtained in step S120 for each sentence (step S130). For the statistical model machine learning, the techniques of Non-Patent Document 1 and Reference Document 3 can be used, but any conventional technique may be used. For example, the first statistical model learning unit 24 obtains the average, variance, transition probability between each state, and output probability of 153 dimensional acoustic feature values for each phoneme clustering unit, and performs a three-state HMM acoustic model by statistical model machine learning. Ask for. The first statistical model learning unit 24 adds information indicating the style “calm” to the generated statistical model including the acoustic model for each phoneme clustering unit and writes the information to the first statistical model storage unit 31.

(Reference 3) Takayoshi Yoshimura, 4 others, "Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis", in Proc. EUROSPEECH, p.2347-2350, 1999

  The second acoustic analysis unit 26 reads the second learning voice data to which the style “anger” information is added from the second voice storage unit 25. The second acoustic analysis unit 26 acquires the acoustic feature amount in units of frames of each sentence from the speech waveform indicated by the read second learning speech data (step S140). The second learning language analysis unit 27 performs context-dependent phoneme notation from each sentence of the utterance content of the second learning speech data indicated by the second learning text data by the same processing as the first learning language analysis unit 23. Acquired and used as a language feature amount (step S150). The context-dependent phoneme notation acquired by the second learning language analysis unit 27 is manually checked and corrected.

  The second statistical model learning unit 28 uses the acoustic feature obtained in step S140 and the language feature obtained in step S150 for each sentence, and performs a statistical model by machine learning similar to the first statistical model learning unit 24. Is generated (step S160). The second statistical model learning unit 28 adds information indicating the style “anger” to the generated statistical model and writes the information to the second statistical model storage unit 32.

Note that the audio processing device 1 may execute the process of step S110 and the process of step S120 in parallel or in a switched manner. Similarly, the speech processing apparatus 1 may execute the process of step S140 and the process of step S150 in parallel or in a switched manner. In addition, the voice processing device 1 may execute the processes of Steps S110 to S130 and the processes of Steps S140 to S160 in parallel, or may be executed by changing the order. Further, when there is one pre-conversion style and one post-conversion style, information indicating the style may not be added to the first learning audio data, the second learning audio data, and the statistical model.
In addition, when the sentence read out in the “seduce” style and the sentence read out in the “anger” style are the same, only the first learning text data or the second learning text data is stored in the speech processing apparatus 1. You may enter. The first learning language analysis unit 23 or the second learning language analysis unit 27 outputs the obtained language feature amount to the first statistical model learning unit 24 and the second statistical model learning unit 28.

FIG. 6 is a diagram illustrating a processing flow of the voice processing by the voice processing device 1.
The speech processing apparatus 1 receives input speech data, input speech text data indicating the utterance content of the input speech data, pre-conversion style data, and post-conversion style data. When the style of input audio data or the style of audio data desired to be obtained by conversion is determined in advance, the input of pre-conversion style data or post-conversion style data can be omitted. The speaker of the input voice data may be different from the speaker of the first learning voice data and the second learning voice data.

  The speech processing language analyzing unit 41 acquires the language feature amount of the sentence indicated by the input speech text data by the same processing as the first learning language analyzing unit 23 and the second learning language analyzing unit 27 (step S210). .

  The first statistical model selection unit 42 reads out the statistical model of the style “calm” indicated by the pre-conversion style data from the first statistical model storage unit 31 (step S220). The first acoustic feature value generation unit 43 uses the statistical model read by the first statistical model selection unit 42 and the language feature value output from the speech processing language analysis unit 41 as a time-series acoustic feature value. A certain first acoustic feature value generation value is generated (step S230).

  The second statistical model selection unit 44 reads out the statistical model of the style “anger” indicated by the converted style data from the second statistical model storage unit 32 (step S240). The second acoustic feature value generation unit 45 uses the statistical model read by the second statistical model selection unit 44 and the language feature value output from the speech processing language analysis unit 41 as a time-series acoustic feature value. A second acoustic feature value generation value is generated (step S250).

  The first corresponding frame detection unit 461 of the processing information generation unit 46 includes a first acoustic feature amount generation value generated by the first acoustic feature amount generation unit 43 and a second acoustic feature generated by the second acoustic feature amount generation unit 45. The quantity generation value is associated with each frame based on the similarity of values (step S260). For this association, for example, dynamic programming (DTW) is used. The processing information calculation unit 462 calculates the difference between the first acoustic feature value generation value and the second acoustic feature value generation value for each corresponding frame, and sets the difference as the acoustic feature value processing information (step S270).

  The audio processing acoustic analysis unit 47 acquires the acoustic feature quantity of the audio waveform indicated by the input audio data (step S280). The second corresponding frame detection unit 481 of the sound processing unit 48 uses the first sound feature value generation value generated by the first sound feature value generation unit 43 and the sound feature value acquired by the sound processing sound analysis unit 47. Corresponding in units of frames based on the similarity of values (step S290). The processing information adding unit 482 uses the first acoustic feature value generation value of the corresponding frame as the acoustic feature value of each frame acquired by the sound processing acoustic analysis unit 47, and the acoustic feature value generated by the processing information generation unit 46. Are added (step S300). The voice synthesizer 483 synthesizes the voice data of the acoustic feature amount of the input voice data to which the processing information is added to obtain processed voice data. The voice synthesizer 483 outputs the generated processed voice data as output voice data from the voice processing device 1 (step S310).

  Note that the audio processing device 1 may execute the processes of Steps S220 to S230 and the processes of Steps S240 to S250 in parallel or interchanged. In addition, the voice processing device 1 may execute the processing in steps S210 to S270 and the processing in step S280 in parallel or in replacement.

The voice processing shown in FIG. 6 will be described with reference to the data diagram.
FIG. 7 is a diagram for explaining an acoustic feature amount acquisition process from text in the speech processing apparatus 1. This figure shows the processing of step S210 to step S250 of FIG. By this process, the first acoustic feature value generation value and the second acoustic feature value generation value are generated from the kanji kana mixed text indicated by the input voice text data.

In step S210 of FIG. 6, the speech processing language analyzing unit 41 obtains a language feature amount of context-dependent phoneme notation from a kanji-kana mixed sentence indicated by input speech text data.
In step S230 of FIG. 6, the first acoustic feature value generation unit 43 connects the acoustic models of each phoneme clustering unit indicated by the statistical model of style “calm” according to the arrangement of the context-dependent phonemes. The first acoustic feature value generation unit 43 generates an acoustic feature value for each frame shift of 5 ms by selecting a combination that minimizes the connection probability. The acoustic feature value of each frame generated here is a 51-dimensional static characteristic composed of a 1-dimensional pitch and a 50-dimensional spectral coefficient. For example, the method of Non-Patent Document 1 can be used to generate the acoustic feature amount, but any existing technique for obtaining an acoustic feature amount from text data and an acoustic model may be used. The generated acoustic feature value is a first acoustic feature value generation value for each time-series frame whose style is “seduce”.

  Similarly, in step S250 of FIG. 6, the second acoustic feature value generation unit 45 obtains a second acoustic feature value generation value for each time-series frame whose style is “anger”. That is, the second acoustic feature value generation unit 45 connects the acoustic models of each phoneme clustering unit indicated by the statistical model of the style “anger” according to the arrangement of the context-dependent phonemes. The second acoustic feature value generation unit 45 selects a combination that minimizes the connection probability, and generates a 51-dimensional acoustic feature value for each 5 ms frame shift. The generated acoustic feature amount is a second acoustic feature amount generation value for each time-series frame whose style is “anger”.

  FIG. 8 is a diagram for explaining processing information generation processing in the voice processing device 1. This figure shows the processing from step S260 to step S270 in FIG. Through the processing shown in FIG. 7, the first acoustic feature value generation unit 43 generates a first acoustic feature value generation value for each 5 ms frame shift whose style is “Silence”, and the second acoustic feature value generation unit 45 A first acoustic feature value generation value is generated for each 5 ms frame shift whose style is “anger”. The first acoustic feature value generation value of the i-th frame (i is an integer of 1 or more) is described as Ai, and the second acoustic feature value generation value of the i-th frame (i is an integer of 1 or more) is described as Bi. To do. 6, the first corresponding frame detection unit 461 of the processing information generation unit 46 obtains the first acoustic feature value generation values A1, A2,... And the second acoustic feature value generation values B1, B2,. , Using a distance scale based on 50-dimensional spectral coefficients, and by using dynamic programming (DTW) or the like. The first corresponding frame detection unit 461 performs this association on the entire sentence.

  In step S270 of FIG. 6, the processing information calculation unit 462 calculates the difference between the first acoustic feature value generation value and the second acoustic feature value generation value of the associated frame and sets it as processing information. When the first acoustic feature value generation value of one frame and the second acoustic feature value generation values of a plurality of frames correspond to each other, the processing information calculation unit 462 corresponds to the second frame corresponding to the first acoustic feature value generation value. Processing information is generated on the assumption that there are as many acoustic feature value generation values as the number of frames. In addition, when the first acoustic feature value generation value of a plurality of frames and the second acoustic feature value generation value of one frame correspond to each other, the processing information calculation unit 462 has a first acoustic feature value generation value of the plurality of frames. For each of these, processing information is generated based on the difference from the second acoustic feature value generation value of the corresponding frame. The processing information of the i-th frame is described as Ci.

For example, since the first acoustic feature value generation value A1 and the second acoustic feature value generation value B1 correspond to each other, the processing information calculation unit 462 calculates the difference between them and sets it as the processing information C1. That is, the processing information C1 = second acoustic feature value generation value B1−first acoustic feature value generation value A1.
Since the first acoustic feature value generation value A2 corresponds to the second acoustic feature value generation value B2 and the second acoustic feature value generation value B3, the second acoustic feature value generation value B2 and the second acoustic feature value generation value It is assumed that there are two frames of first acoustic feature value generation values A2 corresponding to each of B3. The processing information calculation unit 462 calculates a difference between the first acoustic feature value generation value A2 and the second acoustic feature value generation value B2 to obtain processing information C2, and the first acoustic feature value generation value A2 and the second acoustic feature value. The difference from the generated value B3 is calculated and used as processing information C3. That is, processing information C2 = second acoustic feature value generation value B2—first acoustic feature value generation value A2 and processing information C3 = second acoustic feature value generation value B3—first acoustic feature value generation value A2.
Since the first acoustic feature value generation value A3 corresponds to the second acoustic feature value generation value B4, the processing information calculation unit 462 calculates a difference between them to be processing information C4. That is, the processing information C4 = second acoustic feature value generation value B4-first acoustic feature value generation value A3.
The first acoustic feature value generation value A4 and the first acoustic feature value generation value A5 correspond to the second acoustic feature value generation value B5. Therefore, the processing information calculation unit 462 calculates a difference between the first acoustic feature value generation value A4 and the second acoustic feature value generation value B5 as processing information C5, and the first acoustic feature value generation value A5 and the second acoustic feature value A5. The difference from the feature value generation value B5 is calculated and set as processing information C6. That is, processing information C5 = second acoustic feature value generation value B5-first acoustic feature value generation value A4, and processing information C6 = second acoustic feature value generation value B5-first acoustic feature value generation value A5.
The first acoustic feature value generation value A6 corresponds to the second acoustic feature value generation value B6 and the second acoustic feature value generation value B7. Therefore, processing information C7 = second acoustic feature value generation value B6-first acoustic feature value generation value A6, and processing information C8 = second acoustic feature value generation value B7-first acoustic feature value generation value A6.
The machining information calculation unit 462 generates machining information C1, C2,... By repeating the same processing.

FIG. 9 is a diagram for explaining input voice processing in the voice processing apparatus 1. This figure shows the processing from step S280 to step S300 in FIG.
In step S280 in FIG. 6, the sound processing acoustic analysis unit 47 obtains an acoustic feature amount having a frame length of 25 ms and a frame shift of 5 ms from the input speech data whose style is “serious”. The acoustic feature value obtained here is a 51-dimensional static characteristic composed of a 1-dimensional pitch and a 50-dimensional spectral coefficient. The acoustic feature quantity of the i-th frame obtained from the input speech data is described as acoustic feature quantity Di.

  In step S290 of FIG. 6, the second corresponding frame detection unit 481 of the audio processing unit 48 receives the acoustic feature amounts D1, D2,... Of the input audio data and the first acoustic feature amount generation values A1, A2,. Similarly to the first corresponding frame detection unit 461, the correspondence is made by dynamic programming (DTW) or the like using a distance scale based on 50-dimensional spectral coefficients.

  Note that the correspondence between frames is performed for the entire sentence, but may be performed for each phoneme. In the case of performing correspondence for each phoneme, it is determined which part of speech corresponds to which phoneme by performing alignment processing using the input speech and the phoneme string.

  In step 300 of FIG. 6, the processing information adding unit 482 adds the acoustic feature quantity generated from the first acoustic feature quantity generation value associated with the acoustic feature quantity to the acoustic feature quantity of each frame of the input voice data. Add processing information. When the acoustic feature quantity of one frame of the input voice data corresponds to the first acoustic feature quantity generation values of the plurality of frames, the processing information adding unit 482 corresponds to the acoustic feature quantities of the input voice data. The average of the processing information generated from each of the acoustic feature value generation values is added. Further, when the acoustic feature values of a plurality of frames of the input audio data correspond to the first acoustic feature value generation value of one frame, the processing information adding unit 482 corresponds to each of the acoustic feature values of the plurality of frames. The processing information generated from the first acoustic feature value generation value to be added is added. The acoustic feature quantity of the i-th frame obtained by processing is described as Ei.

For example, since the acoustic feature quantity D1 and the first acoustic feature quantity generation value A1 correspond, the processing information adding unit 482 adds the processing information C1 generated from the first acoustic feature quantity generation value A1 to the acoustic feature quantity D1. These are added to obtain the acoustic feature quantity E1. That is, acoustic feature amount E1 = acoustic feature amount D1 + processing information C1.
The acoustic feature amounts D2 and D3 correspond to the first acoustic feature amount generation value A2, and processing information C2 and C3 are generated from the first acoustic feature amount generation value A2. Therefore, the processing information adding unit 482 adds the first processing information C2 generated using the first acoustic feature value generation value A2 to the acoustic feature amount D2 to obtain the acoustic feature amount E2, and the acoustic feature amount D3. The second processing information C3 generated using the first acoustic feature value generation value A2 is added to obtain an acoustic feature value E3. That is, acoustic feature amount E2 = acoustic feature amount D2 + processing information C2, and acoustic feature amount E3 = acoustic feature amount D3 + processing information C3.
Since the acoustic feature amount D4 corresponds to the first acoustic feature amount generation value A3, the processing information adding unit 482 adds the processing information C4 generated using the first acoustic feature amount generation value A3 to the acoustic feature amount D4. These are added to obtain an acoustic feature quantity E4. That is, acoustic feature amount E4 = acoustic feature amount D4 + processing information C4.
Since the acoustic feature amount D5 corresponds to two frames of the first acoustic feature amount generation values A4 and A5, the processing information adding unit 482 generates the acoustic feature amount D5 from the first acoustic feature amount generation value A4. The average of the processed information C5 and the processed information C6 generated from the first acoustic feature amount generation value A5 is added to obtain an acoustic feature amount E5. That is, acoustic feature amount E5 = acoustic feature amount D5 + Avg (processing information C5 + processing information C6). Avg (x + y) represents an average of x and y.
The acoustic feature amount D6 corresponds to the first acoustic feature amount generation value A6, and processing information C7 and processing information C8 are generated from the first acoustic feature amount generation value A6. The processing information adding unit 482 adds the average of the processing information C7 and the processing information C8 to the acoustic feature amount D6 to obtain an acoustic feature amount E6. That is, acoustic feature amount E6 = acoustic feature amount D6 + Avg (processing information C7 + processing information C8).
The processing information calculation unit 462 repeats the same processing, and changes the time-series acoustic feature values of the input voice data to the acoustic feature values E1, E2,.

  In step S310 in FIG. 6, the speech synthesizer 483 synthesizes speech data composed of the acoustic feature amounts E1, E2,..., And obtains processed speech data obtained by converting the pitch and spectrum of the input speech data. For this conversion, for example, the methods of Patent Documents 1 and 2 can be used, but any conventional speech synthesis technique may be used. The speech synthesis unit 483 compresses the processed speech data so that the time length corresponds to the number of frames of the second acoustic feature value generation value, and outputs the compressed speech data as output speech data. As a result, a speech waveform style-converted into an “anger” emotion can be obtained.

  FIG. 10 is a diagram showing input voice data and processed voice data that have been style-converted using the voice processing apparatus 1. In the figure, the pitch, spectrum, phoneme label, and speech waveform are shown in order from the top. The horizontal axis represents time, and the vertical axis represents frequency in the pitch and spectrum, and volume in the speech waveform, each representing a change over time.

  In the above-described embodiment, the pitch and the spectrum are used for the acoustic feature amount, but only one of them may be used.

  Also, the emotional expression varies from speaker to speaker. Therefore, a statistical model of the pre-conversion style and the post-conversion style may be created for each speaker, and it may be specified which of the speaker's learning speech data is used in the speech processing. Thereby, it is possible to specify which speaker's emotional expression is to be added to the input voice data.

FIG. 11 is a diagram illustrating specifications of a subjective evaluation experiment performed on the speech processing apparatus 1.
Ten sentences whose sentence meanings were determined to be emotionless by a prior experiment were read out in a “seduce” style and used as input speech data. The input voice data was converted into “joy”, “anger”, and “sorrow” styles by the voice processing device 1 and processed into voice data of 30 sentences. The test subjects were five men and five women, and the converted voice data was presented to the test subjects by speakers in a general laboratory.
The experimental method is as follows. That is, it selected at random from all the audio | voice data of 30 sentences processed with the audio | voice processing apparatus 1, and showed it to the test subject. The test subject expressed the emotional expression of the presented voice data based on the six emotions of “joy”, “surprise”, “anger”, “disgust”, “sadness”, “fear” ”And“ Unknown ”were selected from eight categories. Ten subjects judged about one voice data.

FIG. 12 is a diagram showing determination emotions for the learning data used to generate the statistical model used in the subjective evaluation experiment performed on the speech processing apparatus 1. The speaker of the learning voice data is the same as the speaker of the input voice data.
Regarding the voice data used as the first learning voice data in order to generate the statistical model of the style “no emotion”, 85.65% of the subjects determined that the style is “no emotion”.
In addition, regarding the voice data used as the second learning voice data in order to generate the statistical model of the style “joy”, 80.92% of the subjects determined that the style was “joy”.
Similarly, regarding the voice data used as the second learning voice data to generate the statistical model of the style “anger”, 78.00% of the subjects determined that the style is “anger”.
Of the voice data used as the second learning voice data to generate the statistical model of the converted style “sorrow”, 61.12% of the subjects determined that the style was “sorrow (sorrow)”. there were.
Thus, the rate at which the style was correctly determined for the learning data was 61.12% to 85.65%.

FIG. 13 is a diagram illustrating an evaluation result of a subjective evaluation experiment performed on the audio processing device 1.
The voice processing apparatus 1 uses the statistical model of the style “no emotion” before conversion and the statistical model of the style “joy” after conversion to the voice data obtained by converting the input voice data of the style “calm”. In contrast, 49% of subjects judged the style to be “joy”.
Further, the voice processing apparatus 1 uses the statistical model of the style “no emotion” before conversion and the statistical model of the style “anger” after conversion, and the voice obtained by converting the input voice data of the style “calm” For the data, 71.0% of the subjects determined the style to be “angry”.
In addition, the voice processing device 1 uses the statistical model of the style “no emotion” before conversion and the statistical model of the style “sorrow” after conversion, and the voice obtained by converting the input voice data of the style “calm” For the data, 77.0% of subjects determined the style to be “sorrow”.
According to the above, the correct answer rate was 49.0% for “joy”, 71.0% for “anger” and 77.0% for “sorrow”, and the average correct answer rate was 65.7%. .
Thus, the correct answer rate of the subjective evaluation when the emotion extracted from the same speaker is given to the voice uttered by a specific speaker is 60% or more, and the effectiveness of this embodiment is confirmed.

  According to the embodiment described above, the voice processing apparatus 1 can perform voice style conversion easily and accurately.

  Note that the above-described speech processing apparatus 1 has a computer system therein. The operation process of the sound processing apparatus 1 is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer system reading and executing this program. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

DESCRIPTION OF SYMBOLS 1 Speech processing apparatus 2 Learning part 3 Storage part 4 Speech processing part 21 First speech storage part 22 First acoustic analysis part 23 First learning language analysis part 24 First statistical model learning part 25 Second speech storage part 26 Second Acoustic analysis unit 27 Second learning language analysis unit 28 Second statistical model learning unit 31 First statistical model storage unit 32 Second statistical model storage unit 41 Language processing unit for speech processing (language analysis unit)
42 1st statistical model selection part 43 1st acoustic feature-value production | generation part 44 2nd statistical model selection part 45 2nd acoustic feature-value production | generation part 46 Processing information generation part 47 Sound analysis acoustic analysis part (acoustic analysis part)
48 voice processing unit 461 first corresponding frame detection unit 462 processing information calculation unit 481 second corresponding frame detection unit 482 processing information addition unit 483 voice synthesis unit

Claims (5)

A language analysis unit for acquiring a language feature of a sentence indicated by text data;
Based on the language feature amount acquired by the language analysis unit and a statistical model related to the acoustic feature amount generated from the speech data of the first style utterance, a time-series frame-wise acoustic feature amount is generated. An acoustic feature generation unit;
Based on the language feature acquired by the language analysis unit and a statistical model related to the acoustic feature generated from the speech data of the second style utterance, a time-series frame-based acoustic feature is generated. A two-acoustic feature generation unit;
A first acoustic feature value generation value that is the acoustic feature value generated by the first acoustic feature value generation unit, and a second acoustic feature value generation value that is the acoustic feature value generated by the second acoustic feature value generation unit. The first acoustic feature value generation value frame and the second acoustic feature value generation value frame are associated with each other based on the similarity to the first acoustic feature value generation value. A processing information generating unit that generates processing information based on a difference between the value and the second acoustic feature value generation value;
An acoustic analysis unit that acquires acoustic features in units of time-series frames from the voice data of the sentence indicated by the text data;
Based on the similarity between the acoustic feature quantity acquired by the acoustic analysis unit and the first acoustic feature quantity generation value, the acoustic feature quantity frame and the first acoustic feature quantity generation value frame are associated with each other. An audio processing unit that processes the acoustic feature amount of each frame based on the processing information generated by the processing information generation unit using the first acoustic feature amount generation value of the corresponding frame;
An audio processing apparatus comprising: The acoustic feature amount includes at least one of information on pitch and information on frequency spectrum.
The speech processing apparatus according to claim 1. The style of the voice data of the sentence indicated by the text data is the first style.
The speech processing apparatus according to claim 1 or 2, wherein The voice processing unit synthesizes voice data based on the processed acoustic feature amount.
The speech processing apparatus according to any one of claims 1 to 3, wherein Computer
Language analysis means for acquiring a language feature of a sentence indicated by text data;
Based on the language feature acquired by the language analysis means and a statistical model related to the acoustic feature generated from the speech data of the first style utterance, a time-series frame-wise acoustic feature is generated. One acoustic feature generation means;
Based on the language feature acquired by the language analysis means and a statistical model related to the acoustic feature generated from the speech data of the second style utterance, a time-series frame-wise acoustic feature is generated. Two acoustic feature generation means;
The first acoustic feature value generation value that is the acoustic feature value generated by the first acoustic feature value generation unit, and the second acoustic feature value generation value that is the acoustic feature value generated by the second acoustic feature value generation unit. The first acoustic feature value generation value frame and the second acoustic feature value generation value frame are associated with each other based on the similarity to the first acoustic feature value generation value. Processing information generating means for generating processing information by the difference between the value and the second acoustic feature value generation value;
Acoustic analysis means for acquiring acoustic features in units of time-series frames from voice data of the sentence indicated by the text data;
Based on the similarity between the acoustic feature quantity acquired by the acoustic analysis means and the first acoustic feature quantity generation value, the acoustic feature quantity frame and the first acoustic feature quantity generation value frame are associated with each other. Voice processing unit for processing the acoustic feature amount of each frame based on the processing information generated by the processing information generation unit using the first acoustic feature amount generation value of the corresponding frame;
A program for causing a voice processing apparatus to function.