JP4826493B2 - Speech synthesis dictionary construction device, speech synthesis dictionary construction method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To construct a speech synthesis dictionary capable of generating synthesized speech which gives natural impression, since pitch variations are moderately large. <P>SOLUTION: A speech synthesis dictionary constructing apparatus temporarily makes a preliminary speech synthesis dictionary 223, by performing phoneme HMM (hidden Markov Model) learning which is a known processing that uses a speech database 221. Then, based on the analysis and the comparison result of the synthesized speech created via the temporary speech synthesis dictionary 223 and speech collected in the speech database 221, an editing method of pitch sequence data for emphasizing the pitch fluctuations base formants of the speech is determined. Ultimately, the apparatus performs phoneme HMM learning again, by first including the editing process beforehand, where the editing method is adopted, and thereby the speech synthesis dictionary 227 is constructed. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to a speech synthesis dictionary construction device, a speech synthesis dictionary construction method, and a program for constructing a speech synthesis dictionary used for speech synthesis and the like.

  Speech recognition technology and speech synthesis technology based on a Hidden Markov Model (hereinafter referred to as HMM) are widely used as speech recognition and speech synthesis technology.

  A speech recognition technique and a speech synthesis technique based on the HMM are disclosed in, for example, Patent Document 1.

JP 2002-268660 A

  In speech synthesis based on the HMM, a speech synthesis dictionary in which a correspondence relationship between phoneme labels and spectrum parameter data strings is recorded is required.

  The speech synthesis dictionary is constructed by a speech synthesis dictionary construction device. The speech synthesis dictionary construction apparatus generally uses a melody for data recorded in a database (hereinafter referred to as a speech database) configured from a set of speech data, phoneme monophone label data, and phoneme triphone label data. A speech synthesis dictionary is constructed by cepstrum analysis and pitch extraction, and through a learning process based on HMM.

  When a conventional speech synthesis dictionary construction device constructs a speech synthesis dictionary, the pitch sequence data generated as a result of pitch extraction is used as it is for learning based on the HMM without any particular processing or the like. Was building.

  However, the pitch variation of the synthesized speech generated using the speech synthesis dictionary constructed as described above is smaller than the pitch variation of the original speech.

  For this reason, the synthesized speech using the speech synthesis dictionary constructed by the conventional speech synthesis dictionary construction device is unnatural that gives a flat impression as compared with the natural speech of human beings.

  The present invention has been made in view of the above circumstances, and includes a speech synthesis dictionary construction device, a speech synthesis dictionary construction method, and a program capable of constructing a speech synthesis dictionary capable of synthesizing speech that gives a natural impression. The purpose is to provide.

In order to achieve the above object, a speech synthesis dictionary construction device according to the first aspect of the present invention provides:
A phoneme label string and recorded voice data corresponding to the phoneme label string are acquired from the voice database, and the recorded voice pitch series data is extracted from the acquired recorded voice data, and the extracted recorded voice pitch series data and the acquired phoneme are extracted. A temporary construction unit that constructs a temporary speech synthesis dictionary by HMM (Hidden Markov Model) learning based on the label sequence;
A synthesized data generation unit that relies on the temporary speech synthesis dictionary to generate synthesized speech data and extracts synthesized speech pitch series data from the generated synthesized speech data;
The recorded speech pitch sequence data extracted by the temporary construction unit from the recorded speech data corresponding to the phoneme label sequence, and the synthesized data generation unit extracted from the synthesized speech data corresponding to the phoneme label sequence Based on the result of comparing the synthesized voice pitch series data, an editing unit that edits the recorded voice pitch series data to generate edited pitch series data;
A reconstructing unit that constructs a speech synthesis dictionary by HMM learning based on the phoneme label string and the edited pitch sequence data generated by the editing unit;
Is provided.

  The pitch extracted from the original natural speech is compared with the pitch extracted from the synthesized speech, which is a flat and unnatural speech once synthesized through the temporary speech synthesis dictionary. According to such a comparison, it is naturally clarified what processing should have been performed on the original voice data in order to prevent the synthesized voice from becoming such unnatural voice. More specifically, it is possible to efficiently and easily determine a policy on how much it is appropriate to increase the pitch variation of the original audio data. The speech synthesis dictionary reconstructed based on the speech data subjected to such adjustment contributes to the generation of synthesized speech that gives a natural impression.

The speech synthesis dictionary construction device
A plurality of audio data, a monophone label generated for each of the audio data, a start point pointer and an end point pointer indicating the time corresponding to the start point and end point of the monophone label, and a triphone label generated for each of the audio data Receiving, extracting pitch sequence data from the audio data, generating mel cepstrum coefficient sequence data up to a predetermined order from the audio data, the monophone label, the start point pointer, the end point pointer, the triphone label, and the A first learning unit that constructs a temporary speech synthesis dictionary from pitch sequence data and the mel cepstrum coefficient sequence data by HMM (Hidden Markov Model) learning;
A synthesis unit that generates a plurality of synthesized speech data based on the temporary speech synthesis dictionary and the triphone label;
Editing and editing the pitch sequence data according to an editing policy determined based on the result of comparing the synthesized speech pitch sequence data extracted from the synthesized speech data and the pitch sequence data extracted by the first learning unit An editing unit for generating pitch series data;
A second learning unit that constructs a speech synthesis dictionary by HMM (Hidden Markov Model) learning from the monophone label, the start point pointer, the end point pointer, the triphone label, the edited pitch sequence data, and the mel cepstrum coefficient sequence data When,
May be provided.

  The editing unit generates a synthesized monophone label and a synthesized start point pointer and a synthesized end point pointer indicating a time corresponding to a start point and an end point of the synthesized monophone label for each synthesized voice data, and the synthesized voice pitch sequence data Editing determined based on a result of comparing the pitch series data with reference to the composite monophone label, the composite start point pointer, the composite end point pointer, the monophone label, the start point pointer, and the end point pointer The edited pitch sequence data may be generated by editing the pitch sequence data according to a policy.

  Considering the monophonic label data of each of the recorded voice and synthesized voice, it becomes possible to compare the pitches of both voices in units of phoneme labels, so that an appropriate editing policy can be determined finely.

  It is preferable that the editing unit generates the edited pitch series data by increasing a pitch variation of the pitch series data.

  The reason why the conventional synthesized speech gives a flat and unnatural impression is thought to be because the pitch fluctuation is small. Therefore, if the recorded voice is processed in advance so that the pitch fluctuation is increased, the pitch fluctuation of the synthesized voice is also increased, and a synthesized voice giving a more natural impression is generated.

  It is desirable that the editing unit generates the edited pitch series data by enlarging the pitch series data around the reference pitch level with a predetermined pitch level as a reference pitch level.

  For example, the editing unit generates the edited pitch series data by enlarging the pitch series data around the reference pitch level with an average value of the pitch series data as a reference pitch level.

  In this way, it is possible to increase the pitch variation while keeping the pitch average constant.

  The editing unit may generate the edited pitch series data by, for example, enlarging the pitch series data around the reference pitch level with a zero level of the pitch series data as a reference pitch level.

  Even in this case, the pitch fluctuation can be increased.

  The editing unit obtains, for example, an average pitch for each voice that is an average value of the pitch series data for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data for each synthesized voice data For each voice data, a voice-specific pitch difference maximum absolute value that is the maximum absolute value of the difference between the pitch series data and the voice-specific average pitch is obtained, and the synthesized voice pitch series is obtained for each synthesized voice data. The maximum absolute value of the pitch difference for each synthesized voice, which is the maximum absolute value of the difference between the data and the average pitch for each synthesized voice, is obtained, and the voice that is the maximum absolute value of the pitch difference for each voice in all the voice data Obtain the maximum absolute value of the total pitch difference, and determine the maximum absolute value of the synthesized speech total pitch difference, which is the maximum value of the pitch difference maximum absolute value for each synthesized speech in all the synthesized speech data. A total magnification for editing, which is a value obtained by dividing the maximum absolute value of the voice total pitch difference by the maximum absolute value of the synthesized voice total pitch difference, and the pitch series data around the reference pitch level as the total magnification for editing. The edited pitch sequence data is generated by enlarging.

  Such a voice total pitch difference maximum absolute value and a synthesized voice total pitch difference maximum absolute value can be said to be indices representing the degree of pitch fluctuation of the recorded voice and the degree of pitch fluctuation of the synthesized voice, respectively. Therefore, if a value obtained by dividing the former absolute value by the latter absolute value is adopted as the pitch fluctuation expansion rate, the pitch fluctuation is appropriately expanded.

  The editing unit obtains, for example, an average pitch for each voice that is an average value of the pitch series data for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data for each synthesized voice data Obtaining the maximum pitch difference for each voice, which is the maximum value obtained by subtracting the average pitch for each voice from the pitch series data for each voice data, and calculating the average for each voice from the pitch series data for each voice data. Obtain a minimum pitch difference for each voice, which is the minimum value obtained by subtracting the pitch, obtain a maximum voice total pitch difference, which is a maximum value of the maximum pitch difference for each voice in all the voice data, and determine all the voices. A voice total pitch difference minimum value, which is a minimum value of the voice-specific pitch difference minimum values in the data, is obtained, and the synthesized voice pitch sequence data is pre- The maximum value of the pitch difference for each synthesized speech, which is the maximum value obtained by subtracting the average pitch for each synthesized speech, and the minimum value obtained by subtracting the average pitch for each synthesized speech from the synthesized speech pitch sequence data for each synthesized speech data The synthesized speech-specific pitch difference minimum value is obtained, and the synthesized speech overall pitch difference maximum value, which is the maximum value of the synthesized speech-specific pitch differences in all the synthesized speech data, is determined, and the synthesized speech data in all the synthesized speech data The minimum synthesized speech total pitch difference minimum value, which is the minimum pitch difference for each synthesized speech, is obtained, and the upper overall magnification for editing, which is a value obtained by dividing the maximum speech total pitch difference by the maximum synthesized speech total pitch difference, is obtained. An editing lower overall magnification, which is a value obtained by dividing the voice total pitch difference minimum value by the synthesized voice total pitch difference minimum value, is obtained, and the reference pitch of the pitch series data is obtained. Those above the reference pitch level are enlarged at the upper overall magnification for editing, and those below the reference pitch level among the pitch series data are used for the editing around the reference pitch level. The edit pitch series data is generated by enlarging with the lower overall magnification.

  By adopting different pitch enlargement rates depending on whether the pitch series data is above or below the reference pitch level, the pitch variation is further appropriately enlarged.

  The editing unit obtains, for example, an average pitch for each voice that is an average value of the pitch series data for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data for each synthesized voice data For each voice data, a voice-specific pitch difference maximum absolute value that is the maximum absolute value of the difference between the pitch series data and the voice-specific average pitch is obtained, and the synthesized voice pitch series is obtained for each synthesized voice data. The maximum absolute value of the pitch difference for each synthesized speech, which is the maximum absolute value of the difference between the data and the average pitch for each synthesized speech, is determined, and the maximum absolute value of the pitch difference for each speech is associated with the speech data. An editing voice-specific magnification obtained by dividing the synthesized voice data by the synthesized voice-specific pitch difference maximum absolute value is obtained, and the pitch sequence data is centered on the reference pitch level for each voice data. Generating the edited pitch series data by enlarging the data in the editing voice by magnification.

  Since the pitch editing process is completed for each audio data, the pitch variation is expanded according to the characteristics of each audio data, and fine and appropriate editing is performed.

  The editing unit obtains, for example, an average pitch for each voice that is an average value of the pitch series data for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data for each synthesized voice data Obtaining the maximum pitch difference for each voice, which is the maximum value obtained by subtracting the average pitch for each voice from the pitch series data for each voice data, and calculating the average for each voice from the pitch series data for each voice data. A minimum pitch difference for each voice, which is a minimum value obtained by subtracting the pitch, is obtained, and for each synthesized voice data, each synthesized voice data is a maximum value obtained by subtracting the average pitch for each synthesized voice from the synthesized voice pitch sequence data. A maximum pitch difference value is obtained, and for each synthesized voice data, the synthesized voice pitch is a minimum value obtained by subtracting the synthesized voice average pitch from the synthesized voice pitch sequence data. H for each voice data, and for each voice data, the maximum pitch difference for editing is obtained by dividing the maximum value for each voice pitch difference by the maximum pitch difference for each synthesized voice corresponding to the voice data. Obtaining the lower voice-specific magnification for editing by dividing the voice-specific pitch difference minimum value for each voice data by the synthesized voice-specific pitch difference minimum value of the synthesized voice data corresponding to the voice data; The pitch series data that exceeds the reference pitch level is enlarged at the magnification for each upper voice for editing around the reference pitch level, and the pitch series data that is below the reference pitch level The edit pitch series data is generated by enlarging the reference pitch level around the reference pitch level with the lower voice editing magnification.

  Since the pitch fluctuation expansion rate is changed between the upper side and the lower side of the reference pitch level and in units of audio data, fine and appropriate editing is performed.

  The editing unit, for example, for the pitch sequence data specified by the audio data and the monophone label for each audio data from which the pitch sequence data to be edited is extracted and for each monophone label. A value obtained by dividing the averaged result from the start time to the end time of the phone label by the averaged result from the start time to the end time of the synthesized monophone label equal to the monophone label for the synthesized pitch series data is obtained, The editing monophone magnification for each audio data and each monophone label is set, and the editing is performed by multiplying the pitch series data by the magnification for each editing monophone for each audio data and the monophone label of the extraction source. Generate pitch series data.

  Since the pitch series data is edited in units of phoneme labels, which is a smaller unit than the voice data, further fine and appropriate editing is performed.

In order to achieve the above object, a speech synthesis dictionary construction method according to a second aspect of the present invention includes:
A phoneme label string and recorded voice data corresponding to the phoneme label string are acquired from the voice database, and the recorded voice pitch series data is extracted from the acquired recorded voice data, and the extracted recorded voice pitch series data and the acquired phoneme are extracted. A temporary construction step of constructing a temporary speech synthesis dictionary by HMM (Hidden Markov Model) learning based on the label sequence;
Generating a synthesized voice data based on the provisional voice synthesis dictionary, and extracting a synthesized voice pitch series data from the generated synthesized voice data; and
The recorded voice pitch sequence data extracted by the temporary construction step from the recorded voice data corresponding to the phoneme label string and the synthesized data generation step extracted from the synthesized voice data corresponding to the phoneme label string An editing step of editing the recorded voice pitch series data to generate edited pitch series data based on the result of comparing the synthesized voice pitch series data;
A reconstructing step of constructing a speech synthesis dictionary by HMM learning based on the phoneme label string and the editing pitch sequence data generated by the editing step;
Consists of

In order to achieve the above object, a computer program according to the third aspect of the present invention provides:
On the computer,
A phoneme label string and recorded voice data corresponding to the phoneme label string are acquired from the voice database, and the recorded voice pitch series data is extracted from the acquired recorded voice data, and the extracted recorded voice pitch series data and the acquired phoneme are extracted. A temporary construction step of constructing a temporary speech synthesis dictionary by HMM (Hidden Markov Model) learning based on the label sequence;
Generating a synthesized voice data based on the provisional voice synthesis dictionary, and extracting a synthesized voice pitch series data from the generated synthesized voice data; and
The recorded voice pitch sequence data extracted by the temporary construction step from the recorded voice data corresponding to the phoneme label string and the synthesized data generation step extracted from the synthesized voice data corresponding to the phoneme label string An editing step of editing the recorded voice pitch series data to generate edited pitch series data based on the result of comparing the synthesized voice pitch series data;
A reconstructing step of constructing a speech synthesis dictionary by HMM learning based on the phoneme label string and the editing pitch sequence data generated by the editing step;
Is executed.

  According to the present invention, a temporary speech synthesis dictionary is once constructed, speech is synthesized based on the dictionary, and the speech is compared with the original speech. Therefore, the editing process to be performed on the pitch sequence data related to the original voice in order to fill the difference between the two voices from the viewpoint of pitch fluctuation is easily and accurately determined. Then, since the speech synthesis dictionary is reconstructed based on the speech thus processed, it is finally possible to construct a speech synthesis dictionary that contributes to the generation of synthesized speech that gives a natural impression that is not flat.

  Hereinafter, the speech synthesis dictionary construction device according to the embodiment of the present invention will be described in detail. The embodiment will be described separately in the first embodiment and the second embodiment. Of these, the first embodiment will be described in more detail by way of editing specific example 1, a modification of editing specific example 1, a specific example 2 of editing, and a modification of specific example 2 of editing.

(Embodiment 1)
2 to 4 show the functional configuration of the speech synthesis dictionary construction device according to Embodiment 1 of the present invention.

  The speech synthesis dictionary construction device according to Embodiment 1 of the present invention includes a first learning unit 111 (FIG. 2), a first speech synthesis dictionary 223 (FIG. 2), a synthesis unit 113 (FIG. 3), and a second learning. And a unit 117 (FIG. 4).

  The speech synthesis dictionary construction device is a device for constructing the second speech synthesis dictionary 227 (FIG. 4) based on the first speech database 221 (FIG. 2).

  The first voice database 221 (FIG. 2) is a well-known voice database. Here, a large number of sets of voice data, monophone label data, and triphone label data obtained by recording a voice of a person who reads out a predetermined sentence are stored. For each piece of audio data identified by the counter m, there is monophone label data and triphone label data corresponding to the audio data. In order to facilitate understanding of this situation, the procedure from the state in which only the voice data is stored in the voice database to the completion of the voice database after the label data is created will be described with reference to FIG.

  In order to create the label data and complete the voice database, for example, a general computer device as described later with reference to FIG. 5 is used. In other words, for example, it has an interface for accessing an audio database that exists as a removable hard disk, and performs a predetermined process by loading data from the removable hard disk, and temporarily holds the result of the process. A device having a function of storing in a removable hard disk or the like is used.

It is assumed that N _Sp speech data Sp _m (1 ≦ m ≦ N _Sp ) is stored in the incomplete speech database.

In pitch extraction and mel cepstrum analysis described below, a certain length of time frame is set for the sound data, and the time frame is set at a predetermined period (frame period) so that the time frames overlap. By processing while shifting the pitch sequence data and mel cepstrum coefficient sequence data at each point in time, the symbol fm (0 ≦ fm ≦ N _fm [m]) is the number of this frame period. This represents a number indicating.

  First, the above-described computer device has a voice data identification counter m therein, and initializes m = 1 (step S11 in FIG. 1).

The computer device loads audio data Sp _m from an incomplete audio database, and monophon label data MLabData _m [ml] (1 ≦ ml ≦ ML _Sp [m]) from the audio data by any known method. Is generated (step S13). Here, ML _Sp [m] is the number of monophone labels included in the audio data Sp _m .

The monophone label data MLabData _m [ml] is a pointer indicating the monophone label MLab _m [ml] and the time corresponding to the start point and the end point of the monophone label in the duration of the audio data Sp _m by the frame cycle number. A start frame MFrameS _m [ml] and an end frame MFrameE _m [ml].

The monophone label data MLabData _m [ml] is stored in the voice database (step S15).

Subsequently, the computer apparatus obtains triphone label data TLabData _m [tl] (1 ≦ tl ≦ TL _Sp [m]) from the audio data Sp _m that remains loaded by any known method. Generate (step S17). Here, the triphone label data is the triphone label itself, and TL _Sp [m] is the number of triphone labels included in the audio data Sp _m .

The triphone label data TLabData _m [tl] is stored in the voice database (step S19).

Subsequently, it is determined whether m has reached N _Sp (step S21). If it is determined that it has not been reached (step S21; No), m is incremented by 1 (step S23), and then the process returns to step S13.

When the processing is completed, the monophonic label data MLabData _m [ml] and the triphone label data TLabData _m [tl] for all the audio data Sp _m are stored in the audio database. In this way, the speech database is completed.

The first learning unit 111 (FIG. 2) of the speech synthesis dictionary construction device according to the first exemplary embodiment of the present invention uses the speech data Sp _m (1 ≦ 1) from the first speech database 221 that is the speech database completed as described above. m ≦ N _Sp ), monophone label data MLabData _m [ml] (1 ≦ ml ≦ ML _Sp [m]), triphone label data TLabData _m [tl] (1 ≦ tl ≦ TL _Sp [m]), , Get. Then, the first learning unit 111 constructs the first speech synthesis dictionary 223, which is a speech synthesis dictionary used for generating synthesized speech, by phoneme HMM learning, which is a known method. The contents stored in the first speech synthesis dictionary 223 will be referred to as a first learning result.

  The first learning unit 111 includes a first pitch extraction unit 311, a mel cepstrum analysis unit 313, and a first phoneme HMM learning unit 315.

The first pitch extraction unit 311 receives the audio data Sp _m (1 ≦ m ≦ N _Sp ) from the first audio database 221, and pitch sequence data Pit _m [fm] from the m-th audio data by any known method. Is generated and delivered to the first phoneme HMM learning unit 315 and a second learning unit 117 (FIG. 4) described later.

The mel cepstrum analysis unit 313 (FIG. 2) receives the audio data Sp _m (1 ≦ m ≦ N _Sp ) from the first audio database 221 and performs a D-th order mel cepstrum analysis which is a known method on the audio data. Apply. As a result, the mel cepstrum analysis unit 313, for all frames fm (0 ≦ fm ≦ N _fm [m]) of the m-th speech data, mel cepstrum coefficient series data MC _m ^d [fm ] (0 ≦ d ≦ D) is generated and delivered to the first phoneme HMM learning unit 315 and the second learning unit 117 (FIG. 4) described later.

The first phoneme HMM learning unit 315 (FIG. 2) obtains monophone label data MLabData _m [ml] (1 ≦ m ≦ N _Sp , 1 ≦ ml ≦ ML _Sp [m]) and triphone labels from the first speech database 221. Data TLabData _m [tl] (1 ≦ m ≦ N _Sp , 1 ≦ tl ≦ TL _Sp [m]) is received. The first phoneme HMM learning unit 315 also receives pitch sequence data Pit _m [fm] (1 ≦ m ≦ N _Sp , 0 ≦ fm ≦ N _fm [m]) from the first pitch extraction unit 311, and a mel cepstrum analysis unit From 313, mel cepstrum coefficient series data MC _m ^d [fm] (1 ≦ m ≦ N _Sp , 0 ≦ d ≦ D, 0 ≦ fm ≦ N _fm [m]) is received. The first phoneme HMM learning unit 315 generates a first learning result that is a learning result from the received data by phoneme HMM learning that is a known method, and stores the first learning result in the first speech synthesis dictionary 223. More precisely, the first learning result is stored in an empty database, whereby the empty database is completed as the first speech synthesis dictionary 223.

  The synthesis unit 113 illustrated in FIG. 3 includes a phoneme HMM sequence generation unit 321, a time series data generation unit 323, an excitation sound source generation unit 325, and an MLSA synthesis filter unit 327.

The synthesizer 113 acquires triphone label data TLabData _m [tl] from the first speech database 221 (FIG. 2), acquires the first learning result from the first speech synthesis dictionary 223, and synthesizes speech data SynSp _m (1 ≦ m ≦ N _Sp ) is output. The output synthesized speech data SynSp _m is delivered to the second learning section 117 will be described later (FIG. 4).

Since the triphone label data TLabData _m [tl] is acquired from the first voice database 221, the synthesizer 113 puts the same speech as the voice data stored in the first voice database 221 into a synthesized voice. Will be emitted. Therefore, as a matter of course, each synthesized speech data is identified by the symbol m as in the original speech data, and the number of synthesized speech data is N _Sp as the number of the original speech data.

  As shown in FIG. 2, the synthesized speech here is generated based on the result of phoneme HMM learning well known in the art. It is known that such synthesized speech becomes unnatural that gives a flat impression as compared to the natural speech of human beings as the original speech. This is because the pitch variation of the synthesized speech is generally smaller than the pitch variation of the original speech.

The phoneme HMM string generation unit 321 in FIG. 3 receives the triphone label data TLabData _m [tl] from the first speech database 221 in FIG. 2, and receives the first learning result from the first speech synthesis dictionary 223 in FIG. Then, based on the received first learning result, the phoneme HMM sequence generation unit 321 in FIG. 3 uses the known method to generate the phoneme HMM sequence data related to the pitch and the melodies from the received triphone label data TLabData _m [tl]. Phoneme HMM sequence data related to the cepstrum, and the time series data generation unit 323.

  The time series data generation unit 323 generates pitch time series data and mel cepstrum time series data by a known method from the phoneme HMM series data related to the delivered pitch and the phoneme HMM series data related to the mel cepstrum. Is passed to the excitation sound source generator 325 and the mel cepstrum time series data is passed to the MLSA synthesis filter unit 327, respectively.

  The excitation sound source generation unit 325 generates excitation sound source data from the delivered pitch time series data by a known method, and delivers it to the MLSA synthesis filter unit 327.

The MLSA synthesis filter unit 327 defines its specifications as an MLSA (Mel Log Spectrum Approximation) filter by a known method based on the mel cepstrum time series data delivered from the time series data generation unit 323. When the excitation sound source data generated by the excitation sound source generation unit 325 is input to the MLSA synthesis filter unit 327 for which such definition has been completed, synthesized speech data SynSp _m is output. The output synthesized speech data SynSp _m is sent to the second learning section 117 in FIG.

  The second learning unit 117 illustrated in FIG. 4 includes a second pitch extraction unit 341, a policy determination unit 343, an editing unit 345, and a second phoneme HMM learning unit 347.

The second learning unit 117 acquires the triphone label data TLabData _m [tl] and the monophone label data MLabData _m [ml] from the first speech database 221 (FIG. 2), and from the first learning unit 111 (FIG. 2). The pitch sequence data Pit _m [fm] and the mel cepstrum coefficient sequence data MC _m ^d [fm] are received, and the synthesized speech data SynSp _m is received from the synthesizer 113 (FIG. 3), and based on these data as described below. Phoneme HMM learning is performed, and the learning result is output as the second learning result.

The second pitch extraction unit 341 in FIG. 4 has the same function as the first pitch extraction unit 311 in FIG. 2 and performs almost the same thing. The difference is that the input data is not speech data Sp _m but synthetic speech data SynSp _m and that the upper limit of _fm is M _fm [m] which does not necessarily match N _fm [m]. Because of this difference, the data generated by the second pitch extraction unit 341 will be referred to as synthesized speech pitch sequence data SynPit _m [fm] (0 ≦ fm ≦ M _fm [m]). The data is delivered to the policy determining unit 343.

The policy determining unit 343 collects pitch sequence data Pit _m [fm] and synthesized speech pitch sequence data SynPit _m [fm]. The former is generated on the basis of speech data collected from human natural speech, while the latter is generated on the basis of synthesized speech data once issued through a speech synthesis dictionary. Since the policy decision unit 343 collects these two types of data, these can be compared. Therefore, in order to prevent the synthesized speech from becoming flat and unnatural compared to the original speech, the policy decision unit 343 originally performs any processing on the original speech in advance. Consider what you should have left. Specifically, the policy determination unit 343 determines an editing policy for how the pitch sequence data Pit _m [fm] should be edited before the phoneme HMM learning. At least qualitatively, if the pitch sequence data Pit _m [fm] is edited in advance so that the pitch variation of the original speech becomes large, the synthesized speech becomes more natural.

  Details of the editing policy will be described later with an example.

The policy determination unit 343 transmits the editing policy of the pitch series data Pit _m [fm] determined as a result of the comparative study to the editing unit 345.

The editing unit 345 edits the pitch sequence data Pit _m [fm] in accordance with the transmitted editing policy, generates edited pitch sequence data EdPit _m [fm], and delivers it to the second phoneme HMM learning unit 347.

The second phoneme HMM learning unit 347 has the same function as the first phoneme HMM learning unit 315 in FIG. 2 and performs substantially the same processing. The difference is that the edited pitch sequence data EdPit _m [fm] is used instead of the pitch sequence data Pit _m [fm]. That is, the second phoneme HMM learning unit 347 (FIG. 4) includes the monophone label data MLabData _m [ml], the triphone label data TLabData _m [tl], the edited pitch sequence data EdPit _m [fm], and the mel cepstrum. The coefficient series data MC _m ^d [fm] is received, a second learning result as a learning result is generated from the received data by phoneme HMM learning, and stored in the second speech synthesis dictionary 227. More precisely, by storing the second learning result in an empty database, the empty database is completed as the second speech synthesis dictionary 227.

This second speech synthesis dictionary 227 is the speech synthesis dictionary targeted by the speech synthesis dictionary construction apparatus according to the present embodiment. Compared with the synthesized speech generated based on the first speech synthesis dictionary 223 (FIG. 2) constructed by the conventional technique, the synthesized speech generated based on the second speech synthesis dictionary 227 has a sufficient pitch variation. It will be a big natural thing. As described above, in the policy decision unit 343 (FIG. 4), the process to be performed on the original voice data in order to prevent the synthesized voice from becoming flat and unnatural voice, that is, the pitch fluctuation of the original voice data is greatly increased. The editing policy of the pitch sequence data Pit _m [fm] is determined, and the phoneme HMM learning is performed using the editing pitch sequence data EdPit _m [fm] generated by the editing unit 345 according to the editing policy. It is.

  The speech synthesis dictionary construction device described so far with reference to FIGS. 2 to 4 is physically configured by a general computer device 511 as shown in FIG.

  A CPU (Central Processing Unit) 521, a ROM (Read Only Memory) 523, a storage unit 525, an operation key input processing unit 533, and a data input / output interface (hereinafter referred to as I / F) 555, They are connected to each other via a system bus 541. The system bus 541 is a transmission path for transferring commands and data.

  The CPU 521 incorporates various registers (not shown) such as a counter register and a general-purpose register, and according to an operation program read from the ROM 523, appropriately loads a numeric string to be processed from the storage unit 525 into the register. Then, a predetermined calculation is performed on the loaded numerical sequence, and the result is stored in the storage unit 525 or the like.

ROM523 generates, in addition to the known operating program for the phoneme HMM learning, in particular, in the present embodiment, to determine the editorial policy of pitch series data Pit _m [fm] Edit pitch series data EdPit _m [fm] An operation program for storing is stored.

  The storage unit 525 includes a RAM (Random Access Memory) 527 and a built-in hard disk 529, and temporarily stores voice data, label data, pitch series data, mel cepstrum coefficient series data, phoneme HMM, and the like. These data and the like are transmitted from a built-in register of the CPU 521 or transmitted from a removable hard disk described later.

  In particular, in the present embodiment, it is assumed that the internal hard disk 529 functions as the first speech synthesis dictionary 223 (FIG. 2). The first speech synthesis dictionary 223 is merely an intermediate product for the speech synthesis dictionary construction device according to the present embodiment, and is not given from the outside or finally removed from the device and used. This is because it only needs to be memorized.

  The operation key input processing unit 533 receives an operation signal from the operation key 531 which is a user I / F, and inputs a key code signal corresponding to the operation signal to the CPU 521. The CPU 521 determines the operation content based on the input key code signal.

It is preferable that the user can change the operation setting of the speech synthesis dictionary construction device according to the present embodiment to a desired one via the operation key 531. For example, in a procedure for generating edit pitch sequence data EdPit _m [fm], which will be described later, from pitch sequence data Pit _m [fm], one of the specific examples of edits described later is selected and set in advance in the ROM 523 as an editing policy. If desired, the user may be able to change the setting via the operation key 531.

  The data input / output I / F 555 is an interface for connecting to the first removable hard disk 551 containing original data and the second removable hard disk 553 for recording processed data. The I / F can connect two such removable hard disks at the same time to improve work efficiency. The I / F is of a general specification capable of bidirectional data communication with both the first and second removable hard disks 551 and 553, and a bidirectional white arrow is illustrated in that sense. . Of course, in communication with the first removable hard disk 551, the original data is mainly read from the disk, whereas in communication with the second removable hard disk 553, processed data is mainly written to the disk. Information is transmitted mainly in the direction indicated by solid arrows.

  As the original data, data stored in the first speech database 221 of FIG. 2 is assumed, and as the processed data, the second learning result stored in the second speech synthesis dictionary 227 of FIG. 4 is assumed. That is, the first removable hard disk 551 corresponds to the first speech database 221 in FIG. 2, and the second removable hard disk 553 corresponds to the second speech synthesis dictionary 227 in FIG.

  When the user wants to construct a speech synthesis dictionary using the speech synthesis dictionary construction apparatus according to the present embodiment, the given first speech database 221, that is, the first removable hard disk 551 and the empty second removable hard disk 553 are stored. These are connected to predetermined positions of the data input / output I / F 555. Thereafter, the user operates the operation key 531 to operate the speech synthesis dictionary construction device. Then, various processes are performed under the control of the CPU 521. For example, data is input / output between the computer device 511 and the first and second removable hard disks 551 and 553 via the data input / output I / F 555. When such an operation is finished, the second learning result shown in FIG. 4 is written in the second removable hard disk 553. That is, the disc is in a state where all data necessary for functioning as the second speech synthesis dictionary 227 of FIG. 4 has been written. Thereafter, when the user wishes to generate synthesized speech, the disc is removed from the user I / F 555 and attached to a speech synthesizer that can be connected as a speech synthesis dictionary. By operating it, synthesized speech can be generated.

As shown in FIG. 4, the feature of the speech synthesis dictionary construction device according to the present embodiment is that the policy determining unit 343 determines the editing policy of the pitch sequence data Pit _m [fm], and the editing unit 345 follows the editing policy. The pitch sequence data Pit _m [fm] is edited to generate edited pitch sequence data EdPit _m [fm].

Editing the editing unit 345 performs, if processing for moderately increasing the pitch variation of the audio data Sp _m, may be any process. However, particularly in the case of the present embodiment, the guidelines for such processing are based on the pitch sequence data Pit _m [fm] collected in the policy determination unit 343 and the synthesized speech pitch sequence data SynPit _m [fm]. It is important to decide efficiently, accurately and simply.

(Specific examples of editing)
Hereinafter, a typical procedure of such editing processing will be described.

Note that at least qualitatively, if the difference between the individual pitch sequence data Pit _m [fm] and the average value is the edit pitch sequence data EdPit _m [fm], the pitch variation of the audio data Sp _m Becomes bigger. Therefore, the following explanation of the specific example of editing will be focused on the explanation on how to specifically increase the difference between the average value and the individual value, and the basics. In this case, the difference is expanded by multiplying the difference by some value larger than 1 for editing to obtain the edit pitch series data EdPit _m [fm].

  However, it is considered that the synthesized speech generated based on the speech data in which the above-described difference is enlarged too much becomes unnatural because the degree of flatness is too small. Therefore, any value greater than 1 for editing described above should not be too large and is preferably reasonably large. It is considered appropriate to determine such an appropriate size by comparing the original voice data with the synthesized voice data according to the prior art generated based on the original voice data. This is because the difference between the former data and the latter data corresponds to the degree of flatness that occurs during the phoneme HMM learning process, so that an editing policy is established based on the difference and the former data is flattened in advance. If the data is edited to a data with a reduced degree of noise, the flatness in the synthesized speech data generated based on the data subjected to such editing is moderately small and the naturalness is given to the listener. It is possible. As is clear from the following description, the editing policy determined by the policy determination unit 343 in FIG. 4 is in line with such consideration.

  Which one of the plurality of procedures described below is optimal to adopt depends on the nature of the sample data recorded in the first speech database 221 (FIG. 2) and the construction of the speech synthesis dictionary according to the present embodiment. Since it depends on various factors such as the processing capability of the CPU 521 of the computer device 511 (FIG. 5) used as the device, the content to be uttered as synthesized speech, and the way the listener hears the synthesized speech, generally I cannot conclude. It is reasonable to try several procedures and determine which is the optimal procedure under the various conditions given.

  Although various procedures are conceivable, as described above, these procedures are consistent in terms of the determination of the editing policy by the policy determination unit 343 in FIG. 4 and the execution of editing of the pitch sequence data corresponding thereto. That is, the various procedures shown below are variations within the scope of the technical idea.

As illustrated in FIG. 5, the computer device 511 that functions as the speech synthesis dictionary construction device according to the present embodiment includes a built-in register of the CPU 521, a RAM 527 in the storage unit 525, and a built-in hard disk 529 as a storage device. In addition, the first and second removable hard disks 551 and 553, which are practically part of the computer device 511, are connected to the data input / output I / F 555 during the construction of the speech synthesis dictionary. . In the following, in order to facilitate understanding, storage devices other than the register, where various operations are performed, are collectively referred to simply as a storage unit 525. Then, the storage unit 525 stores the sound data Sp _m , the monophone label data MLabData _m [ml], and the triphone label data TLabData _m [tl] from the beginning. In the following, pitch sequence data Pit _m [fm], mel cepstrum coefficient sequence data MC _m ^d [fm], and synthesized speech pitch sequence data SynPit _m [fm] are already obtained and stored in the storage unit 525. And

The frames are classified into voiced sound frames and unvoiced sound frames. It is assumed that which frame is a voiced sound frame and which frame is an unvoiced sound frame is already obtained by any known method, and the result is also stored in the storage unit 525. In the following, when “Pit _m [fm]” or “SynPit _m [fm]” is described, it means a specific value of the pitch, and the frame fm corresponding to it does not need to be specified. It also means that a distinction has already been made between a voiced sound frame and an unvoiced sound frame.

(Specific example 1 of editing)
A specific example 1 of editing will be described with reference to the flowcharts shown in FIGS.

First, as shown in FIG. 6, the recording average value AvePit _m for editing is calculated. For this purpose, 1 is stored in the counter register (not shown) of the CPU 521 in FIG. 5 as the initial value of the counter m (step S111). This m is a variable for identifying to which audio data the pitch sequence data of interest belongs.

Next, the CPU 521 provides an area for storing the recording average value AvePit _m for editing in an internal general-purpose register (not shown), and sets the recording average value AvePit _m for editing to 0. Then, the CPU 521 stores the frame identification counter fm in a counter register different from the counter register that stores the m, and sets its initial value to 0. Then, CPU 521 stores counter N _Vfm [m] in another counter register, and sets its initial value to 0 (step S113).

The N _Vfm [m] is a variable for counting how many voiced frames exist in the m-th audio data.

  Subsequently, the CPU 521 determines whether or not the frame specified by fm is a voiced sound frame (step S115).

When it is determined that the frame specified by fm is a frame of voiced sound (step S115; Yes), the CPU 521 loads the pitch sequence data Pit _m [fm] from the storage unit 525 (step S117). Then, CPU 521 is such Pit _m [fm] with a new AvePit _m in addition to AvePit _m, 1 is added to N _Vfm [m] and a new N _Vfm [m] (step S119). That is, AvePit _m is updated to AvePit _m + Pit _m [fm], and N _Vfm [m] is updated to N _Vfm [m] +1. Thereafter, the process proceeds to step S121.

  If it is determined that the frame specified by fm is not a voiced sound frame (step S115; No), the process immediately proceeds to step S121.

In step S121, the CPU 521 determines whether or not the processing for all the frames of the mth audio data has been completed. That is, the CPU 521 determines whether or not fm ≧ N _fm [m].

If it is determined that fm ≧ N _fm [m] is not satisfied (step S121; No), the CPU 521 increases fm by 1 in order to perform processing for the next frame (step S123). Then, the process returns to step S115. On the other hand, if it is determined that fm ≧ N _fm [m] (step S121; Yes), the process proceeds to step S125.

At the time of proceeding to step S125, the value of AvePit _m is the total value of the pitch sequence data Pit _m [fm] in the frame of the voiced sound in the mth audio data. Therefore, in step S125, the CPU 521 divides AvePit _m by N _Vfm which is the number of voiced sound frames, and AvePit _m as an average value of pitch sequence data in the voiced sound frame of the m-th sound data. Ask for. That is, AvePit _m is updated to AvePit _m / N _Vfm . The CPU 521 stores the updated AvePit _m in the storage unit 525 (Step S127).

The CPU 521 determines whether or not processing for all audio data has been completed (step S129). That is, the CPU 521 determines whether m ≧ N _Sp is satisfied.

When it is determined that m ≧ N _Sp is not satisfied (step S129; No), the CPU 521 increments m by 1 to perform processing for the next audio data (step S131). Then, the process returns to step S113. On the other hand, when it is determined that m ≧ N _Sp (step S129; Yes), the CPU 521 ends the process.

Subsequently, as shown in FIG. 7, a composite average value for editing AveSynPit _m is calculated (steps S161 to S181). The calculation procedure is almost the same as the procedure for calculating the recording average value AvePit _m for editing shown in FIG. The main difference is that an average value for the synthesized speech pitch sequence data SynPit _m [fm] is obtained instead of an average value for the pitch sequence data Pit _m [fm] (step S167, step S169, step S175), m Since the upper limit of the frame number corresponding to the second audio data does not necessarily match N _{fm [} m], another variable M _fm [m] is used (step S171).

Subsequently, as shown in FIG. 8, the maximum absolute value mxAbsDiffPit _m for each recording sound for editing is calculated.

The CPU 521 sets the audio data identification counter m to m = 1 (step S211). The CPU 521 further sets mxAbsDiffPit _m to a sufficiently small value such as 0, and sets the frame identification counter fm to fm = 0 (step S213).

  The CPU 521 determines whether or not the frame specified by fm is a voiced sound frame (step S215).

  When it is determined that the frame specified by fm is not a voiced sound frame (step S215; No), the process proceeds to step S225.

When it is determined that the frame specified by fm is a frame of a voiced sound (step S215; Yes), the CPU 521 reads the pitch sequence data Pit _m [fm] from the storage unit 525, as shown in FIG. AvePit _m obtained by the indicated procedure is loaded (step S217). Then, the CPU 521 calculates TmpmxAbsDiffPit _m = | Pit _m [fm] −AvePit _m | (step S219), and determines whether TmpmxAbsDiffPit _m ≧ mxAbsDiffPit _m (step S221). If it is determined that TmpmxAbsDiffPit _m ≧ mxAbsDiffPit _m (step S221; No), the process immediately proceeds to step S225. If it is determined that TmpmxAbsDiffPit _m ≧ mxAbsDiffPit _m (step S221; Yes), mxAbsDiffPit _m is set. After updating as mxAbsDiffPit _m = TmpmxAbsDiffPit _m (step S223), the process proceeds to step S225.

In step S225, the CPU 521 determines whether or not fm ≧ N _fm [m].

If it is determined that fm ≧ N _fm [m] is not satisfied (step S225; No), fm is incremented by 1 (step S227), and the process returns to step S215. On the other hand, if it is determined that fm ≧ N _fm [m] (step S225; Yes), the process proceeds to step S229.

In step S229, the CPU 521 stores mxAbsDiffPit _m in the storage unit 525.

In step S231, the CPU 521 determines whether m ≧ N _Sp is satisfied.

If it is determined that m ≧ N _Sp is not satisfied (step S231; No), m is incremented by 1 (step S233), and the process returns to step S213. On the other hand, when it is determined that m ≧ N _Sp (step S231; Yes), the process ends.

Subsequently, as shown in FIG. 9, to calculate the maximum specific editing synthetic speech absolute value mxAbsDiffSynPit _m (step S261~S283). This calculation procedure is almost the same as the procedure for calculating the maximum absolute value mxAbsDiffPit _m for each recording sound for editing shown in FIG. The main difference is that the synthesized speech pitch sequence data SynPit _m [fm] is handled instead of the pitch sequence data Pit _m [fm] (steps S267 and S269), and the editing is obtained by the procedure shown in FIG. As the upper limit of the frame number corresponding to the m-th audio data, the edit composite average value AveSynPit _m obtained by the procedure shown in FIG. 7 instead of the recording average value AvePit _m is used. The point is that M _fm [m] is used instead of N _{fm [} m] (step S275).

  Subsequently, as shown in FIG. 10, an editing recording maximum absolute value MaxAbsDiffPit is calculated.

  The CPU 521 sets MaxAbsDiffPit to a sufficiently small value such as 0, and sets the audio data identification counter m to m = 1 (step S311).

The CPU 521 loads mxAbsDiffPit _m obtained by the procedure shown in FIG. 8 from the storage unit 525 (step S313) and sets TmpMaxAbsDiffPit = mxAbsDiffPit _m (step S315).

  The CPU 521 determines whether or not TmpMaxAbsDiffPit ≧ MaxAbsDiffPit (step S317). If it is determined that TmpMaxAbsDiffPit ≧ MaxAbsDiffPit is not satisfied (step S317; No), the process immediately proceeds to step S321. On the other hand, if it is determined that TmpMaxAbsDiffPit ≧ MaxAbsDiffPit (step S317; Yes), MaxAbsDiffPit is updated as MaxAbsDiffPit = TmpMaxAbsDiffPit (step S319), and then the process proceeds to step S321.

In step S321, the CPU 521 determines whether m ≧ N _Sp is satisfied.

If it is determined that m ≧ N _Sp is not satisfied (step S321; No), m is incremented by 1 (step S323), and the process returns to step S313. On the other hand, if it is determined that m ≧ N _Sp (step S321; Yes), MaxAbsDiffPit is stored in the storage unit 525 (step S325), and the process ends.

Subsequently, as shown in FIG. 11, the combined maximum synthetic absolute value MaxAbsDiffSynPit is calculated (steps S361 to S375). The calculation procedure is almost the same as the procedure for calculating the editing total recording maximum absolute value MaxAbsDiffPit shown in FIG. The main difference is, using the maximum synthetic speech-edited absolute value MxAbsDiffSynPit _m obtained by the procedure shown in editing recorded speech by the maximum absolute value mxAbsDiffPit in _m rather 9 obtained by the procedure shown in FIG. 8 (Step S363, Step S365).

  As described above, FIGS. 6 and 7, FIGS. 8 and 9, and FIGS. 10 and 11 are procedures for pitch sequence data of the original speech stored in the first speech database 221 (FIG. 2), respectively. And a procedure for pitch sequence data of synthesized speech based on the first speech synthesis dictionary 223 (FIG. 2).

The procedure for finally obtaining the edit pitch sequence data EdPit _m [fm] in the specific example of this editing is shown in the flowchart of FIG.

In FIG. 12, the edited pitch sequence data EdPit _m [fm] in the present embodiment is represented as first edited pitch sequence data EdPit _m [fm], but the symbol EdPit _m [fm] itself will be described later. In another example, it is assumed that the editing pitch sequence data generated by the editing unit 345 (FIG. 4) is indicated.

In order to obtain the first editing pitch series data EdPit _m [fm], first, the CPU 521 obtains the recording total maximum absolute value MaxAbsDiffPit for editing obtained by the procedure shown in FIG. 10 and the procedure shown in FIG. The edited composition total maximum absolute value MaxAbsDiffSynPit is loaded from the storage unit 525 (step S411).

  Subsequently, the CPU 521 sets the audio data identification counter m to m = 1 (step S413), and sets the frame identification counter fm to fm = 0 (step S415).

The CPU 521 loads the pitch series data Pit _m [fm] (step S417), and determines whether or not the frame specified by fm is a voiced sound frame (step S419).

When it is determined that the frame specified by fm is a frame of voiced sound (step S419; Yes), the CPU 521 stores the recording average value AvePit _m for editing obtained by the procedure shown in FIG. Is loaded (step S421). Then, the CPU 521 obtains the first edited pitch series data “Epit _m [fm]”,
Edpit _m [fm] = (Pit _m [fm]-AvePit _m ) × (MaxAbsDiffPit / MaxAbsDiffSynPit) + AvePit _m
(Step S423) and stored in the storage unit 525 (step S427).

On the other hand, when it is determined that the frame specified by fm is not a voiced sound frame (step S419; No), the CPU 521 sets Pit _m [fm] as it is as the first edited pitch sequence data Edpit _m [fm]. (Step S425), the data is stored in the storage unit 525 (Step S427).

The CPU 521 determines whether or not fm ≧ N _fm [m] (step S429).

If it is determined that fm ≧ N _fm [m] is not satisfied (step S429; No), fm is incremented by 1 (step S431), and the process returns to step S417. On the other hand, when it is determined that fm ≧ N _fm [m] (step S429; Yes), the process proceeds to step S433.

In step S433, the CPU 521 determines whether m ≧ N _Sp is satisfied.

If it is determined that m ≧ N _Sp is not satisfied (step S433; No), m is incremented by 1 (step S435), and the process returns to step S415. On the other hand, if it is determined that m ≧ N _Sp (step S433; Yes), the process is terminated.

  As described above, generally, according to the conventional method, the synthesized speech becomes an unnatural speech giving a flat impression, because the pitch variation of the synthesized speech is changed to the pitch variation of the original natural speech. This is because it becomes smaller than that.

  Incidentally, as is clear from the procedure shown in FIG. 10, the total recording maximum absolute value MaxAbsDiffPit for editing indicates the maximum value of the deviation from the pitch average in all the original audio data. On the other hand, as is clear from the procedure shown in FIG. 11, the editing combined maximum maximum absolute value MaxAbsDiffSynPit indicates the maximum deviation from the pitch average in all the synthesized speech data.

  Therefore, the total recording maximum absolute value for editing MaxAbsDiffPit is an index representing the degree of pitch fluctuation in the original natural speech, and the synthetic total maximum absolute value for editing MaxAbsDiffSynPit is an index representing the degree of pitch fluctuation in the synthesized speech. Can think.

  Since the synthesized speech is flat as described above, it is expected that MaxAbsDiffSynPit <MaxAbsDiffPit almost certainly. Then, in order to make the pitch fluctuation of the synthesized speech the same as the pitch fluctuation of the original natural voice, the pitch fluctuation in the original natural voice is expected to be a value appropriately larger than 1 in advance ( It is considered appropriate to enlarge to (MaxAbsDiffPit / MaxAbsDiffSynPit) times. This is the reason why the difference between the pitch sequence data and the average value is multiplied by the value (MaxAbsDiffPit / MaxAbsDiffSynPit) in step S423 in FIG.

  In this way, the phoneme HMM learning is performed by the second phoneme HMM learning unit 347 (FIG. 4) using the pitch series data in which the pitch variation is increased in advance, so that the second speech synthesis dictionary 227 (in which such learning results are stored) FIG. 4) is useful for synthesizing speech that gives a natural impression.

(Modification of specific example 1 of editing)
A modification of the first specific example of editing will be described with reference to the flowcharts shown in FIGS. 13 to 21 and FIGS. 6 and 7 already described.

As shown in FIG. 13, the maximum value mxDiffPit _m for each recording sound for editing is calculated. The CPU 521 sets the audio data identification counter m to m = 1 (step S9111). Further, the CPU 521 sets mxDiffPit _m to a sufficiently small value such as 0, and sets the frame identification counter fm to fm = 0 (step S9113).

  The CPU 521 determines whether or not the frame fm is a voiced sound frame (step S9115).

  If it is determined that the frame fm is not a voiced frame (step S9115; No), the process immediately proceeds to step S9125.

When it is determined that the frame fm is a voiced sound frame (step S9115; Yes), the CPU 521 determines the pitch sequence data Pit _m [fm] and the recording average value AvePit for editing obtained by the procedure shown in FIG. _m is loaded from the storage unit 525 (step S9117). Then, the CPU 521 calculates TmpmxDiffPit _m = Pit _m [fm] −AvePit _m (step S9119), and determines whether TmpmxDiffPit _m ≧ mxDiffPit _m (step S9121). When it is determined that TmpmxDiffPit _m ≧ mxDiffPit _m is not satisfied (step S9121; No), the process immediately proceeds to step S9125. When it is determined that TmpmxDiffPit _m ≧ mxDiffPit _m is satisfied (step S9121; Yes), mxDiffPit _m is set. After updating as mxDiffPit _m = TmpmxDiffPit _m (step S9123), the process proceeds to step S9125.

In step S9125, the CPU 521 determines whether or not fm ≧ N _fm [m]. If it is determined that fm ≧ N _fm [m] is not satisfied (step S9125; No), fm is incremented by 1 (step S9127), and the process returns to step S9115. On the other hand, if it is determined that fm ≧ N _fm [m] (step S9125; Yes), the CPU 521 stores mxDiffPit _m in the storage unit 525 (step S9129), and then proceeds to step S9131.

In step S9131, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S9131; No), m is incremented by 1 (step S9133), and the process returns to step S9113. On the other hand, if it is determined that m ≧ N _Sp (step S9131; Yes), the process ends.

Subsequently, as shown in FIG. 14, a minimum value mnDiffPit _m for each recording sound for editing is calculated. The CPU 521 sets the audio data identification counter m to m = 1 (step S9161). The CPU 521 further sets mnDiffPit _m to a sufficiently large value such as 0 and sets the frame identification counter fm to fm = 0 (step S9163).

Incidentally, voice recording by the minimum value MnDiffPit _m for editing, as is clear from a later step and eventually becomes 0 following values. Therefore, as described above, in step S9163, if the initial value of mnDiffPit _m is set to 0, for example, it can be said that the initial value of mnDiffPit _m is set to a sufficiently large value.

  The CPU 521 determines whether or not the frame fm is a voiced sound frame (step S9165).

  If it is determined that the frame fm is not a voiced frame (step S9165; No), the process immediately proceeds to step S9175.

When it is determined that the frame fm is a voiced sound frame (step S9165; Yes), the CPU 521 determines the pitch sequence data Pit _m [fm] and the recording average value AvePit for editing obtained by the procedure shown in FIG. _m is loaded from the storage unit 525 (step S9167). Then, the CPU 521 calculates TmpmnDiffPit _m = Pit _m [fm] −AvePit _m (step S9169), and determines whether or not TmpmnDiffPit _m ≦ mnDiffPit _m (step S9171). If it is determined that TmpmnDiffPit _m ≦ mnDiffPit _m is not satisfied (step S 9171; No), the process immediately proceeds to step S 9175, and if it is determined that TmpmnDiffPit _m ≦ mnDiffPit _m (step S 9171; Yes), mnDiffPit _m is set. After updating as mnDiffPit _m = TmpmnDiffPit _m (step S9173), the process proceeds to step S9175.

In step S9175, the CPU 521 determines whether or not fm ≧ N _fm [m]. If it is determined that fm ≧ N _fm [m] is not satisfied (step S9175; No), fm is incremented by 1 (step S9177), and the process returns to step S9165. On the other hand, if it is determined that fm ≧ N _fm [m] (step S9175; Yes), the CPU 521 stores mnDiffPit _m in the storage unit 525 (step S9179), and then proceeds to step S9181.

In step S9181, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S9181; No), m is incremented by 1 (step S9183), and the process returns to step S9163. On the other hand, if it is determined that m ≧ N _Sp (step S9181; Yes), the process ends.

  Subsequently, as shown in FIG. 15, the editing maximum recording maximum value MaxDiffPit is calculated. The CPU 521 sets MaxDiffPit to a sufficiently small value such as 0, and sets the audio data identification counter m to m = 1 (step S9211).

The CPU 521 loads the maximum value mxDiffPit _m for each recording sound for editing obtained by the procedure shown in FIG. 13 from the storage unit 525 (step S9213), sets TmpMaxDiffPit = mxDiffPit _m (step S9215), and whether TmpMaxDiffPit ≧ MaxDiffPit. Is determined (step S9217).

  When it is determined that TmpMaxDiffPit ≧ MaxDiffPit is not satisfied (step S9217; No), the process immediately proceeds to step S9221, and when it is determined that TmpMaxDiffPit ≧ MaxDiffPit (step S9217; Yes), MaxDiffPit is expressed as MaxDiffPit = TmpMaxDiffPit. (Step S9219), the process proceeds to step S9221.

In step S9221, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S9221; No), m is increased by 1 (step S9223), and the process returns to step S9213. On the other hand, if it is determined that m ≧ N _Sp (step S9221; Yes), the CPU 521 stores MaxDiffPit in the storage unit 525 (step S9225) and ends the process.

  Subsequently, as shown in FIG. 16, the editing total recording minimum value MinDiffPit is calculated. The CPU 521 sets MinDiffPit to a sufficiently large value such as 0, and sets the audio data identification counter m to m = 1 (step S9261).

  It should be noted that the total recording minimum value MinDiffPit for editing finally becomes a value of 0 or less, as is apparent from the subsequent procedure. Therefore, as described above, in step S9261, if the initial value of MinDiffPit is set to 0, for example, it can be said that the initial value of MinDiffPit is set to a sufficiently large value.

The CPU 521 loads the minimum value mnDiffPit _m for each recorded sound for editing obtained by the procedure shown in FIG. 14 from the storage unit 525 (step S9263), sets TmpMinDiffPit = mnDiffPit _m (step S9265), and whether TmpMinDiffPit ≦ MinDiffPit. Is determined (step S9267).

  If it is determined that TmpMinDiffPit ≦ MinDiffPit is not satisfied (step S9267; No), the process immediately proceeds to step S9271. If it is determined that TmpMinDiffPit ≦ MinDiffPit (step S9267; Yes), MinDiffPit is expressed as MinDiffPit = TmpMinDiffPit. (Step S9269), the process proceeds to step S9271.

In step S9271, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S9271; No), m is increased by 1 (step S9273), and the process returns to step S9263. On the other hand, if it is determined that m ≧ N _Sp (step S9271; Yes), the CPU 521 stores MinDiffPit in the storage unit 525 (step S9275), and ends the process.

  Subsequently, various editing coefficients for the synthesized speech are obtained. The flowcharts showing the procedures are shown in FIGS. 17 to 20, but the procedures shown in these figures are substantially parallel to the procedures shown in FIGS. 13 to 16 for the original voice described above. ing. Therefore, in order to avoid complicated description, detailed description of each step will be omitted below, and only main points of attention will be described.

According to the procedure shown in FIG. 17, it calculates the editing synthetic speech-specific maximum value mxDiffSynPit _m (step S9311~S9333). This procedure is very similar to the procedure for calculating the maximum value mxDiffPit _m for each recording sound for editing shown in FIG. The main difference is that the synthesized speech pitch sequence data SynPit _m [fm] is handled instead of the pitch sequence data Pit _m [fm] (steps S9317 and S9319), and the editing recording obtained by the procedure shown in FIG. that it uses the editing synthesis average AveSynPit _m determined by the procedure shown in FIG. 7 rather than the average value AvePit _m (step S9317, step S9319) and, N _fm upper limit of the number of frames corresponding to the m-th audio data The point is that M _fm [m] is used instead of [m] (step S9325).

According to the procedure shown in FIG. 18, it calculates the editing synthetic speech by minimum mnDiffSynPit _m (step S9361~S9383). This procedure is very similar to the procedure for calculating the minimum value mnDiffPit _m for each recording sound for editing shown in FIG. The main difference is that the synthesized speech pitch sequence data SynPit _m [fm] is handled instead of the pitch sequence data Pit _m [fm] (steps S9367 and S9369), and the editing recording obtained by the procedure shown in FIG. that it uses the editing synthesis average AveSynPit _m determined by the procedure shown in FIG. 7 rather than the average value AvePit _m (step S9367, step S9369) and, N _fm upper limit of the number of frames corresponding to the m-th audio data The point is that M _fm [m] is used instead of [m] (step S9375).

In accordance with the procedure shown in FIG. 19, the editing combined maximum value MaxDiffSynPit is calculated (steps S9411 to S9425). This procedure is very similar to the procedure for calculating the total recording maximum value MaxDiffPit for editing shown in FIG. Lord differences, using the editing synthetic speech-specific maximum value MxDiffSynPit _m determined by the procedure shown in editing recorded voice-specific maximum value mxDiffPit in _m without 17 was determined by the procedure shown in FIG. 13 (Step S9413 Step S9415).

In accordance with the procedure shown in FIG. 20, the edit synthesis minimum value MinDiffSynPit is calculated (steps S9461 to S9475). This procedure is very similar to the procedure for calculating the total recording minimum value MinDiffPit for editing shown in FIG. Lord differences, using the editing synthetic speech by minimum MnDiffSynPit _m determined by the procedure shown in FIG. 18 rather than editing recorded sound by minimum MnDiffPit _m was determined by the procedure shown in FIG. 14 (Step S9463 Step S9465).

The flowchart of FIG. 21 shows a procedure for finally obtaining the edit pitch series data EdPit _m [fm] in this modification. The edit pitch series data in this modification will be referred to as first modified edit pitch series data.

In order to obtain the first modified editing pitch series data EdPit _m [fm], first, the CPU 521 obtains the recording maximum recording maximum value MaxDiffPit obtained by the procedure shown in FIG. 15 and the procedure shown in FIG. The editing total synthesis maximum value MaxDiffSynPit, the editing total recording minimum value MinDiffPit obtained by the procedure shown in FIG. 16, and the editing synthesis total minimum value MinDiffSynPit obtained by the procedure shown in FIG. Loading is performed from the storage unit 525 (step S9511).

  Subsequently, the CPU 521 sets the audio data identification counter m to m = 1 (step S9513), and sets the frame identification counter fm to fm = 0 (step S9515).

The CPU 521 loads the pitch series data Pit _m [fm] (step S9517), and determines whether or not the frame fm is a voiced sound frame (step S9519).

When it is determined that the frame fm is a voiced sound frame (step S9519; Yes), the CPU 521 loads the recording average value AvePit _m for editing obtained by the procedure shown in FIG. 6 from the storage unit 525 ( Step S9521). Then, the CPU 521 determines whether or not Pit _m [fm] −AvePit _m [fm] ≧ 0 (step S9523).

If it is determined that Pit _m [fm] −AvePit _m ≧ 0 (step S9523; Yes), the first modified edit pitch sequence data Edpit _m [fm] is
Edpit _m [fm] = (Pit _m [fm]-AvePit _m ) × (MaxDiffPit / MaxDiffSynPit) + AvePit _m
(Step S9525) and stored in the storage unit 525 (step S9531).

If it is determined that Pit _m [fm] -AvePit _m [fm] ≧ 0 is not satisfied (step S9523; No), the first modified edit pitch sequence data Edpit _m [fm] is
Edpit _m [fm] = (Pit _m [fm]-AvePit _m ) × (MinDiffPit / MinDiffSynPit) + AvePit _m
(Step S9527) and stored in the storage unit 525 (step S9531).

If it is determined in step S9519 that the frame fm is not a voiced sound frame (step S9519; No), the CPU 521 uses Pit _m [fm] as it is as the first edited pitch sequence data Edpit _m [fm] (step S9529). ) And stored in the storage unit 525 (step S9531).

The CPU 521 determines whether or not fm ≧ N _fm [m] (step S9533). If it is determined that fm ≧ N _fm [m] is not satisfied (step S9533; No), fm is incremented by 1 (step S9535), and the process returns to step S9517. On the other hand, if it is determined that fm ≧ N _fm [m] (step S9533; Yes), the process proceeds to step S9537.

In step S9537, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S9537; No), m is incremented by 1 (step S9539), and the process returns to step S9515. On the other hand, if it is determined that m ≧ N _Sp (step S9537; Yes), the process ends.

  In specific example 1 of editing, a value of (MaxAbsDiffPit / MaxAbsDiffSynPit) is uniformly used as a multiplier for enlarging pitch fluctuation. On the other hand, in this modification, as shown in steps S9523 to S9527 in FIG. 21, different multipliers are used depending on whether the pitch coefficient series data is above the average value or below the average value. Thereby, although the calculation is slightly complicated, it is expected that the pitch fluctuation can be expanded more appropriately.

  As a multiplier in the case of exceeding the above, MaxDiffPit which is the maximum value of the deviation from the pitch average to the upper side obtained by scanning all the original audio data is almost certainly smaller than that, It is considered to be the simplest and most reliable to use the value divided by MaxDiffSynPit obtained by scanning the entire synthesized speech data in the same way. Similarly, MinDiffPit, which is the maximum deviation from the average pitch to the lower side, obtained by scanning the entire original audio data, is almost certainly more absolute than that. As the value, it is considered to be the simplest and most reliable to use a value that is a small value divided by MinDiffSynPit obtained by scanning all the synthesized speech data. The procedure shown in FIGS. 13 to 21 is a result of reflecting such a consideration result in a specific processing procedure.

  Note that MinDiffPit and MinDiffSynPit are almost certainly negative in the calculation process, but the above multiplier is the result of division (MinDiffPit ÷ MinDiffSynPit), which is almost certainly positive. Value.

  Also, as already mentioned, since the conventional synthesized speech has less pitch fluctuation than the original speech, the above two types of multipliers (MaxDiffPit / MaxDiffSynPit) and (MinDiffPit / MinDiffSynPit) are almost certainly 1 It is suitable as a multiplier for increasing the pitch variation.

(Specific example 2 of editing)
FIG. 22 is a flowchart showing a procedure for calculating the second edited pitch series data EdPit _m [fm], which is the edited pitch series data in the specific example of this editing. Hereinafter, description will be given with reference to this flowchart.

The CPU 521 sets the audio data identification counter m to m = 1 (step S511), and loads the editing recorded audio maximum absolute value mxAbsDiffPit _m and the editing synthesized audio maximum absolute value mxAbsDiffSynPit _m from the storage unit 525. (Step S513) The frame identification counter fm is set to fm = 0 (Step S515), the pitch sequence data Pit _m [fm] is loaded from the storage unit 525 (Step S517), and the frame fm is a frame of voiced sound. It is determined whether or not there is (step S519).

Here, the editing recorded speech by the maximum absolute value MxAbsDiffPit _m editing synthetic speech specific maximum absolute value MxAbsDiffSynPit _m, respectively, already mentioned, by the procedure shown in FIGS. 8 and 9 is obtained.

When it is determined that the frame fm is a voiced sound frame (step S519; Yes), the CPU 521 loads the editing recording average value AvePit _m obtained by the procedure shown in FIG. 6 (step S521), and edits it. Pitch series data EdPit _m [fm]
Ed Pit _m [fm] = (Pit _m [fm]-AvePit _m ) × (mxAbsDiffPit _m / mxAbsDiffSynPit _m ) + AvePit _m
(Step S523) and stored in the storage unit 525 (step S527).

When it is determined that the frame fm is not a voiced sound frame (step S519; No), the CPU 521 uses the pitch sequence data Pit _m [fm] as it is as edited pitch sequence data EdPit _m [fm] (step S525). The data is stored in the storage unit 525 (step S527).

In step S529, the CPU 521 determines whether or not fm ≧ N _fm [m]. If it is determined that fm ≧ N _fm [m] is not satisfied (step S529; No), fm is incremented by 1 (step S531), and the process returns to step S517. On the other hand, when it is determined that fm ≧ N _fm [m] (step S529; Yes), the process proceeds to step S533.

In step S533, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S533; No), m is incremented by 1 (step S535), and the process returns to step S513. On the other hand, when it is determined that m ≧ N _Sp (step S533; Yes), the process ends.

In the specific example of the editing, as in the specific example 1 of the editing, the pitch variation is increased by vertically expanding the pitch series data on the basis of the average value. For this purpose, a coefficient that is the difference between the pitch sequence data and the average value is multiplied. However, in the specific example 1 of editing, the value (MaxAbsDiffPit / MaxAbsDiffSynPit) that is a value that does not depend on audio data (that is, a value that does not depend on the audio data identification counter m) is uniformly adopted as the coefficient. The specific example of editing differs in that (mxAbsDiffPit _m / mxAbsDiffSynPit _m ), which is a value obtained for each audio data (that is, a value depending on the audio data identification counter m), is adopted as the coefficient. That is, in the specific example of this editing, when editing a certain pitch sequence data to generate the edited pitch sequence data, only a set of pitch sequence data corresponding to the audio data to which the frame used for calculation of the pitch sequence data belongs. Is referenced. In other words, it can be said that the editing process is completed for each audio data.

  In this way, the above-described coefficient increases compared to the case of the specific example 1 of editing, and thus the processing becomes complicated. However, since the editing coefficient is changed according to the characteristics of each audio data, more appropriate editing can be performed. Achieved. For example, in the case of the specific example 1 of editing, there is a possibility that even if there is only one specific audio data, the above-described coefficient is affected by it, and the editing of the pitch series data for all the audio data is not appropriately performed. However, in the specific example of this editing, the existence of such unique data does not adversely affect the entire editing process.

(Modification of specific example 2 of editing)
FIG. 23 is a flowchart showing a procedure for calculating the second modified editing pitch sequence data EdPit _m [fm], which is the editing pitch sequence data in the modified example of the specific example 2 of editing. Hereinafter, description will be given with reference to this flowchart.

The CPU 521 sets the audio data identification counter m to m = 1 (step S9611), the editing recording voice maximum value mxDiffPit _m , the editing synthesized voice maximum value mxDiffSynPit _m, and the editing recording voice minimum value mnDiffPit _m. And the minimum value mnDiffSynPit _m for each synthesized voice for editing are loaded from the storage unit 525 (step S9613), the frame identification counter fm is set to fm = 0 (step S9615), and the pitch sequence data Pit _m [fm] is stored. It is loaded from the unit 525 (step S9617), and it is determined whether or not the frame fm is a voiced sound frame (step S9619).

Here, the editing recorded sound-specific maximum value MxDiffPit _m editing synthetic speech-specific maximum value MxDiffSynPit _m editing and recording audio specific minimum MnDiffPit _m editing synthetic speech by minimum MnDiffSynPit _m, respectively, already It is obtained by the procedure shown in FIG. 13, FIG. 17, FIG. 14, and FIG.

When it is determined that the frame fm is a voiced sound frame (step S9619; Yes), the CPU 521 loads the editing average recording value AvePit _m obtained by the procedure shown in FIG. 6 (step S9621), and Pit. It is determined whether or not _m [fm] −AvePit _m ≧ 0 (step S9623).

If it is determined that Pit _m [fm] −AvePit _m ≧ 0 (step S9623; Yes), the CPU 521 sets the edit pitch sequence data EdPit _m [fm] as
Ed Pit _m [fm] = (Pit _m [fm]-AvePit _m ) × (mxDiffPit _m / mxDiffSynPit _m ) + AvePit _m
(Step S9625) and stored in the storage unit 525 (step S9631). On the other hand, when it is determined that Pit _m [fm] −AvePit _m ≧ 0 is not satisfied (step S9623; No), the CPU 521 sets the edit pitch sequence data EdPit _m [fm] as
Ed Pit _m [fm] = (Pit _m [fm]-AvePit _m ) × (mnDiffPit _m / mnDiffSynPit _m ) + AvePit _m
(Step S9627) and stored in the storage unit 525 (step S9631).

If it is determined that the frame fm is not a voiced frame (step S9619; No), the CPU 521 uses the pitch sequence data Pit _m [fm] as it is as edited pitch sequence data EdPit _m [fm] (step S9629). The data is stored in the storage unit 525 (step S9631).

In step S9633, the CPU 521 determines whether or not fm ≧ N _fm [m]. If it is determined that fm ≧ N _fm [m] is not satisfied (step S9633; No), fm is incremented by 1 (step S9635), and the process returns to step S9617. On the other hand, if it is determined that fm ≧ N _fm [m] (step S9633; Yes), the process proceeds to step S9637.

In step S9637, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S9637; No), m is increased by 1 (step S9639), and the process returns to step S9613. On the other hand, if it is determined that m ≧ N _Sp (step S9637; Yes), the process ends.

  The present modification is advantageous in that the coefficient for pitch fluctuation expansion is changed depending on whether or not the pitch series data exceeds the average value in the modification of the first specific example of editing described above, and the editing described above. In the second specific example, the advantage of completing the processing for each audio data is also provided.

(Embodiment 2)
The speech synthesis dictionary construction device according to Embodiment 2 of the present invention includes the second learning unit 117 shown in FIG. 4 among the functional blocks of the speech synthesis dictionary construction device according to Embodiment 1 shown in FIGS. The second learning unit 119 with phoneme label data comparison function shown in FIG. 24 is substituted.

  The second learning unit 119 with phoneme label data comparison function (FIG. 24) is obtained by adding a phoneme label data generation unit 331 for monophone to the second learning unit 117 (FIG. 4).

The monophone phoneme label data generation unit 331 receives the synthesized speech data SynSp _m output from the synthesis unit 113 (FIG. 3), and synthesized speech monophone label data mLabData _m [ml] (monophonic label data of synthesized speech) 1 ≦ ml ≦ ML _SynSp [m], where ML _SynSp [m] is the number of monophone labels in the synthesized speech SynSp _m ).

The synthesized speech monophone label data mLabData _m [ml] is a pointer indicating the time corresponding to the start point of the synthesized speech monophone label in the duration of the synthesized speech monophone label mLab _m [ml] and the synthesized speech data SynSp _m. Is composed of a synthesized speech start frame mFrameS _m [ml] and a synthesized speech end frame mFrameE _m [ml] which is a pointer indicating the time corresponding to the end point.

In addition, the policy determination unit 343 in the second learning unit 117 collects pitch sequence data Pit _m [fm] and synthesized speech pitch sequence data SynPit _m [fm], but has a phoneme label data comparison function. In addition to these, the policy determination unit 343 in the second learning unit 119 generates monophone label data MLabData _m [ml] stored in the first speech database 221 (FIG. 2) and phoneme label data for monophones. The synthesized voice monophone label data mLabData _m [ml] output from the unit 331 is collected.

Unlike the policy determination unit 343 in the second learning unit 117, the policy determination unit 343 in the second learning unit 119 with phoneme label data comparison function is different from the pitch sequence data Pit _m [fm] and the synthesized speech pitch sequence data SynPit. The editing policy of the pitch sequence data Pit _m [fm] is not determined only from _m [fm], but in addition to these, the monophone label data MLabData _m [ml] and the synthesized speech monophone label data mLabData _m [ml] Are comprehensively examined and the editing policy of the pitch sequence data Pit _m [fm] is determined.

That is, what is delivered to the policy determination unit 343 in the second learning unit 119 with the phoneme label data comparison function is monophone label data generated based on speech data collected from human natural speech. MLabData _m [ml] and pitch sequence data Pit _m [fm] and synthesized speech monophone label data mLabData _m [ml] and synthesized once based on synthesized speech data generated through the speech synthesis dictionary Voice pitch sequence data SynPit _m [fm] is collected. Since the policy decision unit 343 collects these four types of data, these can be compared.

Hereinafter, the editing process executed by the editing unit 345 in the second learning unit 119 with phoneme label data comparison function according to the editing policy determined by the policy determination unit 343 in the second learning unit 119 with phoneme label data comparison function This procedure will be described with reference to the flowchart shown in FIG. In FIG. 25, the edit pitch sequence data in the present embodiment is represented as third edit pitch sequence data EdPit _m [fm].

As described in the section where the specific example of editing in the first embodiment has already been described, at least qualitatively, an edit pitch obtained by increasing the difference between each pitch series data Pit _m [fm] and its average value is used. If the series data Pit _m [fm] is used, the pitch variation of the audio data Sp _m increases. However, even without particular bound on the average value, also by the pitch-series data Pit _m [fm] Edit an enlarged overall pitch sequence data Pit _m [fm], the pitch of speech data Sp _m is growing. In other words, the pitch fluctuation can be increased by enlarging the whole with reference to the zero level instead of using the average value as a reference. In the present embodiment, as an editing policy, a policy of enlarging the entire pitch sequence data Pit _m [fm] is adopted.

However, in the present embodiment, the same editing policy as that of the specific example of editing in the first embodiment may be adopted. For example, in step S619 described later, if the frame fm is a voiced sound,
EdPit _m [fm] = (Pit _m [fm] -AvePit _m ) × (AvePitLab _m [ml] / AveSynPitLab _m [ml]) + AvePit _m
And if the frame fm is silent
EdPit _m [fm] = Pit _m [fm]
It is good.

Speech data Sp _m , monophone label data MLabData _m [ml], triphone label data TLabData _m [tl], pitch sequence data Pit _m [fm], mel cepstrum coefficient sequence data MC _m ^d [fm], synthesized speech pitch sequence It is assumed that the data SynPit _m [fm] and the synthesized audio monophone label data mLabData _m [ml] have already been obtained and stored in the storage unit 525.

  First, the CPU 521 sets the audio data identification counter m to m = 1 (step S611), and sets the frame identification counter fm to fm = 0 (step S613).

Next, the CPU 521 investigates to which monophone label data MLabData _m [ml] the frame fm corresponds. Specifically, the CPU 521 searches the storage unit 525 to obtain a monophone number ml ′ that satisfies MFrameS _m [ml ′] ≦ fm ≦ MFrameE _m [ml ′], and for monophone label data identification The counter ml is set to ml = ml ′ (step S615).

Subsequently, the CPU 521 calculates AvePitLab _m [ml] and AveSynPitLab _m [ml], which are the results of averaging the pitch sequence data in units of monophones for the original voice and the synthesized voice (step S617). A specific procedure for such calculation will be described later with reference to FIG.

Subsequently, the CPU 521 obtains third edit pitch sequence data EdPit _m [fm], which is edit pitch sequence data in the present embodiment,
EdPit _m [fm] = Pit _m [fm] × (AvePitLab _m [ml] / AveSynPitLab _m [ml])
(Step S619) and stored in the storage unit 525 (step S621).

Subsequently, the CPU 521 determines whether or not fm ≧ N _fm [m] is satisfied (step S623). If it is determined that fm ≧ N _fm [m] is not satisfied (step S623; No), fm is incremented by 1 (step S625), and the process returns to step S615. On the other hand, when it is determined that fm ≧ N _fm [m] (step S623; Yes), the process proceeds to step S627.

In step S627, the CPU 521 determines whether m ≧ N _Sp is satisfied. If it is determined that m ≧ N _Sp is not satisfied (step S627; No), m is increased by 1 (step S629), and the process returns to step S613. On the other hand, if it is determined that m ≧ N _Sp (step S627; Yes), the process ends.

The procedure for calculating AvePitLab _m [ml] and AveSynPitLab _m [ml] in step S617 is as shown in FIG.

The CPU 521 loads the start frame MFrameS _m [ml], the end frame MFrameE _m [ml], the synthesized speech start frame mFrameS _m [ml], and the synthesized speech end frame mFrameE _m [ml] from the storage unit 525 (step S661). ), Pit _m [MFrameS _m [ml]], Pit _m [MFrameS _m [ml] +1], Pit _m [MFrameE _m [ml] -1], Pit _m [MFrameE _m [ml]] and synthetic speech mel cepstrum coefficient series data SynPit _m [mFrameS _m [ml]], SynPit _m [mFrameS _m [ml] +1], ..., SynPit _m [mFrameE _m [ml] -1] SynPit _m [mFrameE _m [ml]] is loaded (step S663), and AvePitLab _m [ml] and AveSynPitLab _m [ml] are calculated according to the following equations (step S665).
AvePitLab _m [ml]
= (Pit _m [MFrameS _m [ml]] + Pit _m [MFrameS _m [ml] +1] + ...
+ Pit _m [MFrameE _m [ml] -1] + Pit _m [MFrameE _m [ml]])
÷ (MFrameE _m [ml] -MFrameS _m [ml] +1),
AveSynPitLab _m [ml]
= (SynPit _m [mFrameS _m [ml]] + SynPit _m [mFrameS _m [ml] +1] + ...
+ SynPit _m [mFrameE _m [ml] -1] + SynPit _m [mFrameE _m [ml]])
÷ (mFrameE _m [ml] -mFrameS _m [ml] +1)

As described above, AvePitLab _m [ml] and AveSynPitLab _m [ml] are the results of averaging the pitch sequence data in monophone units for the original voice and the synthesized voice, respectively. This is clear from the above equation.

  In the monophone unit, which is a unit smaller than the sound data, the time zone corresponding to each unit generally includes only the peak portion or the valley portion of the pitch due to the short time zone.

As already described, it is generally known that the synthesized speech is an unnatural speech that gives a flat impression as compared to the original speech. This corresponds to the pitch crest portion being lowered and the pitch trough portion being raised. In the present embodiment, the enlargement ratio AvePitLab _m [ml] / AveSynPitLab _m [ml], which is a value multiplied by the original pitch, is determined in units of monophone labels. In the monophone label unit corresponding to the peak portion of the pitch, AvePitLab _m [ml]> AveSynPitLab _m [ml] corresponding to the fact that the peak portion becomes lower as described above. Then, the enlargement ratio becomes larger than 1. This is because in the original speech, pitch editing is performed so that the pitch peak is raised beforehand in anticipation that it will be lower after speech synthesis, and the pitch peak after speech synthesis is not lower than the original speech. Means that On the other hand, in the monophone label unit corresponding to the valley portion of the pitch, AvePitLab _m [ml] <AveSynPitLab _m [ml] corresponding to the fact that the valley portion becomes higher as described above. Then, the enlargement ratio becomes smaller than 1. This is because in the original speech, pitch editing is performed so that the valley of the pitch is dug in advance in anticipation that it will be higher after speech synthesis, and the pitch valley after speech synthesis is not higher than the original speech. Means that As described above, in the present embodiment, the enlargement ratio is larger or smaller than 1, but the same as in the case of the first embodiment in that the pitch fluctuation is expected to be expanded by the pitch editing.

  In addition, this invention is not limited to the above-mentioned embodiment, a specific example, and a modification, A further various deformation | transformation and application are possible. The above-described hardware configuration, block configuration, and flowchart are examples for explanation, and do not limit the scope of the present invention.

For example, in the speech synthesis dictionary construction apparatus according to Embodiment 1 shown in FIGS. 2 to 4, the pitch time series data generated by the time series data generation unit 323 (FIG. 3) (this is the synthesized speech pitch series data SynPit _m [ fm] is directly input to the policy determination unit 343 (FIG. 4). In this modification, the excitation sound source generation unit 325 (FIG. 3), the MLSA synthesis filter unit 327 (FIG. 3), and the second pitch extraction unit 341 (FIG. 4) can be omitted.

  Similar modifications can be considered in the speech synthesis dictionary construction apparatus according to the second embodiment shown in FIGS. 2, 3, and 24. However, in this case, it is necessary to send information indicating a range corresponding to the pitch time-series data of each monophone label to the policy determining unit 343 (FIG. 24).

  Alternatively, for example, when editing a frame of voiced sound, a policy may be adopted in which the pitch variation is increased by adopting a policy of increasing the shift enlargement rate as the shift from the average of the pitch sequence data increases.

It is a figure which shows the flow of preparation of label data for constructing a general voice database. It is a functional lineblock diagram of the 1st learning part etc. which make a part of speech synthesis dictionary construction device concerning Embodiment 1 of the present invention. It is a function block diagram of the synthetic | combination part which makes a part of the speech synthesis dictionary construction apparatus which concerns on Embodiment 1 of this invention. It is a functional lineblock diagram of the 2nd learning part etc. which make a part of speech synthesis dictionary construction device concerning Embodiment 1 of the present invention. It is a figure which shows the physical structure of the speech synthesis dictionary construction apparatus which concerns on embodiment of this invention. It is a diagram showing the flow of processing for calculating the edit recording average AvePit _m needed to edit the pitch sequence data. It is a diagram showing the flow of processing for calculating the editing synthesis average AveSynPit _m needed to edit the pitch sequence data. It is a diagram showing the flow of processing for calculating the edit voice recording-specific maximum absolute value MxAbsDiffPit _m needed to edit the pitch sequence data. It is a diagram showing the flow of processing for calculating the maximum synthetic speech-edited absolute value MxAbsDiffSynPit _m needed to edit the pitch sequence data. It is a figure which shows the flow of a process which calculates the audio recording total maximum absolute value MaxAbsDiffPit required in order to edit pitch series data. It is a figure which shows the flow of a process which calculates the synthetic | combination synthetic | combination synthetic | combination maximum absolute value MaxAbsDiffSynPit required in order to edit pitch series data. Is a diagram showing the flow of processing for calculating a first edit pitch series data is edited pitch series data EdPit _m [fm] in the specific example 1 of the editing. It is a diagram showing the flow of processing for calculating the edit voice recording-specific maximum value MxDiffPit _m needed to edit the pitch sequence data. Is a diagram showing the flow of a process for calculating an editing recorded sound by minimum MnDiffPit _m needed to edit the pitch sequence data. It is a figure which shows the flow of a process which calculates the recording total recording maximum value MaxDiffPit required in order to edit pitch series data. It is a figure which shows the flow of a process which calculates the audio recording total minimum value MinDiffPit for edit required in order to edit pitch series data. It is a diagram showing the flow of processing for calculating the editing synthetic speech-specific maximum value MxDiffSynPit _m needed to edit the pitch sequence data. It is a diagram showing the flow of processing for calculating the editing synthetic speech by minimum MnDiffSynPit _m needed to edit the pitch sequence data. It is a figure which shows the flow of a process which calculates synthetic | combination synthetic | combination synthetic | combination maximum value MaxDiffSynPit required in order to edit pitch series data. It is a figure which shows the flow of a process which calculates the synthetic | combination synthetic | combination synthetic | combination minimum value MinDiffSynPit required in order to edit pitch series data. Is a diagram showing the flow of processing for calculating a first modified edit pitch series data is edited pitch series data EdPit _m [fm] in a modification of the embodiment 1 of the editing. Is a diagram showing the flow of processing for calculating a second edit pitch series data is edited pitch series data EdPit _m [fm] in the specific example 2 of the editing. Is a diagram showing the flow of processing for calculating a second modification edit pitch series data is edited pitch series data EdPit _m [fm] in a modification of the embodiment 2 of the editing. It is functional block diagrams, such as a 2nd learning part with a speech label data comparison function which makes a part of the speech synthesis dictionary construction apparatus concerning Embodiment 2 of this invention. Is a diagram showing the flow of processing for calculating a third editing pitch series data is edited pitch series data EdPit _m [fm] in the second embodiment of the present invention. Required to edit the pitch sequence data is a diagram showing the flow of a process for calculating the AvePitLab _m [ml] and AveSynPitLab _m [ml] and the pitch average of at phoneme.

Explanation of symbols

  111 ... 1st learning part, 113 ... synthesis | combination part, 117 ... 2nd learning part, 119 ... 2nd learning part with audio | voice label data comparison function, 221 ... 1st audio | voice database, 223 ... 1st speech synthesis dictionary, 227 ... 2nd speech synthesis dictionary, 311 ... 1st pitch extraction part, 313 ... Mel cepstrum analysis part, 315 ... 1st phoneme HMM learning part, 321 ... Phoneme HMM sequence generation unit, 323 ... Time series data generation unit, 325 ... Excitation sound source generation unit, 327 ... MLSA synthesis filter unit, 331 ... Phoneme label data generation unit for monophone, 341 ... second pitch extraction unit, 343 ... policy determination unit, 345 ... editing unit, 347 ... second phoneme HMM learning unit, 511 ... computer device, 521 ... CPU, 523, ..ROM, 25... Storage unit, 527... RAM, 529... Built-in hard disk, 531... Operation key, 533 .. operation key input processing unit, 541. Removable hard disk, 553 ... Second removable hard disk, 555 ... Data input / output I / F

Claims (14)

A phoneme label string and recorded voice data corresponding to the phoneme label string are acquired from the voice database, and the recorded voice pitch series data is extracted from the acquired recorded voice data, and the extracted recorded voice pitch series data and the acquired phoneme are extracted. A temporary construction unit that constructs a temporary speech synthesis dictionary by HMM (Hidden Markov Model) learning based on the label sequence;
A synthesized data generation unit that relies on the temporary speech synthesis dictionary to generate synthesized speech data and extracts synthesized speech pitch series data from the generated synthesized speech data;
The recorded speech pitch sequence data extracted by the temporary construction unit from the recorded speech data corresponding to the phoneme label sequence, and the synthesized data generation unit extracted from the synthesized speech data corresponding to the phoneme label sequence Based on the result of comparing the synthesized voice pitch series data, an editing unit that edits the recorded voice pitch series data to generate edited pitch series data;
A reconstructing unit that constructs a speech synthesis dictionary by HMM learning based on the phoneme label string and the edited pitch sequence data generated by the editing unit;
A speech synthesis dictionary construction device comprising: A plurality of audio data, a monophone label generated for each of the audio data, a start point pointer and an end point pointer indicating the time corresponding to the start point and end point of the monophone label, and a triphone label generated for each of the audio data Receiving, extracting pitch sequence data from the audio data, generating mel cepstrum coefficient sequence data up to a predetermined order from the audio data, the monophone label, the start point pointer, the end point pointer, the triphone label, and the A first learning unit that constructs a temporary speech synthesis dictionary from pitch sequence data and the mel cepstrum coefficient sequence data by HMM (Hidden Markov Model) learning;
A synthesis unit that generates a plurality of synthesized speech data based on the temporary speech synthesis dictionary and the triphone label;
Editing and editing the pitch sequence data according to an editing policy determined based on the result of comparing the synthesized speech pitch sequence data extracted from the synthesized speech data and the pitch sequence data extracted by the first learning unit An editing unit for generating pitch series data;
A second learning unit that constructs a speech synthesis dictionary by HMM (Hidden Markov Model) learning from the monophone label, the start point pointer, the end point pointer, the triphone label, the edited pitch sequence data, and the mel cepstrum coefficient sequence data When,
A speech synthesis dictionary construction device comprising: The editing unit
Generating a synthesized monophone label and a synthesis start point pointer and a synthesis end point pointer indicating a time corresponding to a start point and an end point of the synthesized monophone label for each synthesized voice data; and the synthesized voice pitch sequence data and the pitch sequence data; In accordance with an editing policy determined based on a result of comparison with reference to the composite monophone label, the composite start point pointer, the composite end point pointer, the monophone label, the start point pointer, and the end point pointer. Editing the data to generate the edited pitch series data;
The speech synthesis dictionary construction device according to claim 2. The editing unit
Generating the edited pitch series data by increasing the pitch variation of the pitch series data;
The speech synthesis dictionary construction apparatus according to claim 2 or 3, The editing unit
Generating the edited pitch series data by enlarging the pitch series data around the reference pitch level with a predetermined pitch level as a reference pitch level;
The speech synthesis dictionary construction device according to any one of claims 2 to 4, wherein the speech synthesis dictionary construction device. The editing unit
Generating the edited pitch series data by enlarging the pitch series data around the reference pitch level with an average value of the pitch series data as a reference pitch level;
The speech synthesis dictionary construction device according to any one of claims 2 to 5, The editing unit
Generating the edited pitch series data by enlarging the pitch series data around the reference pitch level with a zero level of the pitch series data as a reference pitch level;
The speech synthesis dictionary construction device according to any one of claims 2 to 5, The editing unit
An average pitch for each voice that is an average value of the pitch series data is obtained for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data is obtained for each synthesized voice data. A maximum pitch difference for each voice, which is a maximum absolute value of a difference between the pitch series data and the average pitch for each voice, and the average for each synthesized voice pitch series data and the average for each synthesized voice The maximum absolute value of the pitch difference for each synthesized voice, which is the maximum absolute value of the difference from the pitch, is obtained. Determining a synthesized speech total pitch difference maximum absolute value that is a maximum value of the synthesized speech-specific pitch difference maximum absolute values in all the synthesized speech data, An editing overall magnification that is a value obtained by dividing the maximum absolute value by the synthesized speech overall pitch difference maximum absolute value is obtained, and the editing is performed by enlarging the pitch series data at the editing overall magnification around the reference pitch level. Generate pitch series data,
The speech synthesis dictionary construction device according to any one of claims 5 to 7. The editing unit
An average pitch for each voice that is an average value of the pitch series data is obtained for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data is obtained for each synthesized voice data. The maximum value of the pitch difference for each voice, which is the maximum value obtained by subtracting the average pitch for each voice from the pitch series data, is determined, and the minimum of the values obtained by subtracting the average pitch for each voice from the pitch series data for each voice data A voice-specific pitch difference minimum value for each voice data, a voice total pitch difference maximum value that is a maximum value of the voice-specific pitch differences for all the voice data, and a voice-specific pitch difference for all the voice data. The minimum value of the total speech pitch difference, which is the minimum value of the minimum values, is obtained, and the average pitch for each synthesized speech from the synthesized speech pitch sequence data for each synthesized speech data The maximum value of the pitch difference for each synthesized speech, which is the maximum value of the subtracted values, is obtained, and the pitch for each synthesized speech that is the minimum value obtained by subtracting the average pitch for each synthesized speech from the synthesized speech pitch sequence data. A minimum difference value is obtained, a synthesized speech total pitch difference maximum value that is a maximum value of the synthesized speech-specific pitch differences in all the synthesized speech data is obtained, and a synthesized speech-specific pitch difference minimum in all the synthesized speech data A minimum value of the synthesized speech total pitch difference, which is a minimum value, and an upper overall magnification for editing, which is a value obtained by dividing the maximum value of the synthesized speech pitch difference by the maximum value of the synthesized speech overall pitch difference, The lower overall magnification for editing, which is a value obtained by dividing the minimum difference value by the minimum synthesized speech total pitch difference value, is obtained, and exceeds the reference pitch level in the pitch series data. Is enlarged at the upper overall magnification for editing centered on the reference pitch level, and the pitch series data below the reference pitch level is enlarged at the lower overall magnification for editing around the reference pitch level. Generating the edited pitch sequence data by enlarging,
The speech synthesis dictionary construction device according to any one of claims 5 to 7. The editing unit
An average pitch for each voice that is an average value of the pitch series data is obtained for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data is obtained for each synthesized voice data. A maximum pitch difference for each voice, which is a maximum absolute value of a difference between the pitch series data and the average pitch for each voice, and the average for each synthesized voice pitch series data and the average for each synthesized voice A synthesized voice-specific pitch difference maximum absolute value, which is a maximum absolute value of a difference from the pitch, is obtained, and for each voice data, the synthesized voice data corresponding to the voice data is used as the synthesized voice data. Magnification by editing voice divided by the maximum absolute value of the pitch difference by voice is obtained, and the pitch series data is divided by the editing voice by using the reference pitch level for each voice data. Generating the edited pitch series data by enlarging at a rate,
The speech synthesis dictionary construction device according to any one of claims 5 to 7. The editing unit
An average pitch for each voice that is an average value of the pitch series data is obtained for each voice data, and an average pitch for each synthesized voice that is an average value of the synthesized voice pitch series data is obtained for each synthesized voice data. To determine the maximum value of the pitch difference for each voice, which is the maximum value obtained by subtracting the average pitch for each voice from the pitch series data, A pitch difference minimum value for each voice that is a value, and a maximum pitch difference value for each synthesized voice data that is a maximum value of a value obtained by subtracting the average pitch for each synthesized voice from the synthesized voice pitch series data, For each of the synthesized speech data, a minimum pitch difference value for each synthesized speech, which is a minimum value obtained by subtracting the average pitch for each synthesized speech from the synthesized speech pitch sequence data, For each voice data, the maximum pitch difference for editing is obtained by dividing the maximum pitch difference value for each voice by the maximum pitch difference value for each synthesized voice corresponding to the voice data. A lower voice-specific magnification for editing is obtained by dividing another minimum pitch difference value by the synthesized voice-by-synchronized pitch difference minimum value of the synthesized voice data corresponding to the voice data. Those that exceed the reference pitch level are enlarged at the magnification for each upper voice for editing centered on the reference pitch level, and those that are below the reference pitch level among the pitch series data are centered on the reference pitch level. Generating the edit pitch series data by enlarging at the lower voice-specific magnification for editing,
The speech synthesis dictionary construction device according to any one of claims 5 to 7. The editing unit
The pitch sequence data specified by the audio data and the monophone label for each of the audio data from which the pitch sequence data to be edited is extracted and for each monophone label, end from the start time of the monophone label A value obtained by dividing the averaged result up to the time point by the result of averaging the synthesized pitch sequence data from the start time to the end time of the synthesized monophone label equal to the monophone label is obtained. The editing monophone magnification for each phone label is set, and the edit pitch series data is generated by multiplying the pitch series data by the extraction monophone magnification for each of the audio data of the extraction source and for each monophone label.
The speech synthesis dictionary construction apparatus according to claim 3. A phoneme label string and recorded voice data corresponding to the phoneme label string are acquired from the voice database, and the recorded voice pitch series data is extracted from the acquired recorded voice data, and the extracted recorded voice pitch series data and the acquired phoneme are extracted. A temporary construction step of constructing a temporary speech synthesis dictionary by HMM (Hidden Markov Model) learning based on the label sequence;
Generating a synthesized voice data based on the provisional voice synthesis dictionary, and extracting a synthesized voice pitch series data from the generated synthesized voice data; and
The recorded voice pitch sequence data extracted by the temporary construction step from the recorded voice data corresponding to the phoneme label string and the synthesized data generation step extracted from the synthesized voice data corresponding to the phoneme label string An editing step of editing the recorded voice pitch series data to generate edited pitch series data based on the result of comparing the synthesized voice pitch series data;
A reconstructing step of constructing a speech synthesis dictionary by HMM learning based on the phoneme label string and the editing pitch sequence data generated by the editing step;
A speech synthesis dictionary construction method comprising: On the computer,
A phoneme label string and recorded voice data corresponding to the phoneme label string are acquired from the voice database, and the recorded voice pitch series data is extracted from the acquired recorded voice data, and the extracted recorded voice pitch series data and the acquired phoneme are extracted. A temporary construction step of constructing a temporary speech synthesis dictionary by HMM (Hidden Markov Model) learning based on the label sequence;
Generating a synthesized voice data based on the provisional voice synthesis dictionary, and extracting a synthesized voice pitch series data from the generated synthesized voice data; and
The recorded voice pitch sequence data extracted by the temporary construction step from the recorded voice data corresponding to the phoneme label string and the synthesized data generation step extracted from the synthesized voice data corresponding to the phoneme label string An editing step of editing the recorded voice pitch series data to generate edited pitch series data based on the result of comparing the synthesized voice pitch series data;
A reconstructing step of constructing a speech synthesis dictionary by HMM learning based on the phoneme label string and the editing pitch sequence data generated by the editing step;
A computer program that executes

Images