JP2006195207A - Device and method for synthesizing voice, and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice synthesizer or the like permitting to obtain a synthesized voice of a long time length within a short time by simple processing or without having anything to do with a storage capacity of a temporary storage device. <P>SOLUTION: The voice synthesizer makes a sound piece editing part 5 retrieve and output a sound piece data of a sound piece agreeing with a sound piece in reading in a fixed form message from a sound piece data base 7. When an ID is contained in the fixed form message, the voice synthesizer makes the sound piece editing part retrieve and output a sound piece data made to correspond to a sound piece reading data agreeing with the reading of this ID. On the other hand, the sound piece editing part 5 selects one by one a sound piece best agreeable with each sound piece in the fixed form message from the retrieved sound piece data. As to a sound piece which could not be selected, the sound editing part 5 makes a sound processing part 41 supply a waveform data presenting a waveform for each unit voice. Then, a synthesized voice is reproduced by performing a prescribed processing to the selected sound piece data and the waveform data supplied by the sound processing part 41 by the portion which can be stored in a cache memory simultaneously. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a speech synthesizer, a speech synthesis method, and a program.

As a technique for synthesizing speech, there is a technique called a recording editing system. The recording / editing system is used in a station voice guidance system, an in-vehicle navigation system, and the like.
In the recording and editing method, a word is associated with speech piece data representing a speech that reads out the word, a sentence to be synthesized is divided into words, and the speech piece data associated with these words is obtained. It is a method of acquiring and joining.

However, when the piece data are simply joined together, the synthesized speech becomes unnatural because the frequency of the pitch component of the speech changes discontinuously at the boundary between the piece pieces data. Therefore, in order to solve this problem, multiple pieces of speech data representing speech that reads out the same phoneme with different prosody are prepared, while prosodic prediction is performed on the text to be synthesized, and it matches the prediction result. A technique of selecting and connecting sound piece data is performed (for example, see Patent Document 1).
Japanese Patent Application No. 2004-142907

However, if the speech to be synthesized is long, if prosodic prediction is performed on a sentence representing the entire speech, it takes a long time to complete the prosodic prediction, and the time taken until the synthesized speech is actually output. Will also be long. In addition, when the speech piece data to be selected for speech synthesis is known in advance, a part corresponding to the speech represented by the speech piece data in the sentence is converted from a phonetic character string to a plurality of speech pieces. Since processing such as searching for data and selecting optimum sound piece data is not necessary, if such useless processing is performed, the time until the synthesized speech is output is also prolonged. Therefore, an ID is assigned to a phrase having a length equal to or longer than a predetermined length, and sound piece data corresponding to this phrase is stored in association with this ID. When this ID is supplied, this ID is stored. It has been considered to synthesize desired speech using sound piece data associated with.
However, in the conventional speech synthesis, the text to be synthesized is acquired as an ideographic character string, and a phonogram string is generated from the ideographic character string by performing morphological analysis on the ideographic character string. There is a case in which a method of selecting speech piece data corresponding to each part of the phonetic character string may be taken. In this case, when using the above-described method of synthesizing speech using sound piece data associated with the supplied ID, complicated processes such as processing for preventing morphological analysis from being performed on the ID by mistake. A separate process has become necessary.

On the other hand, the upper limit of the amount of speech piece data that can be processed continuously at once by a device that performs speech synthesis (processing for generating speech data representing continuous speech) using speech piece data is generally a cache memory or the like. Depending on the storage capacity of the temporary storage device. For this reason, when sound piece data exceeding the storage capacity of the temporary storage device is supplied, there is a problem that it takes a long time to obtain an audio output.
In other words, when speech synthesis is performed by receiving supply of sound piece data in an amount exceeding this upper limit, such a device, when the data amount of the supplied sound piece data exceeds the storage capacity of the temporary storage device, Of this piece of piece data, the piece of piece of piece data that can be stored in the temporary storage device is stored in this piece of temporary storage device. When this process ends, the audio data generated by this process is stored in a second storage device (other work memory, HDD, etc.) other than the temporary storage device. Next, of the remaining sound piece data, the amount that can be stored in the temporary storage device is stored in the temporary storage device, and predetermined processing is performed only for this amount, and when this processing ends, the audio data generated by this processing Is stored in the second storage device after the previously stored audio data. Thereafter, this is repeated, and when all the speech synthesis processing using the supplied speech piece data is completed, the speech data stored in the second storage device is output to reproduce the speech. Since the sound is reproduced through the processing as described above, it takes a long time to obtain the sound output.

The present invention has been made in view of the above circumstances, and a speech synthesizer and speech synthesis method capable of obtaining synthesized speech having a long time in a short time by simple processing, and such speech synthesizer and speech synthesis. An object is to provide a program for realizing the method.
The present invention also realizes a speech synthesizer and speech synthesis method capable of obtaining speech output in a short time regardless of the storage capacity of a temporary storage device such as a cache memory, and such a speech synthesizer and speech synthesis method. The purpose is to provide a program to do this.

In order to achieve the above object, a speech synthesizer according to the first aspect of the present invention provides:
The speech piece data representing the speech piece and having the data amount equal to or smaller than the predetermined amount is associated with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID for identifying the speech piece data. Sound piece storage means for storing;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Temporary storage means for temporarily storing the predetermined amount or less of the piece data selected by the selection means;
Processing means for performing predetermined processing on the sound piece data temporarily stored by the temporary storage means;
Replaying means for playing back the sound represented by the sound piece data processed by the processing means,
It is characterized by that.

Prosody prediction means for predicting the prosody of the ideogram or ID may further be provided,
The sound piece storage means stores the prosody data representing the time change of the pitch of the sound piece represented by the sound piece data or the time change of the pitch of the voice that reads out the ID associated with the sound piece data. It may be stored in association with the data,
The selection means has a predetermined range of prosody predicted by the prosody prediction means from the speech piece data associated with the phonetic character string that matches the phonetic character string obtained as a result of the conversion by the morphological analysis unit. It is also possible to select speech piece data associated with prosodic data representing the prosody that matches with.

  The selection unit may exclude sound piece data associated with prosody data representing a prosody that does not match the prosody predicted by the prosody prediction unit within a predetermined range from selection targets.

The sound piece storage means may not store the prosodic data associated with the sound piece data for the sound piece data associated with the phonetic character representing the reading of the ID,
The selecting means selects speech piece data not associated with prosodic data from speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit. You may choose.

  The predetermined process may include a process of converting the sound piece data so that the time length of the sound piece represented by the sound piece data temporarily stored in the temporary storage unit matches a predetermined time length. .

  In the predetermined processing, the time length of the sound piece represented by the sound piece data temporarily stored by the temporary storage unit matches the predetermined time length, and the pitch frequency of the sound piece represented by the sound piece data is a predetermined condition. It may include a process of converting the sound piece data so as to match.

The predetermined process is:
A first process for converting the sound piece data so that the time length of the sound piece represented by the sound piece data temporarily stored by the temporary storage means matches a predetermined time length;
A second process for generating data representing the product of the instantaneous value of the waveform of the sound piece represented by the sound piece data that has undergone the first process and the instantaneous value of the first local oscillation signal;
A third component that extracts a component having a frequency corresponding to the sum of the frequency of the sound piece data temporarily stored by the temporary storage unit and the frequency of the first local oscillation signal from the data generated by the second process Processing,
A fourth process for generating data representing the product of the instantaneous value of the waveform of the sound piece represented by the component obtained by the third process and the instantaneous value of the second local oscillation signal;
Fifth processing for extracting a component having a frequency corresponding to the difference between the frequency of the component obtained by the third processing and the frequency of the second local oscillation signal from the data generated by the fourth processing It may consist of:
In this case, the fourth process corresponds to, for example, the difference between the frequency of the component obtained by the third process and the frequency of the second local oscillation signal among the data obtained by the fourth process. Based on the prosodic data, the second local part is such that the pitch frequency of the sound piece represented by the component having the frequency matches the pitch frequency of the sound piece represented by the sound piece data before undergoing the first processing. It only has to include a process for determining the frequency of the oscillation signal.

A speech synthesizer according to the second aspect of the present invention provides:
Sound piece storage means for storing sound piece data representing a sound piece in association with a phonetic character representing a reading of the sound piece data or a phonetic character representing an ID reading identifying the sound piece data;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Dividing means for dividing, among the sound piece data selected by the selecting means, sound piece data whose data amount exceeds a predetermined amount into a plurality of sound piece data whose data amount is equal to or less than the predetermined amount;
Temporary storage temporarily stored for the piece of piece data selected by the selection unit and having the data amount equal to or less than the predetermined amount or the piece of piece data obtained as a result of division by the dividing unit. Storage means;
Processing means for performing predetermined processing on the sound piece data temporarily stored by the temporary storage means;
Replaying means for playing back the sound represented by the sound piece data processed by the processing means,
It is characterized by that.

  The dividing unit may divide sound piece data having a data amount exceeding a predetermined amount at a position corresponding to a substantially silent section of sound pieces represented by the sound piece data.

Moreover, the speech synthesis method according to the third aspect of the present invention provides:
The speech piece data representing the speech piece and having the data amount equal to or smaller than the predetermined amount is associated with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID for identifying the speech piece data. Remember,
The ideogram or ID supplied from the outside is converted into a phonogram string representing the reading of the ideogram or ID,
Select the speech piece data associated with the phonetic character string that matches the phonetic character string obtained as a result of the conversion,
Of the selected piece data, the portion below the predetermined amount is temporarily stored,
Apply predetermined processing to the temporarily stored sound piece data,
Play the sound represented by the processed piece data,
It is characterized by that.

A speech synthesis method according to the fourth aspect of the present invention provides:
Storing the speech piece data representing the speech piece in association with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID identifying the speech piece data;
The ideogram or ID supplied from the outside is converted into a phonogram string representing the reading of the ideogram or ID,
Select the speech piece data associated with the phonetic character string that matches the phonetic character string obtained as a result of the conversion,
Of the selected sound piece data, the sound piece data whose data amount exceeds a predetermined amount is divided into a plurality of sound piece data whose data amount is equal to or less than the predetermined amount,
Of the selected piece of piece data whose data amount is less than or equal to the predetermined amount, or of the piece of piece data obtained as a result of the division, temporarily store the amount less than or equal to the predetermined amount,
Apply predetermined processing to the temporarily stored sound piece data,
Play the sound represented by the processed piece data,
It is characterized by that.

A program according to the fifth aspect of the present invention is
Computer
The speech piece data representing the speech piece and having the data amount equal to or smaller than the predetermined amount is associated with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID for identifying the speech piece data. Sound piece storage means for storing;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selecting means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Temporary storage means for temporarily storing less than the predetermined amount of the selected sound piece data;
Processing means for performing predetermined processing on the sound piece data stored in the temporary storage means;
Reproduction means for reproducing the sound represented by the piece data processed by the processing means;
It is for making it function.

A program according to the sixth aspect of the present invention is
Computer
Sound piece storage means for storing sound piece data representing a sound piece in association with a phonetic character representing a reading of the sound piece data or a phonetic character representing an ID reading identifying the sound piece data;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Dividing means for dividing, among the sound piece data selected by the selecting means, sound piece data whose data amount exceeds a predetermined amount into a plurality of sound piece data whose data amount is equal to or less than the predetermined amount;
Temporarily storing temporarily less than the predetermined amount of sound piece data selected by the selection means and having a data amount equal to or less than the predetermined amount or sound piece data obtained as a result of division by the dividing means. Storage means;
Processing means for performing predetermined processing on the sound piece data stored in the temporary storage means;
Playback means for playing back the sound represented by the sound piece data processed by the processing means;
It is for making it function.

According to the present invention, there is provided a speech synthesizer and a speech synthesis method capable of obtaining a synthesized speech having a long time in a short time by a simple process, and a program for realizing such a speech synthesizer and speech synthesis method. Realized.
The present invention also realizes a speech synthesizer and speech synthesis method capable of obtaining speech output in a short time regardless of the storage capacity of a temporary storage device such as a cache memory, and such a speech synthesizer and speech synthesis method. Program is implemented.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking a speech synthesis system as an example.

(First embodiment)
FIG. 1 is a diagram showing a configuration of a speech synthesis system according to the first embodiment of the present invention. As shown in the figure, this speech synthesis system is composed of a main unit M and a sound piece registration unit R.

The main unit M includes a language processing unit 1, a general word dictionary 2, a user word dictionary 3, a rule synthesis processing unit 4, a sound piece editing unit 5, a search unit 6, a sound piece database 7, and an expansion unit. 8.
Among these, the rule synthesis processing unit 4 includes an acoustic processing unit 41, a search unit 42, an extension unit 43, and a waveform database 44.
The sound piece editing unit 5 includes a morphological analysis unit 51, a matching sound piece determination unit 52, a prosody prediction unit 53, and an output synthesis unit 54.

  The language processing unit 1, the acoustic processing unit 41, the search unit 42, the expansion unit 43, the sound piece editing unit 5, the search unit 6, and the expansion unit 8 are all CPU (Central Processing Unit), DSP (Digital Signal Processor), etc. , A RAM (Random Access Memory), and a storage device such as a hard disk device, each of which performs processing to be described later. The storage device is a nonvolatile memory that stores a program to be executed by the processor, a volatile memory that functions as a main memory (main storage device) of the processor, and a volatile memory that functions as a cache memory of the processor. It consists of a memory.

  The output combining unit 54 further includes, for example, a D / A (Digital-to-Analog) converter, an AF (Audio Frequency) amplifier, a speaker, and the like.

  A single processor performs a part or all of the functions of the language processing unit 1, the acoustic processing unit 41, the search unit 42, the expansion unit 43, the sound piece editing unit 5, the search unit 6 and the expansion unit 8. Also good. Therefore, for example, a processor that performs the function of the decompression unit 43 may perform the function of the decompression unit 8, or a single processor may perform the functions of the acoustic processing unit 41, the search unit 42, and the decompression unit 43. Good.

  The general word dictionary 2 is composed of a nonvolatile memory such as a PROM (Programmable Read Only Memory) or a hard disk device. In the general word dictionary 2, words including ideographic characters (for example, kanji) and phonograms (for example, kana and phonetic symbols) representing the reading of these words are the manufacturer of this speech synthesis system. Etc., and stored in advance in association with each other.

The user word dictionary 3 includes a nonvolatile memory capable of rewriting data such as an EEPROM (Electrically Erasable / Programmable Read Only Memory) or a hard disk device, and a control circuit for controlling writing of data to the nonvolatile memory. . The processor may perform the function of the control circuit, and part of the language processing unit 1, the sound processing unit 41, the search unit 42, the expansion unit 43, the sound piece editing unit 5, the search unit 6, and the expansion unit 8 or A processor that performs all the functions may perform the function of the control circuit of the user word dictionary 3.
The user word dictionary 3 obtains words including ideograms and phonograms representing readings of these words from the outside according to user operations, and stores them in association with each other. It is sufficient that the user word dictionary 3 stores words and the like that are not stored in the general word dictionary 2 and phonograms representing the readings.

  The waveform database 44 is composed of a nonvolatile memory such as a PROM or a hard disk device. In the waveform database 44, a phonetic character and a voice constituting a phoneme represented by the phonetic character (that is, a voice of one cycle (or other predetermined number of cycles) of a voice waveform constituting one phoneme). ) And the compressed waveform data obtained by entropy coding the segment waveform data representing the data are stored in advance in association with each other by the manufacturer of the speech synthesis system. Note that the segment waveform data before entropy encoding may be, for example, PCM digital data.

The sound piece database 7 is composed of a nonvolatile memory such as a PROM or a hard disk device.
The sound piece database 7 stores, for example, data having a data structure shown in FIG. That is, as shown in the figure, the data stored in the sound piece database 7 is divided into four types: a header part HDR, an index part IDX, a directory part DIR, and a data part DAT.

  The data storage in the sound piece database 7 is performed, for example, in advance by the manufacturer of the speech synthesis system and / or by the sound piece registration unit R performing an operation described later.

  The header portion HDR stores data for identifying the sound piece database 7 and data indicating the index portion IDX, the data amount of the directory portion DIR and the data portion DAT, the format of the data, the attribution of the copyright, and the like.

The data portion DAT stores compressed sound piece data obtained by entropy encoding sound piece data representing a sound piece waveform.
Note that a sound piece refers to a continuous section including one or more phonemes in speech, and usually includes a section for one word or a plurality of words. Sound pieces may contain conjunctions.
Further, the sound piece data before entropy encoding is obtained from data in the same format as the waveform data before entropy encoding for generating the compressed waveform data (for example, data in digital format converted to PCM). It only has to be. Note that the length (data amount) of one piece of speech piece data before entropy coding is an amount that does not exceed the storage capacity of the cache memory.

In the directory part DIR, for each compressed sound piece data,
(A) phonetic data (speech reading data) representing phonetic characters indicating the reading of the speech piece represented by the compressed speech piece data;
(B) data representing the head address of the storage location where the compressed sound piece data is stored;
(C) data representing the data length of this compressed sound piece data;
(D) data (speed initial value data) representing the utterance speed of the sound piece represented by this compressed sound piece data (time length when played back),
(E) As an indication of the prosody of this sound piece, prosodic data (pitch component data) representing the time change of the frequency of the pitch component of this sound piece,
Are stored in association with each other. (It is assumed that an address is assigned to the storage area of the sound piece database 7.)

  In FIG. 2, as data included in the data portion DAT, compressed sound piece data having a data amount of 1410 h bytes representing a waveform of a sound piece whose reading is “Saitama” is in a logical position starting at the address 001A36A6h. The case where it is stored is illustrated. (In this specification and drawings, the number with “h” at the end represents a hexadecimal number.)

It should be noted that at least the data (A) (that is, the speech piece reading data) of the data sets (A) to (E) is sorted according to the order determined based on the phonetic characters represented by the speech piece reading data. (For example, if the phonetic character is kana, the phonetic characters are arranged in descending order of addresses in the order of the Japanese syllabary) and are stored in the storage area of the speech database 7.
In addition, the above-described pitch component data includes, for example, as shown in the figure, when the frequency of the pitch component of the sound piece is approximated by a linear function of the elapsed time from the head of the sound piece, What is necessary is just to consist of the data which show the value of gradient (alpha). (The unit of the gradient α may be [Hertz / second], for example, and the unit of the intercept β may be [Hertz], for example.)
Further, the pitch component data further indicates whether the sound piece represented by the compressed sound piece data is a voiced sound or an unvoiced sound, whether it is a nasal muffled sound, and whether it is unvoiced. Data is also included.

The compressed sound piece data is obtained by entropy encoding sound piece data having a length that does not exceed the storage capacity of the cache memory, obtained by dividing one long piece of sound piece data that exceeds the storage capacity of the cache memory. In some cases.
It is assumed that an ID (IDentifier) for identifying the compressed sound piece data is assigned in advance to such compressed sound piece data, and the sound piece reading data associated with such compressed sound piece data is Suppose that it is not a reading of the sound piece represented by the sound piece data, but a phonetic character string representing the reading of the ID of the compressed sound piece data. Further, the pitch component data associated with such compressed sound piece data is not the time change of the pitch component of the sound piece represented by the compressed sound piece data, but the pitch component of the voice that reads the ID of the compressed sound piece data. It is assumed to consist of prosodic data representing temporal changes.

That is, for example, the sound piece database 7 represents a voice that reads out one long sentence “scroll the list to display one page before. You cannot press while the first page is displayed”. 4 pieces of speech that represent the parts that are read out as “long scrolling list”, “displaying the previous page”, “while displaying the first page” and “cannot press” It is divided into data (the length of these sound piece data shall not exceed the storage capacity of the cache memory), and four pieces of compressed sound piece data each consisting of entropy-coded data are stored. It shall be.
Of these four compressed sound piece data, for example, the sound piece data corresponding to the sound piece “Scroll the list” is associated with “A AB” as an ID. For each piece of speech data corresponding to the pieces “display the previous page”, “while the first page is displayed” or “cannot be pressed”, for example, “C D D”, “ It is assumed that “E F” or “G key” is assigned.
In this case, each sound piece data corresponding to the sound pieces “scroll the list”, “display the previous page”, “while the first page is displayed” or “cannot be pressed” It is assumed that “ABI”, “SHIUDIE”, “IOFuka”, or “Zikiech” are sequentially stored as one-side reading data in association with each other.
In addition, each piece of speech data corresponding to the sound pieces “scroll the list”, “display the previous page”, “while the first page is displayed” or “cannot be pressed” As data, it is assumed that prosody data representing temporal changes in the pitch components of speech that reads out “ABI,” “Siudie,” “IOFka,” or “Zikiech” is stored in association with each other.

  The index part IDX stores data for specifying the approximate logical position of the data in the directory part DIR based on the sound piece reading data. Specifically, for example, assuming that the sound piece reading data represents kana, the address range of the kana characters and the sound piece reading data whose first character is this kana character is in the range. Data (directory address) to be shown is stored in association with each other.

  It should be noted that the general word dictionary 2, the user word dictionary 3, the waveform database 44, the non-volatile memory of the matching sound piece determination unit 52, and a part or all of the functions of the sound piece database 7 are performed by a single non-volatile memory. It may be.

  The sound piece registration unit R includes a recorded sound piece data set storage unit 10, a sound piece database creation unit 11, and a compression unit 12, as illustrated. Note that the sound piece registration unit R may be detachably connected to the sound piece database 7. In this case, the sound piece registration unit R is not used except when new data is written to the sound piece database 7. The main unit M may be made to perform an operation described later in a state where it is separated from the main unit M.

The recorded sound piece data set storage unit 10 is composed of a rewritable nonvolatile memory such as a hard disk device.
The recorded sound piece data set storage unit 10 includes phonetic characters representing the reading of the sound pieces and sound piece data representing the waveforms obtained by collecting the sound pieces actually uttered by a person. They are stored in advance in association with each other by the manufacturer of the speech synthesis system. The sound piece data may be composed of, for example, PCM digital data.

  The sound piece database creation unit 11 and the compression unit 12 include a processor such as a CPU and a memory that stores a program to be executed by the processor, and performs processing described later according to the program.

  Note that a single processor may perform a part or all of the functions of the speech piece database creation unit 11 and the compression unit 12, and the language processing unit 1, the acoustic processing unit 41, the search unit 42, and the expansion unit. 43, a processor that performs some or all of the functions of the sound piece editing unit 5, the search unit 6, and the decompression unit 8 may further perform the functions of the sound piece database creation unit 11 and the compression unit 12. Further, a processor that performs the functions of the sound piece database creation unit 11 and the compression unit 12 may also function as a control circuit of the recorded sound piece data set storage unit 10.

  Next, the operation of this speech synthesis system will be described.

(First Embodiment: Operation of Sound Piece Registration Unit)
First, the operation of the sound piece registration unit R will be described.
When registering a sound piece in the sound piece database 7, first, the sound piece database creating unit 11 reads out the phonetic character and sound piece data associated with each other from the recorded sound piece data set storage unit 10, The time variation of the frequency of the pitch component of the voice represented by the piece data and the utterance speed are specified.

  The utterance speed may be specified by, for example, counting the number of samples of the sound piece data.

  On the other hand, the time change of the frequency of the pitch component may be specified by performing cepstrum analysis on the sound piece data, for example. Specifically, for example, the waveform represented by the sound piece data is divided into a number of small parts on the time axis, and the intensity of each obtained small part is converted to the logarithm of the original value (the base of the logarithm is arbitrary). Convert to a substantially equal value, and use this fast Fourier transform method (or generate data that represents the result of Fourier transform of discrete variables, etc.) (Any method). Then, the minimum value among the frequencies giving the maximum value of the cepstrum is specified as the frequency of the pitch component in this small portion.

  The time change of the frequency of the pitch component is specified based on the pitch waveform data after the sound piece data is converted into the pitch waveform data according to the method disclosed in Japanese Patent Laid-Open No. 2003-108172, for example. A good result can be expected. Specifically, the pitch data is extracted by filtering the piece data, and the waveform represented by the piece data is divided into sections of unit pitch length based on the extracted pitch signal. It is only necessary to convert the sound piece data into a pitch waveform signal by identifying the phase shift based on the correlation and aligning the phases of each section. Then, the obtained pitch waveform signal is handled as sound piece data, and a cepstrum analysis is performed, for example, so that the time change of the frequency of the pitch component may be specified.

On the other hand, the sound piece database creation unit 11 supplies the sound piece data read from the recorded sound piece data set storage unit 10 to the compression unit 12.
The compression unit 12 entropy-encodes the sound piece data supplied from the sound piece database creation unit 11 to create compressed sound piece data, and returns the compressed sound piece data to the sound piece database creation unit 11.

  When the time variation of the voice speed of the speech piece data and the frequency of the pitch component is specified, and this speech piece data is entropy encoded and returned as compressed speech piece data from the compression unit 12, the speech piece database creation unit 11 The compressed sound piece data is written in the storage area of the sound piece database 7 as data constituting the data part DAT.

The speech piece database creation unit 11 also uses the phonetic character database read from the recorded speech piece data set storage unit 10 as the speech piece reading data to indicate the reading of the speech piece represented by the written compressed speech piece data. 7 is written in the storage area.
Further, the head address of the written compressed sound piece data in the storage area of the sound piece database 7 is specified, and this address is written in the storage area of the sound piece database 7 as the data (B) described above.
Further, the data length of the compressed sound piece data is specified, and the specified data length is written in the storage area of the sound piece database 7 as data (C).
In addition, data indicating the result of specifying the time variation of the voice generation speed and pitch component frequency represented by the compressed sound piece data is generated and stored in the storage area of the sound piece database 7 as speed initial value data and pitch component data. Write.

(First Embodiment: Operation of Main Unit)
Next, the operation of the main unit M will be described. In the following, first, it is assumed that the language processing unit 1 has acquired free text data describing a sentence (free text) including an ideogram prepared by the user as a target for synthesizing speech in the speech synthesis system. .

  The language processing unit 1 may acquire any free text data. For example, the language processing unit 1 may acquire the free text data from an external device or a network via an interface circuit (not shown), or may be set in a recording medium drive device (not shown). Alternatively, the data may be read from a recording medium (for example, a floppy (registered trademark) disk, a CD-ROM, or the like) via the recording medium drive device.

Alternatively, the processor performing the function of the language processing unit 1 may deliver the text data used in other processing executed by itself to the processing of the language processing unit 1 as free text data.
As the other processing executed by the processor, for example, voice data representing voice is acquired, and voice recognition is performed on the voice data to identify a phrase represented by the voice. Based on the identified phrase, For example, processing for causing the processor to perform the function of the agent device that specifies the content of the request of the voice speaker and specifies and executes the processing to be executed to satisfy the specified request can be considered.

  When the free text data is acquired, the language processing unit 1 specifies a phonetic character representing the reading of each ideographic character included in the free text by searching the general word dictionary 2 and the user word dictionary 3. . Then, the ideogram is replaced with the specified phonogram. Then, the language processing unit 1 supplies the phonogram string obtained as a result of replacing all ideographic characters in the free text with phonograms to the acoustic processing unit 41 of the rule synthesis processing unit 4.

  When the sound processing unit 41 is supplied with a phonetic character string from the language processing unit 1, for each phonetic character included in the phonetic character string, the waveform of the segment constituting the phoneme represented by the phonetic character The search unit 42 is instructed to search for. The acoustic processing unit 41 supplies the phonetic character string to the prosody prediction unit 53 of the speech piece editing unit 5.

  The search unit 42 searches the waveform database 44 in response to this instruction, and searches for compressed waveform data that matches the contents of this instruction. Then, the searched compressed waveform data is supplied to the decompression unit 43.

  The decompression unit 43 restores the compressed waveform data supplied from the search unit 42 to the segment waveform data before being compressed, and returns it to the search unit 42. The search unit 42 supplies the segment waveform data returned from the decompression unit 43 to the acoustic processing unit 41 as a search result.

  On the other hand, the prosodic prediction unit 53 supplied with the phonetic character string from the acoustic processing unit 41 uses the phonetic character string as a prosody prediction method such as “Fujisaki model” or “ToBI (Tone and Break Indices)”. By adding the analysis based on this, the prosody of the speech represented by the phonetic character string (accent, intonation, stress, phoneme duration, etc.) is predicted, and prosodic prediction data representing the prediction result is generated. Then, this prosodic prediction data is supplied to the acoustic processing unit 41.

  When the acoustic processing unit 41 is supplied with the segment waveform data from the search unit 42 and is supplied with the prosody prediction data from the prosody prediction unit 53, the language processing unit 1 uses the supplied segment waveform data. Speech waveform data representing a speech waveform represented by each phonogram included in the phonogram string is generated.

  Specifically, the acoustic processing unit 41 uses, for example, the prosody supplied from the prosody prediction unit 53 to determine the time length of phonemes configured by the segments represented by the respective segment waveform data supplied from the search unit 42. Identify based on forecast data. Then, an integer closest to the value obtained by dividing the time length of the specified phoneme by the time length of the segment represented by the segment waveform data is obtained, and the segment waveform data is mutually connected by the number equal to the obtained integer. Thus, the speech waveform data may be generated.

  The acoustic processing unit 41 not only determines the time length of the speech represented by the speech waveform data based on the prosodic prediction data, but also processes the segment waveform data constituting the speech waveform data to represent the speech waveform data. The voice may have intensity, intonation, and the like that match the prosody indicated by the prosodic prediction data.

  Then, the sound processing unit 41 synthesizes the generated speech waveform data from the speech piece editing unit 5 in the order in which the phonograms are arranged in the phonogram string supplied from the language processing unit 1. To the unit 54.

  When the voice synthesis data is supplied from the acoustic processing unit 41, the output synthesis unit 54 combines the voice waveform data with each other in the order supplied by the acoustic processing unit 41, thereby expressing the data (synthetic voice). Data). Then, the output synthesizer 54 reproduces the voice represented by the synthesized voice data. This synthesized speech synthesized based on the free text data corresponds to speech synthesized by the rule synthesis method.

  When the synthesized speech data is generated, the output synthesis unit 54, if the data amount of the speech waveform data exceeds a predetermined upper limit (specifically, the storage capacity of the cache memory), first, the voice corresponding to the upper limit is not exceeded. Waveform data is stored in a cache memory. Then, after performing predetermined processing on the amount of audio waveform data stored in the cache memory, the audio waveform data subjected to this processing is stored in the storage area of the main memory, and the amount of data of the remaining audio waveform data is The voice waveform data that does not exceed the predetermined upper limit is stored in the cache memory, and predetermined processing is performed in the same manner as described above. The main memory sequentially stores the processed audio waveform data following the processed audio waveform data previously stored in the storage area of the main memory. When the output synthesizing unit 54 finishes performing the predetermined process on all the audio waveform data supplied from the acoustic processing unit 41, the output synthesizing unit 54 continues the audio waveform data stored in the main memory at this time (ie, synthesized audio data). It is assumed that the sound represented by is reproduced.

  The output synthesizer 54 may reproduce the synthesized voice data represented by the synthesized voice data, or may output the synthesized voice data to the outside by any method instead of playing the synthesized voice. Specifically, for example, it may be sent to an external device or a network via an interface circuit (not shown), or written to a recording medium set in a recording medium drive device (not shown) via this recording medium drive device. But you can. Further, the processor that performs the function of the output synthesis unit 54 may deliver the synthesized voice data to another process that is being executed by the processor.

  Next, it is assumed that the acoustic processing unit 41 acquires data representing a phonetic character string (delivery character string data) distributed from the outside. (Note that the method by which the acoustic processing unit 41 acquires the distribution character string data is also arbitrary. For example, the distribution character string data may be acquired by a method similar to the method by which the language processing unit 1 acquires the free text data. )

  In this case, the acoustic processing unit 41 handles the phonetic character string represented by the distribution character string data in the same manner as the phonetic character string supplied from the language processing unit 1. As a result, the compressed waveform data representing the phoneme constituting the phoneme represented by the phonetic character included in the phonetic character string represented by the delivery character string data is retrieved by the search unit 42, and the segment waveform data before being compressed. Is restored by the decompression unit 43. On the other hand, the prosody prediction unit 53 adds an analysis based on the prosody prediction method to the phonetic character string represented by the distribution character string data, and as a result, represents the prediction result of the prosody of the voice represented by the phonetic character string. Prosodic prediction data is generated. Then, the acoustic processing unit 41 converts the speech waveform data representing the speech waveform represented by each phonogram included in the phonogram string represented by the distribution character string data, the restored segment waveform data, and the prosody prediction data. The output synthesizer 54 synthesizes the generated speech waveform data by combining the generated speech waveform data with each other in the order of the phonograms in the phonogram string represented by the distribution character string data. The voice data is generated, the synthesized voice represented by the synthesized voice data is reproduced, or the synthesized voice data is output to the outside. This synthesized voice data synthesized based on the distribution character string data also represents voice synthesized by the rule synthesis method.

  Next, it is assumed that the sound piece editing unit 5 acquires the standard message data, the utterance speed data, and the collation level data.

  The fixed message data is data representing the fixed message as an ideographic character string. This ideographic character string may include the above-mentioned ID. Thus, for example, the creator of the standard message data, for example, reads the above-mentioned long sentence “scroll the list to display the previous page. It cannot be pressed while the first page is displayed.” If you want the main unit M to read out as a part of it, the above-mentioned four IDs “A AB”, “C D D”, “E OFF F” and “G K H” ”May be included in a continuous form.

The utterance speed data is data indicating a specified value of the utterance speed of the standard message represented by the standard message data (specified value of the time length for uttering this standard message).
The collation level data is data for designating a search condition in a search process to be described later performed by the search unit 6, and is assumed to take one of the values “1”, “2”, or “3” below, and “3”. Indicates the strictest search condition.

  Moreover, the method by which the speech piece editing unit 5 acquires the standard message data, the utterance speed data, and the collation level data is arbitrary. For example, the standard message data may be obtained by the same method as the method by which the language processing unit 1 acquires the free text data. And speaking speed data and collation level data may be acquired.

When the standard message data, the utterance speed data, and the collation level data are supplied to the sound piece editing unit 5, the morpheme analysis unit 51 of the sound piece editing unit 5 performs morphological analysis on the fixed message data by a known method. The ideographic character string constituting the standard message data is replaced with the phonetic character string. Then, the obtained phonetic character string is supplied to the matching sound piece determination unit 52.
Note that the morpheme analyzer 51 also converts the ID into a phonetic character string. Therefore, for example, when the fixed message data includes the above-mentioned four IDs “A AB”, “C D D”, “E OF F” and “G KI H KU”, the morphological analysis unit 51 converts these four IDs into four phonetic character strings of “AAVIE”, “SEUDIE”, “IOFKA”, and “Zikiech”.

  When the phonetic character string is supplied from the morphological analysis unit 51, the coincident phone piece determination unit 52 searches for all the compressed phoneme data associated with the phonetic character string that matches the phonetic character string. The search unit 6 is instructed.

  The search unit 6 searches the sound piece database 7 in response to an instruction from the matching sound piece determination unit 52, and reads the corresponding compressed sound piece data and the above-described sound piece reading associated with the corresponding compressed sound piece data. Data, speed initial value data, and pitch component data are retrieved, and the retrieved compressed sound piece data is supplied to the decompression unit 43. Even when a plurality of compressed speech piece data corresponds to a common phonetic character or phonetic character string, all the corresponding compressed speech piece data are searched for as data candidates used for speech synthesis. On the other hand, when there is a sound piece for which compressed sound piece data could not be found, the search unit 6 generates data for identifying the corresponding sound piece (hereinafter referred to as missing portion identification data).

  The decompression unit 43 restores the compressed sound piece data supplied from the search unit 6 to the sound piece data before being compressed, and returns it to the search unit 6. The search unit 6 supplies the sound piece data returned from the decompression unit 43 and the searched sound piece reading data, speed initial value data, and pitch component data to the matching sound piece determination unit 52 as search results. . When missing part identification data is generated, this missing part identification data is also supplied to the matching sound piece determination unit 52.

  When the sound piece data, the sound piece reading data, and the pitch component data are supplied from the search unit 6, the coincidence sound piece determination unit 52 converts the sound piece data from the supplied sound piece data into the waveform of the sound piece constituting the standard message. One piece of sound piece data representing a waveform that can be approximated is selected for each piece of sound piece. However, the matching sound piece determination unit 52 sets, according to the collation level data supplied to the sound piece editing unit 5, what kind of waveform is to be a waveform close to the sound piece of the standard message.

  Specifically, first, the matching sound piece determination unit 52 supplies, for example, fixed message data to the prosody prediction unit 53, and instructs the prosody prediction unit 53 to predict the prosody of the fixed message represented by the fixed message data. . In accordance with this instruction, the prosody prediction unit 53 performs analysis based on the above-described prosodic prediction method, predicts the prosody of this fixed message, generates prosodic prediction data representing the prediction result, and matches the speech piece determination unit Return to 52. The prosody prediction unit 53 also predicts the ID prosody.

When the prosody prediction data is acquired, the matching sound piece determination unit 52, for example,
(1) When the value of the collation level data is “1”, all the speech piece data supplied from the search unit 6 (that is, speech piece data whose reading matches the speech piece in the standard message) Select the one that is close to the waveform of the sound piece inside.

(2) When the value of the collation level data is “2”, the condition of (1) (that is, the condition that the phonetic character representing the reading is matched) is satisfied, and the frequency of the pitch component frequency of the sound piece data is further satisfied. When there is a strong correlation of a predetermined amount or more between the content of the pitch component data representing the time change and the prediction result of the prosody of the speech piece (for example, accent) included in the fixed message (for example, the time difference between the accent positions is a predetermined amount) (If it is the following), the sound piece data is selected as being close to the waveform of the sound piece in the standard message. Note that the prediction result of the accent of the sound piece in the standard message can be specified from the prediction result of the prosody of the standard message, and the matching sound piece determination unit 52 is predicted to have the highest frequency of the pitch component, for example. What is necessary is just to interpret the position which is the predicted position of the accent. On the other hand, for the position of the accent of the sound piece represented by the sound piece data, for example, the position where the frequency of the pitch component is the highest is specified based on the above-described pitch component data, and this position is interpreted as the position of the accent. Good. The prosody prediction may be performed on the entire sentence, or the sentence may be divided into predetermined units and performed on each unit.

(3) When the value of the collation level data is “3”, the condition of (2) (that is, the condition of coincidence of phonetic characters and accents indicating reading) is satisfied, and further, The sound piece data is selected as being close to the waveform of the sound piece in the fixed message only when the presence or absence of nasal muffler or devoicing matches the prosodic prediction result of the fixed message. The coincident sound piece determination unit 52 may determine whether or not the voice represented by the sound piece data is made nasalized or unvoiced based on the pitch component data supplied from the search unit 6.

  In addition, when there are a plurality of pieces of sound piece data that match the conditions set by itself, the matching sound piece determination unit 52 sets the pieces of sound piece data to 1 according to conditions that are stricter than the set conditions. We shall narrow down to pieces.

  Specifically, for example, when the set condition corresponds to the value “1” of the collation level data and there are a plurality of corresponding piece of piece data, it corresponds to the value “2” of the collation level data. If the search condition is also selected and multiple pieces of sound piece data are selected, the selection result that further matches the search condition corresponding to the collation level data value “3” is further selected. Perform operations such as If a plurality of pieces of sound piece data still remain after being narrowed down by the search condition corresponding to the value “3” of the collation level data, the remaining one may be narrowed down to one on an arbitrary basis.

Then, the matching sound piece determination unit 52 supplies the sound piece data selected as satisfying the condition corresponding to the value of the collation level data to the output composition unit 54.
However, the matching sound piece determination unit 52 corresponds to the case where there is a sound piece that cannot select sound piece data that satisfies the condition corresponding to the value of the collation level data from the sound piece data supplied from the search unit 6. The sound piece is determined to be treated as a sound piece for which the search unit 6 cannot find the compressed sound piece data (that is, the sound piece indicated by the above-described missing portion identification data).

  On the other hand, the coincidence sound piece determination unit 52 has received a piece of sound for which no sound piece data satisfying the condition corresponding to the value of the collation level data has been selected when missing portion identification data is also supplied from the search unit 6. In the case, the phonetic character string representing the reading of the sound piece indicated by the missing part identification data (including the sound piece that could not select the sound piece data that satisfies the condition corresponding to the value of the collation level data) It is extracted and supplied to the sound processing unit 41, and an instruction is given to synthesize the waveform of this sound piece.

  Upon receiving the instruction, the acoustic processing unit 41 treats the phonetic character string supplied from the matching sound piece determination unit 52 in the same manner as the phonetic character string represented by the distribution character string data. As a result, the compressed waveform data representing the segments constituting the phonemes represented by the phonetic characters included in the phonetic character string is retrieved by the search unit 42, and the segment waveform data before being compressed is expanded by the decompressing unit 43. Restored. On the other hand, the prosody prediction unit 53 generates prosody prediction data representing the prediction result of the prosody of the speech piece represented by the phonetic character string. Then, the acoustic processing unit 41 generates speech waveform data representing the speech waveform represented by each phonogram included in the phonogram string based on each restored unit waveform data and prosodic prediction data. Then, the generated speech waveform data is supplied to the output synthesis unit 54.

  Note that the coincidence sound piece determination unit 52 generates a portion corresponding to the sound piece indicated by the missing part identification data from the prosodic prediction data already generated by the prosody prediction unit 53 and supplied to the coincidence sound piece determination unit 52. In this case, the acoustic processing unit 41 does not need to cause the prosody prediction unit 53 to perform prosody prediction of the sound piece again. In this way, it is possible to utter more naturally than when prosodic prediction is performed for each fine unit such as a sound piece.

  When the output synthesizing unit 54 is supplied with the sound piece data from the coincidence sound piece determining unit 52 and the sound processing unit 41 is supplied with the sound waveform data generated from the unit waveform data, the sound piece data and the sound are supplied. Generating synthesized voice data by combining the waveform data with each other in the order of each sound piece or phoneme in the standard message indicated by the standard message data, and reproducing the synthesized voice represented by the synthetic voice data, or This synthesized voice data is output to the outside.

  When generating the synthesized speech data composed of a plurality of speech piece data, the output synthesis unit 54 first selects the first sound if the data amount of each piece of speech piece data or each piece of speech waveform data does not exceed the above upper limit. One piece of data or voice waveform data is stored in a cache memory, a predetermined process is performed, and a voice represented by the voice piece data or voice waveform data is reproduced or output to the outside. Subsequently, the sound piece data or audio waveform data to be reproduced and output next is stored in the cache memory, and thereafter the audio is reproduced or output in the same manner as described above. This is repeated until the reproduction and output of all sound piece data or audio waveform data is completed.

  The predetermined process performed by the output synthesis unit 54 on the sound piece data or the sound waveform data in the storage area of the cache memory is, for example, converting the sound piece data or the sound waveform data representing voiced sound to This includes speech speed conversion processing in which the time length of the sound piece represented by the data or the sound waveform data matches the speed indicated by the utterance speed data supplied to the sound piece editing unit 5.

As a specific speech speed conversion process, the output composition unit 54 may perform processes shown as (A1) to (A3) below, for example.
(A1) By converting the time interval of the samples constituting the sound piece data stored in the cache memory, the time length of the sound piece represented by this sound piece data is matched with the speed indicated by the utterance speed data. When the data stored in the cache memory is speech waveform data, for example, the sampling time interval is set at the same rate as the sampling rate conversion rate of the sound piece data that has been subjected to the processing of (A1) immediately before. Perform conversion.
(A2) On the other hand, the pitch component data representing the prosody of the sound piece represented by the sound piece data stored in the cache memory is acquired from the matching sound piece determination unit 52.
(A3) Pitch component data obtained by converting the sound piece data whose time length is converted as a result of the process of (A1) and obtaining the pitch frequency of the sound piece represented by the sound piece data by the process of (A2). To match the frequency indicated by. When the data stored in the cache memory is voice waveform data, for example, conversion is performed so as to represent a voice having the same pitch frequency as that of the sound piece data subjected to the processing of (A3) immediately before. Do.

Specifically, for example, the output combining unit 54 may perform the following processes (A3a) to (A3d) as the process (A3).
(A3a) The sound piece data that has undergone the processing of (A1) is mixed with the first local oscillation signal having a constant amplitude and frequency. That is, data representing the product of the instantaneous value of the waveform of the sound piece represented by the sound piece data subjected to the processing of (A1) and the instantaneous value of the first local oscillation signal is generated.
(A3b) By filtering the data obtained as a result of the processing of (A3a), among the data, the sum of the frequency of the sound piece data that has undergone the processing of (A1) and the frequency of the first local oscillation signal A component having a frequency corresponding to that is extracted.
(A3c) The component obtained as a result of the processing of (A3b) is mixed with the second local oscillation signal whose amplitude and frequency are constant. However, the frequency of the second local oscillation signal has a frequency corresponding to the difference between the frequency of the component obtained by the processing of (A3b) and the frequency of the second local oscillation signal in the data obtained by the mixing. It is assumed that the sound piece represented by the component is determined based on the pitch component data so as to have the pitch frequency indicated by the pitch component data acquired in the process (A2).
(A3d) By filtering the data obtained as a result of the processing of (A3c), the difference between the frequency of the component obtained by the processing of (A3b) and the frequency of the second local oscillation signal among the data A component having a frequency corresponding to that is extracted. This component extracted as a result of the process (A3d) corresponds to the sound piece data subjected to the process (A3) described above.

When generating the synthesized voice data, the output synthesis unit 54 stores in the cache memory the amount not exceeding the upper limit if the amount of the piece data or the voice waveform data exceeds the upper limit. And Then, after performing predetermined processing on the amount stored in the cache memory, the sound piece data or voice waveform data subjected to this processing is stored in the storage area of the main memory, and the remaining sound piece data or voice waveform data is stored. Of these, the predetermined processing is continued after storing the amount not exceeding the upper limit in the storage area of the cache memory. Then, the output synthesizer 54 sequentially stores the sound piece data or the sound waveform data that has undergone the predetermined processing in the storage area of the main memory so as to be continuous with each other. Then, when all of the sound piece data and the sound waveform data have undergone the predetermined processing, the sound represented by the continuation of the sound piece data and the sound waveform data (that is, the synthesized sound data) stored in the main memory at this time is obtained. It shall be played back or output to the outside.
However, as described above, since the data amount of each piece of sound piece data that is the target of output or sound reproduction does not exceed the above-described upper limit, the output composition unit 54 does not go through temporary storage in the main memory and Audio reproduction or output is sequentially performed for each piece of data or audio waveform data. For this reason, there is no delay before the reproduction output is obtained.

  If the missing part identification data is not included in the data supplied from the search unit 6, the sound piece data selected by the sound piece editing unit 5 is immediately selected without instructing the sound processing unit 41 to synthesize the waveform. The phonogram strings in the standard message indicated by the standard message data may be combined with each other in the order in accordance with the data and output as data representing the synthesized speech.

  In the speech synthesis system according to the first embodiment of the present invention described above, the speech piece data representing the waveform of a speech piece that may be a unit larger than a phoneme is naturally converted by the recording and editing method based on the prosodic prediction result. The voice that reads out the standard message is synthesized. The storage capacity of the sound piece database 7 can be reduced as compared with the case of storing a waveform for each phoneme, and can be searched at high speed. Therefore, the speech synthesis system can be configured to be small and light, and can follow high-speed processing.

  In addition, the sound piece data whose data amount exceeds the upper limit is divided in advance into a plurality of sound piece data whose data amount does not exceed the upper limit, so that output synthesis is performed to store the sound piece data in the storage area of the main memory. It is difficult for the unit 54 to temporarily suspend speech synthesis. That is, for example, as a standard message (or part of it), the above long sentence “scroll the list to display the previous page. You cannot press while displaying the first page.” If you want the main unit M to read aloud, the above-mentioned four IDs “A AB”, “C UD D”, “E OF F” and “G KI H KU” are added to the standard message data. By including it in a continuous form, the voice that reads out this long sentence is immediately reproduced without interruption.

  In addition, this speech synthesis system performs speech synthesis normally while adopting a configuration in which morphological analysis is performed on IDs without distinguishing them from other ideographic character strings. For this reason, a synthesized speech having a long time length can be obtained in a short time by a simple process.

  On the other hand, a sound piece for which appropriate sound piece data could not be selected is synthesized in accordance with a rule synthesis method using compressed waveform data representing a piece that is a unit smaller than a phoneme. Since the compressed waveform data represents the waveform of the segment, the storage capacity of the waveform database 44 can be reduced as compared with the case where the compressed waveform data represents the waveform of the phoneme and can be searched at high speed. Therefore, the speech synthesis system can be configured to be small and light, and can follow high-speed processing.

Also, if rule synthesis is performed using segments, unlike the case where rule synthesis is performed using phonemes, speech synthesis can be performed without being affected by special waveforms that appear at the end of phonemes. Natural sounds can be obtained with a small number of segments.
In other words, it is known that in speech uttered by humans, a special waveform affected by both of these phonemes appears at the boundary where the preceding phoneme transitions to the subsequent phoneme. The phonemes already contain this special waveform at the end when they are collected, so when regular synthesis is performed using phonemes, it is possible to reproduce various patterns of waveforms at the boundaries between phonemes. It is necessary to be satisfied by preparing a huge number of phonemes or by synthesizing synthesized speech in which the waveform at the boundary between phonemes is different from natural speech. However, when performing rule synthesis using segments, if the segments are collected from portions other than the end portions of phonemes, the influence of a special waveform at the boundary between phonemes can be eliminated in advance. For this reason, it is not necessary to prepare enormous kinds of segments, and natural speech can be obtained.

Note that the configuration of this speech synthesis system is not limited to that described above.
For example, the segment waveform data does not have to be PCM format data, and the data format is arbitrary. Further, the waveform database 44 does not necessarily store the unit waveform data and sound piece data in a compressed state. When the waveform database 44 stores the segment waveform data in a state where the data is not compressed, the main body unit M does not need to include the decompression unit 43.

The coincidence sound piece determination unit 52 stores prosody registration data representing the prosody of a specific sound piece in advance, and if this specific sound piece is included in the standard message, the prosody represented by this prosody registration data is stored. Alternatively, it may be handled as a result of prosodic prediction.
The coincidence piece determining unit 52 may newly store the results of past prosody prediction as prosodic registration data.

For example, the matching sound piece determination unit 52 acquires free text data together with the language processing unit 1, and sounds that match at least a part of the speech (phonetic character string) included in the free text represented by the free text data. The piece data may be selected by performing substantially the same process as the sound piece data selection process of the standard message, and used for voice synthesis.
In this case, for the sound piece selected by the matching sound piece determination unit 52, the acoustic processing unit 41 does not have to search the search unit 42 for compressed waveform data representing the waveform of the piece of the sound piece. Note that the matching sound piece determination unit 52 notifies the sound processing unit 41 of a sound piece that the sound processing unit 41 does not need to synthesize, and the sound processing unit 41 responds to the notification to form a unit constituting this sound piece. The search for the waveform of the speech segment may be stopped.

  For example, the matching sound piece determination unit 52 acquires the distribution character string data together with the acoustic processing unit 41, and converts the sound piece data representing the phonetic character string included in the distribution character string represented by the distribution character string data into the fixed form. Selection may be made by performing substantially the same process as the process for selecting the message piece data of the message and used for speech synthesis. In this case, for the sound piece represented by the sound piece data selected by the matching sound piece determining unit 52, the acoustic processing unit 41 searches the compressed waveform data representing the waveform of the unit speech element constituting this sound piece. You don't have to search.

  Further, the compressed waveform data stored in the waveform database 44 is not necessarily required to represent a segment, and for example, unit speech waveform data representing a unit speech waveform represented by a phonetic character stored in the waveform database 44, or Data obtained by entropy encoding the unit speech waveform data may be used. The unit speech is speech that is short enough to be used in the rule synthesis method, and specifically, speech that is divided in units such as phonemes and VCV (Vowel-Consonant-Vowel) syllables.

  The waveform database 44 replaces the unit waveform data (or the compressed waveform data obtained by entropy encoding the unit waveform data), and entropy encodes the unit speech waveform data (or unit speech waveform data) representing the unit speech waveform. (Data obtained) is stored, the acoustic processing unit 41 treats unit speech waveform data as speech waveform data instead of generating speech waveform data using the segment waveform data, and from the language processing unit 1 The unit speech waveform data may be supplied to the output synthesizer 54 in the order in which the phonograms are arranged in the supplied phonogram string.

  The waveform database 44 may store both data representing the waveform of the segment and data representing the waveform of the phoneme. In this case, the acoustic processing unit 41 causes the search unit 42 to search for phoneme data represented by the phonetic character included in the distribution character string and the like for the phonetic character for which the corresponding phoneme has not been found. Data representing the phoneme represented by the character may be retrieved by the search unit 42, and data representing the phoneme may be generated using the retrieved data representing the segment.

  The prosody prediction unit 53 predicts the utterance speed, and the coincidence sound piece determination unit 52 is a result of the prediction by the prosody prediction unit 53 of the speech piece data acquired by the search unit 6 under a predetermined discrimination condition. On the other hand, it is also possible to select those that do not match the prediction result and exclude those that match the prediction result. Note that the sound piece database 7 may store a plurality of pieces of sound piece data in which the reading of the sound pieces is common and the utterance speeds are different from each other.

  Further, the output synthesizing unit 54 may arbitrarily set the time length of the phoneme represented by the speech waveform data to match the utterance speed of the speech piece represented by the speech piece data. Therefore, the output synthesis unit 54 specifies, for example, the ratio by which the time length of the phoneme represented by each section included in the speech piece data from the coincidence speech piece determination unit 52 is increased or decreased with respect to the original time length, and then the speech waveform data May be resampled to increase or decrease the number of samples of the speech waveform data to a number corresponding to the length of time that matches the utterance speed indicated by the matching sound piece determination unit 52.

  Moreover, the utterance speed may be different for each sound piece. (Therefore, the utterance speed data may specify different utterance speeds for each sound piece.) Then, the output synthesizer 54 has each sound piece positioned between two sound pieces having different utterance speeds. For speech waveform data of speech, the speech speed of these voices between the two speech pieces is determined by interpolation (for example, linear interpolation) of the speech speeds of the two speech pieces. The voice waveform data representing these voices may be converted so as to match the utterance speed.

  Further, the output synthesizer 54 converts the speech waveform data even if the speech waveform data returned from the acoustic processing unit 41 represents speech that constitutes speech that reads free text or a distributed character string. Thus, the time lengths of these voices may be matched with the speed indicated by the utterance speed data supplied to the matching sound piece determination unit 52, for example.

  In the speech synthesis system described above, for example, the prosody prediction unit 53 may perform prosody prediction (including prediction of speech speed) on the entire sentence, or may perform prosody prediction for each predetermined unit. . Also, when prosodic prediction is performed on the entire sentence, if there is a sound piece that matches the reading, it is further determined whether or not the prosody matches within a predetermined condition, and if it matches, the sound piece is adopted. You may do it. For the part where no matching sound piece exists, the rule synthesis processing unit 4 generates speech based on the segment, provided that the pitch or speed of the part synthesized based on the segment is set to the whole sentence or It is good also as what adjusts based on the result of the prosodic prediction performed for every predetermined unit. As a result, even when a speech piece and a voice generated based on a segment are combined and synthesized, natural speech is performed.

  When the character string input to the language processing unit 1 is a phonetic character string, the language processing unit 1 performs a known natural language analysis process separately from the prosodic prediction, and the matching sound piece determination unit 52 The sound piece may be selected based on the result of the language analysis process. This makes it possible to select a sound piece using the result of interpreting a character string for each word (part of speech such as a noun or verb), compared to simply selecting a sound piece that matches the phonetic character string. Can speak naturally.

  The sound piece database creation unit 11 may include a microphone, an amplifier, a sampling circuit, an A / D (Analog-to-Digital) converter, a PCM encoder, and the like. In this case, instead of obtaining the sound piece data from the recorded sound piece data set storage unit 10, the sound piece database creating unit 11 amplifies a sound signal representing the sound collected by its own microphone, samples it, and performs A / After D conversion, the piece data may be created by performing PCM modulation on the sampled audio signal.

The sound piece database creation unit 11 becomes a material for new compressed sound piece data to be added to the sound piece database 7 from the recording medium set in the recording medium drive device (not shown) via the recording medium drive device. Sound piece data and phonetic character strings may be read.
The sound piece registration unit R does not necessarily need to include the recorded sound piece data set storage unit 10.

  Further, the pitch component data may be data representing a time change of the pitch length of the sound piece represented by the sound piece data. In this case, the matching sound piece determination unit 52 may identify a position having the shortest pitch length (that is, a position having the highest frequency) based on the pitch component data, and interpret this position as an accent position.

Further, when the ID is composed of a character string having a special reading and there is no possibility of confusion between the reading of a word ordinarily used in speech by a person and the reading of the ID, the search unit When the speech piece database 7 is searched using the phonetic character string 6 representing the reading of the ID as a search key, one piece of compressed speech piece data representing the voice corresponding to the ID is specified. For this reason, the sound piece database 7 does not have to store the pitch component data in association with the compressed sound piece data associated with the ID.
The prosody prediction unit 53 may omit prosody prediction for IDs associated with compressed speech piece data that is not associated with pitch component data. If such a configuration is adopted, useless prosody prediction processing is not performed, and processing performed by the main unit M becomes efficient as a whole.

(Second Embodiment)
Next explained is a speech synthesis system according to the second embodiment of the invention.
The configuration of the speech synthesis system according to the second embodiment of the present invention is substantially the same as the configuration of the speech synthesis system according to the first embodiment. However, the data amount before compression of the compressed sound piece data stored in the sound piece database 7 is not necessarily smaller than the above-described upper limit (specifically, the storage capacity of the cache memory) in the first embodiment. Shall not.

(Second Embodiment: Operation of Main Unit)
Next, the operation of the speech synthesis unit according to the second embodiment will be described.
First, the operation of the sound piece registration unit R of the speech synthesis system according to the second embodiment is substantially the same as that in the first embodiment.
The operation when the main unit M acquires the free text data or the distribution character string data is substantially the same as the operation of the main unit M in the first embodiment.

  Next, the operation when the sound piece editing unit 5 of the main unit M acquires the standard message data, the utterance speed data, and the collation level data will be described.

After the sound piece editing unit 5 obtains the standard message data, the utterance speed data, and the collation level data, the sound piece data representing the waveform that can be approximated to the waveform of the sound piece constituting the standard message is obtained as the matching sound piece determination unit 52. The operation of each part of the main body unit M until selection is substantially the same as the operation of each part of the main body unit M of the first embodiment.
The operation of the matching sound piece determination unit 52 when there is a sound piece that cannot select sound piece data that satisfies the condition corresponding to the value of the collation level data from the sound piece data supplied from the search unit 6 is also as follows. This is substantially the same as in the first embodiment.

  For each piece of speech piece data selected as satisfying the condition corresponding to the value of the collation level data, the coincidence piece determination unit 52 sets the amount of the piece of speech piece data to the above-described upper limit (specifically, cache memory). It is determined whether or not the storage capacity is exceeded.

  Then, the matching sound piece determination unit 52 supplies the sound piece data determined that the data amount does not exceed the upper limit to the output synthesis unit 54 as it is, and the sound piece data determined to exceed the upper limit exceeds the upper limit. After being divided into a plurality of non-speech piece data, it is supplied to the output synthesis unit 54.

When the sound piece data is divided, the matching sound piece determination unit 52 specifies a position corresponding to a substantially silent section of the sound pieces and divides the sound piece data at this position. .
Even when the sound piece data is divided, if the sound piece data is divided at a position corresponding to the silent section in this way, the sound reproduced by the output synthesis unit 54 may be interrupted at an unnatural portion. In addition, the main unit M can perform natural sound output.
In addition, since the voice uttered by a person includes a silent section of a certain length or more in most cases, the sound piece data to be divided usually has a position corresponding to the silent section.

  On the other hand, the coincidence sound piece determination unit 52 has received a piece of sound for which no sound piece data satisfying the condition corresponding to the value of the collation level data has been selected when missing portion identification data is also supplied from the search unit 6. In the case, the phonetic character string representing the reading of the sound piece indicated by the missing part identification data (including the sound piece that could not select the sound piece data that satisfies the condition corresponding to the value of the collation level data) It is extracted and supplied to the sound processing unit 41, and an instruction is given to synthesize the waveform of this sound piece. The operation performed by the sound processing unit 41 receiving the instruction is substantially the same as that in the first embodiment.

  Then, when the sound waveform data is supplied from the acoustic processing unit 41 and the sound piece data is supplied from the matching sound piece determination unit 52, the output synthesis unit 54 is substantially the same as that in the first embodiment. By performing the operation, synthesized voice data is generated, and the synthesized voice represented by the synthesized voice data is reproduced, or the synthesized voice data is output to the outside. If the matching sound piece determination unit 52 has divided sound piece data, the output composition unit 54 first supplies the first divided sound piece data of the sound piece data obtained by the division to the cache memory to obtain a predetermined value. Processing (for example, the above-described speech speed conversion processing) is performed, and the voice represented by the sound piece data is reproduced, or the sound piece data is output to the outside. Thereafter, the above-described processing is sequentially performed on the next sound piece data until the sound represented by the last sound piece data obtained by the division is reproduced or the sound piece data is output.

Since the output combining unit 54 of the second embodiment performs substantially the same operation as that in the first embodiment, that is, the output combining unit 54 of the second embodiment is also combined. When the audio data is generated, if the data amount of the speech piece data or the audio waveform data supplied to the audio data exceeds the above upper limit, the amount not exceeding the upper limit is stored in the cache memory. Then, after performing predetermined processing on the amount stored in the cache memory, the sound piece data or voice waveform data subjected to this processing is stored in the storage area of the main memory, and the remaining sound piece data or voice waveform data is stored. Of these, the predetermined processing is continued after storing the amount not exceeding the upper limit in the storage area of the cache memory. Then, the sound piece data or the sound waveform data that has undergone the predetermined processing is sequentially stored in the storage area of the main memory so as to be continuous with each other. Then, when all of the sound piece data and the sound waveform data have undergone the predetermined processing, the sound represented by the continuation of the sound piece data and the sound waveform data (that is, the synthesized sound data) stored in the main memory at this time is obtained. It shall be played back or output to the outside.
However, as described above, since the data amount of each piece of sound piece data that is the target of output or sound reproduction does not exceed the above-described upper limit, the output composition unit 54 does not go through temporary storage in the main memory and Audio reproduction or output is sequentially performed for each piece of data or audio waveform data. For this reason, there is no delay before the reproduction output is obtained. However, when the matching sound piece determination unit 52 does not divide sound piece data that does not have a position corresponding to a substantially silent section, the sound supplied to the output composition unit 54 There is a case where the data amount of one piece of data exceeds the upper limit described above.

  In the speech synthesis system according to the second embodiment of the present invention described above, the speech piece data representing the waveform of the speech piece that can be a unit larger than the phoneme is naturally converted by the recording editing method based on the prosodic prediction result. The voice that reads out the standard message is synthesized. However, a sound piece for which appropriate sound piece data could not be selected is synthesized in accordance with a rule synthesis method using compressed waveform data representing a piece that is a unit smaller than a phoneme.

  In addition, since this speech synthesis system divides sound piece data whose data amount exceeds the upper limit into a plurality of sound piece data representing sound pieces having a time length not exceeding the upper limit, the sound piece data is stored in the storage area of the main memory. Therefore, it is difficult for the output synthesizer 54 to temporarily suspend speech synthesis.

  In addition, this speech synthesis system also performs speech synthesis normally while adopting a configuration in which morphological analysis is performed on IDs without distinguishing them from other ideographic character strings. For this reason, a synthesized speech having a long time length can be obtained in a short time by a simple process.

  Although the embodiment of the present invention has been described above, the speech synthesizer according to the present invention can be realized using a normal computer system, not a dedicated system.

  For example, a program for causing a personal computer connected to an external nonvolatile memory constituting the sound piece database 7 to execute the operations of the recorded sound piece data set storage unit 10, the sound piece database creation unit 11, and the compression unit 12 described above. By installing the program from a recording medium (CD-ROM, flexible disk, etc.) that stores the sound piece, the sound piece registration unit R that executes the above-described processing can be configured.

A personal computer that executes this program and functions as the sound piece registration unit R performs the process shown in FIG. 3 as a process corresponding to the operation of the sound piece registration unit R of the speech synthesis system of FIG. You can also.
FIG. 3 is a flowchart showing processing executed by the personal computer that performs the function of the sound piece registration unit R.

  That is, when the personal computer registers a sound piece in the sound piece database 7, first, the recorded sound piece data set storage unit 10 reads the phonogram string and the sound piece data associated with each other, or The associated phonetic character string and speech piece data are acquired from the outside (FIG. 3, step S001), and the time change of the frequency of the pitch component of the voice represented by the obtained speech piece data and the utterance speed are obtained. Specify (step S002).

In this personal computer, the utterance speed in step S002 may be specified by, for example, counting the number of samples of the sound piece data.
Moreover, what is necessary is just to identify the time change of the frequency of a pitch component, for example by performing a cepstrum analysis to this sound piece data. Specifically, for example, the waveform represented by the sound piece data is divided into a large number of small parts on the time axis, the cepstrum of each obtained small part is obtained, and the minimum of the frequencies giving the maximum value of this cepstrum is obtained. The value may be specified as the frequency of the pitch component in this small portion. As described above, the time change of the frequency of the pitch component is based on the pitch waveform data after the sound piece data is converted into the pitch waveform data according to the method disclosed in, for example, Japanese Patent Laid-Open No. 2003-108172. If you specify it, you can expect good results.

  On the other hand, this personal computer creates compressed sound piece data by entropy encoding the obtained sound piece data (step S003), and writes it in the storage area of the sound piece database 7 as data constituting the data portion DAT (step S003). Step S004).

In step S004, the personal computer writes the phonetic character acquired in step S001 as the sound piece reading data in the storage area of the sound piece database 7 as indicating the reading of the sound piece represented by the written compressed sound piece data. .
Further, the head address of the written compressed sound piece data in the storage area of the sound piece database 7 is specified, and this address is written in the storage area of the sound piece database 7 as the data (B) described above.
Further, the data length of the compressed sound piece data is specified, and the specified data length is written in the storage area of the sound piece database 7 as data (C).
In addition, data indicating the result of specifying the time variation of the voice generation speed and pitch component frequency represented by the compressed sound piece data is generated and stored in the storage area of the sound piece database 7 as speed initial value data and pitch component data. Write.

  Further, the operations of the language processing unit 1, the general word dictionary 2, the user word dictionary 3, the rule synthesis processing unit 4, the sound piece editing unit 5, the search unit 6, the sound piece database 7, and the decompression unit 8 are executed on the personal computer. The main unit M according to the first embodiment of the present invention described above can be configured by installing the program from the recording medium storing the program for causing the program to be executed.

Then, a personal computer that executes this program and functions as the main unit M may perform the processes shown in FIGS. 4 to 6 as the process corresponding to the operation of the main unit M of the speech synthesis system of FIG. it can.
FIG. 4 is a flowchart showing processing when the personal computer that performs the function of the main unit M according to the first and second embodiments of the present invention acquires free text data.
FIG. 5 is a flowchart showing the processing when the personal computer that performs the function of the main unit M according to the first and second embodiments of the present invention acquires the distribution character string data.
FIG. 6 is a flowchart showing processing when the personal computer that performs the function of the main unit M according to the first embodiment of the present invention acquires the fixed message data and the utterance speed data.

  That is, when the personal computer obtains the above-mentioned free text data from the outside (step S101 in FIG. 4), the phonogram representing the reading of each ideographic character included in the free text represented by the free text data. Is identified by searching the general word dictionary 2 and the user word dictionary 3, and the ideogram is replaced with the identified phonogram (step S102). Note that the method of acquiring free text data by this personal computer is arbitrary.

  And when this personal computer obtains a phonetic character string representing the result of replacing all ideographic characters in the free text with phonetic characters, for each phonetic character contained in this phonetic character string, The waveform of the unit speech represented by the phonetic character is searched from the waveform database 44, and compressed waveform data representing the waveform of the segment constituting the phoneme represented by each phonetic character included in the phonetic character string is retrieved (step S103). ), The retrieved compressed waveform data is restored to the segment waveform data before being compressed (step S104).

  On the other hand, the personal computer predicts the prosody of the speech represented by the free text by adding analysis based on the prosody prediction method to the free text data (step S105). Then, speech waveform data is generated based on the segment waveform data restored in step S104 and the prosodic prediction result in step S105 (step S106), and the obtained speech waveform data is generated in the phonetic character string. The synthesized speech data is generated by combining the phonetic characters in the order in which they are arranged, and the synthesized speech represented by the synthesized speech data is subjected to the same processing as the processing performed by the output synthesis unit 54 described above. It reproduces or outputs this synthesized voice data to the outside (step S107). Note that the method by which the personal computer outputs the synthesized voice data is arbitrary.

  When this personal computer obtains the above-mentioned distribution character string data from the outside by an arbitrary method (FIG. 5, step S201), each phonogram included in the phonogram string represented by this distribution character string data In the same manner as in steps S103 to S104 described above, the process of searching for compressed waveform data representing the waveform of the segment constituting the phoneme represented by the phonetic character, and the retrieved compressed waveform data as the segment waveform A process of restoring data is performed (step S202).

  On the other hand, this personal computer predicts the prosody of the speech represented by the distribution character string by adding an analysis based on the prosody prediction method to the distribution character string (step S203), and the segment waveform data restored in step S202. And speech waveform data is generated based on the prosodic prediction result in step S203 (step S204), and the obtained speech waveform data is exchanged with each other in the order in which the phonograms are arranged in the phonogram string. By combining these, synthesized speech data is generated, and the synthesized speech represented by the synthesized speech data is reproduced or the synthesized speech data is output to the outside (step S205) in the same manner as in step S107 described above.

  On the other hand, when this personal computer obtains the above-mentioned fixed message data, collation level data, and utterance speed data from the outside by any method (FIG. 6, step S301), first, morphological analysis is performed on the fixed message data by a known method. Is applied to convert the ideographic character string constituting the standard message data into a phonetic character string (step S302).

  Next, the personal computer searches for all the compressed speech piece data associated with the phonetic character string that matches the phonetic character string obtained in the process of step S302 (step S303).

In step S303, the above-described sound piece reading data, speed initial value data, and pitch component data associated with the corresponding compressed sound piece data are also retrieved. In addition, when a plurality of compressed sound piece data corresponds to one sound piece, all the corresponding compressed sound piece data are searched. On the other hand, if there is a sound piece for which compressed sound piece data could not be found, the above-described missing portion identification data is generated.
The personal computer then restores the retrieved compressed sound piece data to the sound piece data before being compressed (step S304).

  Next, the personal computer predicts the prosody of the standard message by adding an analysis based on the prosody prediction method to the standard message represented by the standard message data (step S305). Then, the above-mentioned matching piece determination unit 52 performs the piece piece data representing the waveform closest to the waveform of the piece constituting the standard message from the piece pieces data in which the time length of the piece is converted, By performing the same process, one piece is selected for each sound piece according to the reference indicated by the collation level data acquired from the outside (step S306).

Specifically, in step S306, the personal computer specifies sound piece data in accordance with, for example, the above conditions (1) to (3). That is, when the value of the collation level data is “1”, all of the piece data whose reading matches the sound piece in the standard message is regarded as representing the waveform of the sound piece in the standard message. When the value of the collation level data is “2”, the phonetic character representing the reading matches, and the content of the pitch component data representing the time change of the frequency of the pitch component of the sound piece data is displayed in the standard message. Only when the predicted result of the accent of the included speech piece matches, this speech piece data is considered to represent the waveform of the speech piece in the standard message. When the value of the collation level data is “3”, the phonetic character and the accent representing the reading match, and whether or not the voice represented by the speech piece data is nasalized or unvoiced is determined by the prosody of the standard message. The sound piece data is regarded as representing the waveform of the sound piece in the standard message only when the result matches the predicted result.
If there are a plurality of pieces of sound piece data that match the criteria indicated by the collation level data for one piece of sound, the plurality of pieces of sound piece data are narrowed down to one according to conditions that are stricter than the set conditions. . In addition, when there is a sound piece that cannot select sound piece data that satisfies the condition corresponding to the value of the collation level data, it is determined that the corresponding sound piece is treated as a sound piece for which compressed sound piece data could not be found. For example, it is assumed that missing part identification data is generated.

On the other hand, when the personal computer generates the missing part identification data, the personal computer extracts a phonetic character string representing the reading of the sound piece indicated by the missing part identification data from the standard message data. By processing the phonetic character string represented by the delivery character string data in the same manner as the processing in steps S202 to S204 described above, the waveform of the voice indicated by each phonetic character in the phonetic character string is obtained. The voice waveform data to be represented is generated (step S307).
However, in step S307, the personal computer may generate speech waveform data using the result of prosody prediction in step S305 instead of performing the process corresponding to the process in step S203.

  Then, the personal computer mutually connects the speech waveform data generated in step S307 and the speech piece data selected in step S306 in the order in which the phonetic character strings in the standard message indicated by the standard message data are arranged. The synthesized voice data is generated by the combination, and the synthesized voice represented by the synthesized voice data is reproduced, or the synthesized voice data is output to the outside (step S308).

  In step S308, when the personal computer generates synthesized speech data composed of a plurality of speech piece data, the data amount of each piece of speech piece data or each piece of speech waveform data is the above-described upper limit (specifically, for example, If the cache memory included in the personal computer is not exceeded, the first piece of speech data or speech waveform data is first stored in the cache memory, subjected to predetermined processing, and the speech piece data or speech waveform data is updated. It is assumed that the voice represented is reproduced or output to the outside. Subsequently, the sound piece data or audio waveform data to be reproduced and output next is stored in the cache memory, and thereafter the audio is reproduced or output in the same manner as described above. This is repeated until the reproduction and output of all sound piece data or audio waveform data is completed.

The predetermined process in step S308 may be a process corresponding to, for example, the speech speed conversion process described above. That is, in step S308, the personal computer may perform the following processing as (B1) and (B2), for example.
(B1) The speed indicated by the utterance speed data indicates the time length of the sound piece represented by the sound piece data by converting the time interval of the samples constituting the sound piece data representing the voiced sound stored in the cache memory. To match. When the data stored in the cache memory is audio waveform data, for example, the sampling time interval is set at the same rate as the conversion rate of the sampling rate of the sound piece data that has been subjected to the processing of (B1) immediately before. Perform conversion.
(B2) The sound piece data in which the sampling rate has been converted as a result of the processing of (B1) described above is converted, and the pitch frequency of the sound piece represented by the sound piece data is calculated as the pitch component data retrieved in step S303. Of the frequencies shown, it is made to coincide with the pitch frequency of the sound piece represented by this sound piece data before the processing of (B1). When the data stored in the cache memory is voice waveform data, for example, conversion is performed so as to represent a voice having the same pitch frequency as the pitch frequency of the sound piece data subjected to the processing (B2) immediately before. Do.

Specifically, the personal computer may perform, for example, the following processes (B2a) to (B2d) as the process (B2) described above.
(B2a) The sound piece data that has undergone the processing of (B1) is mixed with the first local oscillation signal having a constant amplitude and frequency.
(B2b) By filtering the data obtained as a result of the process (B2a), the sum of the frequency of the sound piece data subjected to the process (1) and the frequency of the first local oscillation signal among the data A component having a frequency corresponding to that is extracted.
(B2c) The component obtained as a result of the processing of (B2b) is mixed with the second local oscillation signal whose amplitude and frequency are constant. However, the frequency of the second local oscillation signal has a frequency corresponding to the difference between the frequency of the component obtained by the processing of (B2b) and the frequency of the second local oscillation signal in the data obtained by the mixing. Based on the pitch component data retrieved in step S303 so that the sound piece represented by the component has a pitch frequency that matches the pitch frequency of the sound piece represented by the sound piece data before the processing of (B1). Shall be determined.
(B2d) By filtering the data obtained as a result of the process of (B2c), the difference between the frequency of the component obtained by the process of (B2b) and the frequency of the second local oscillation signal among the data A component having a frequency corresponding to that is extracted. This component extracted as a result of the process (B2d) corresponds to the sound piece data subjected to the process (B2) described above.

When performing the process of step S308, this personal computer does not exceed this upper limit if the data amount of the speech waveform data generated in step S307 or the sound piece data selected in step S306 exceeds the above upper limit. The minutes are stored in the cache memory. Then, after performing predetermined processing on the amount stored in the cache memory, the sound piece data or voice waveform data that has undergone this processing is stored in the storage area of another storage device (for example, main memory) provided in the personal computer. , The remaining sound piece data or voice waveform data that does not exceed the above-mentioned upper limit is stored in the storage area of the cache memory, and then the predetermined processing is continued. Then, the sound piece data or the sound waveform data that has undergone the predetermined processing is sequentially stored in a storage area such as a main memory so as to be continuous with each other. After the above processing, the voice represented by the continuation of the speech piece data and the voice waveform data (that is, synthesized voice data) stored in the main memory or the like at this time is reproduced or output to the outside.
However, as described above, since the data amount of each piece of sound piece data that is the target of output or sound reproduction does not exceed the above upper limit, this personal computer does not go through temporary storage in the main memory, Audio reproduction or output is sequentially performed for each piece of data or audio waveform data.

  In addition, the language processing unit 1, the general word dictionary 2, the user word dictionary 3, the rule synthesis processing unit 4, the sound piece editing unit 5, the search unit 6, the sound in the second embodiment of the present invention described above are connected to a personal computer. The main unit M according to the second embodiment of the present invention can be configured by installing the program from a recording medium storing the program for executing the operations of the fragment database 7 and the decompression unit 8.

The personal computer that executes this program performs substantially the same processing as the processing shown in FIGS. 4 and 5 as processing corresponding to the operation of the main unit M according to the second embodiment of the present invention. In addition, the processing shown in FIG. 7 can also be performed.
FIG. 7 is a flowchart showing a process when a personal computer that performs the function of the main unit M according to the second embodiment of the present invention acquires standard message data and utterance speed data.

  That is, when the personal computer that executes this program acquires the standard message data, the utterance speed data, and the collation level data, first, by performing substantially the same processing as the above-described steps S301 to S306, One piece of sound piece data representing the waveform closest to the waveform of the sound piece constituting the standard message represented by the message data is selected for each piece of sound (FIG. 7, steps S401 to S406).

  When the personal computer generates missing part identification data, the personal computer performs substantially the same process as the process in step S307 described above, thereby generating voice waveform data representing the waveform of the voice of the part indicated by the missing part identification data. Is generated (step S407).

On the other hand, for each piece of sound piece data selected as satisfying the condition corresponding to the value of the collation level data, the data amount of the piece of piece data has a predetermined upper limit (specifically, for example, this personal computer It is determined whether or not the storage capacity of the cache memory provided in the computer is exceeded (step S408). Then, the sound piece data determined that the data amount exceeds the upper limit is divided into a plurality of sound piece data that does not exceed the upper limit (step S409).
When dividing the piece data in step S409, the personal computer specifies a position corresponding to a substantially silent section of the piece, and divides the piece data at this position. .

  Then, the personal computer uses the speech waveform data generated in step S407 and the speech piece data selected in step S406 (including the speech piece data obtained by the division in step S409) in step S308. The synthesized speech data is generated by combining with each other by performing substantially the same processing as described above, and the synthesized speech represented by the synthesized speech data is reproduced, or the synthesized speech data is output to the outside (step S410). If the sound piece data is divided in step S409, in step S410, the first divided sound piece data among the sound piece data obtained by the division is first supplied to the cache memory, and predetermined processing is performed. The sound represented by the piece data is reproduced or the sound piece data is output to the outside. Thereafter, the above-described processing is sequentially performed on the next sound piece data until the sound represented by the last sound piece data obtained by the division is reproduced or the sound piece data is output.

Even when the personal computer performs the process of step S410, if the data amount of the speech waveform data generated by the process of step S407 or the sound piece data selected in step S406 exceeds the upper limit, the upper limit is set. It is assumed that the amount not exceeding 1 is stored in the cache memory. Then, after performing predetermined processing on the amount stored in the cache memory, the sound piece data or voice waveform data that has undergone this processing is stored in the storage area of another storage device (for example, main memory) provided in the personal computer. , The remaining sound piece data or voice waveform data that does not exceed the above-mentioned upper limit is stored in the storage area of the cache memory, and then the predetermined processing is continued. Then, the sound piece data or the sound waveform data that has undergone the predetermined processing is sequentially stored in a storage area such as a main memory so as to be continuous with each other. After the above processing, the voice represented by the continuation of the speech piece data and the voice waveform data (that is, synthesized voice data) stored in the main memory or the like at this time is reproduced or output to the outside.
However, as described above, since the data amount of each piece of sound piece data that is the target of output or sound reproduction does not exceed the above upper limit, this personal computer does not go through temporary storage in the main memory, Audio reproduction or output is sequentially performed for each piece of data or audio waveform data. However, in the case of not dividing the sound piece data that does not have a position corresponding to a substantially silent section, the data amount of the sound piece data used for the processing in step S410 exceeds the above upper limit. It may be exceeded.

The program that causes the personal computer to perform the functions of the main unit M and the sound piece registration unit R may be uploaded to a bulletin board (BBS) of a communication line and distributed via the communication line. The carrier wave may be modulated with a signal representing these programs, the obtained modulated wave may be transmitted, and a device that receives the modulated wave may demodulate the modulated wave to restore these programs.
The above-described processing can be executed by starting up these programs and executing them under the control of the OS in the same manner as other application programs.

  When the OS shares a part of the processing, or when the OS constitutes a part of one component of the present invention, a program excluding the part is stored in the recording medium. May be. Also in this case, in the present invention, it is assumed that the recording medium stores a program for executing each function or step executed by the computer.

It is a block diagram which shows the structure of the speech synthesis system which concerns on 1st and 2nd embodiment of this invention. It is a figure which shows typically the data structure of a sound piece database. It is a flowchart which shows the process which the personal computer which performs the function of the sound piece registration unit which concerns on the 1st and 2nd embodiment of this invention performs. It is a flowchart which shows a process when the personal computer which performs the function of the main body unit which concerns on 1st and 2nd embodiment of this invention acquires free text data. It is a flowchart which shows a process when the personal computer which performs the function of the main body unit which concerns on 1st and 2nd embodiment of this invention acquires delivery character string data. It is a flowchart which shows a process when the personal computer which performs the function of the main body unit which concerns on 1st Embodiment of this invention acquires fixed message data and utterance speed data. It is a flowchart which shows a process when the personal computer which performs the function of the main body unit which concerns on 2nd Embodiment of this invention acquires fixed form message data and utterance speed data.

Explanation of symbols

M Main unit 1 Language processing unit 2 General word dictionary 3 User word dictionary 41 Acoustic processing unit 42 Search units 43 and 8 Expansion unit 44 Waveform database 5 Sound piece editing unit 51 Morphological analysis unit 52 Matched sound piece determination unit 53 Prosody prediction unit 54 Output synthesis unit 6 Search unit 7 Sound piece database R Sound piece registration unit 10 Recorded sound piece data set storage unit 11 Sound piece database creation unit 12 Compression unit HDR Header unit IDX Index unit DIR Directory unit DAT Data unit

Claims (13)

The speech piece data representing the speech piece and having the data amount equal to or smaller than the predetermined amount is associated with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID for identifying the speech piece data. Sound piece storage means for storing;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Temporary storage means for temporarily storing the predetermined amount or less of the piece data selected by the selection means;
Processing means for performing predetermined processing on the sound piece data temporarily stored by the temporary storage means;
Replaying means for playing back the sound represented by the sound piece data processed by the processing means,
A speech synthesizer characterized by the above. Prosody prediction means for predicting the prosody of the ideogram or ID is further provided,
The sound piece storage means stores the prosody data representing the time change of the pitch of the sound piece represented by the sound piece data or the time change of the pitch of the voice that reads out the ID associated with the sound piece data. Stored in association with the data,
The selection means has a predetermined range of prosody predicted by the prosody prediction means from the speech piece data associated with the phonetic character string that matches the phonetic character string obtained as a result of the conversion by the morphological analysis unit. Is to select the piece data associated with the prosody data representing the prosody that matches.
The speech synthesizer according to claim 1. The selection means excludes the speech piece data associated with the prosody data representing the prosody that does not match the prosody predicted by the prosody prediction means within a predetermined range from the selection target.
The speech synthesizer according to claim 2. The speech piece storage means does not store the prosodic data associated with the speech piece data for the speech piece data associated with the phonetic character representing the ID reading,
The selecting means selects speech piece data not associated with prosodic data from speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit. To choose,
The speech synthesizer according to claim 2 or 3. The predetermined process includes a process of converting the sound piece data so that the time length of the sound piece represented by the sound piece data temporarily stored in the temporary storage unit matches a predetermined time length.
The speech synthesizer according to claim 2, 3, or 4. In the predetermined processing, the time length of the sound piece represented by the sound piece data temporarily stored by the temporary storage unit matches the predetermined time length, and the pitch frequency of the sound piece represented by the sound piece data is a predetermined condition. Including processing to convert the sound piece data to match
The speech synthesizer according to claim 2, 3, or 4. The predetermined process is:
A first process for converting the sound piece data so that the time length of the sound piece represented by the sound piece data temporarily stored by the temporary storage means matches a predetermined time length;
A second process for generating data representing the product of the instantaneous value of the waveform of the sound piece represented by the sound piece data that has undergone the first process and the instantaneous value of the first local oscillation signal;
A third component that extracts a component having a frequency corresponding to the sum of the frequency of the sound piece data temporarily stored by the temporary storage unit and the frequency of the first local oscillation signal from the data generated by the second process Processing,
A fourth process for generating data representing the product of the instantaneous value of the waveform of the sound piece represented by the component obtained by the third process and the instantaneous value of the second local oscillation signal;
Fifth processing for extracting a component having a frequency corresponding to the difference between the frequency of the component obtained by the third processing and the frequency of the second local oscillation signal from the data generated by the fourth processing And consists of
The fourth process includes a component having a frequency corresponding to a difference between the frequency of the component obtained by the third process and the frequency of the second local oscillation signal in the data obtained by the fourth process. Based on the prosodic data, the frequency of the second local oscillation signal is set so that the pitch frequency of the sound piece to be represented matches the pitch frequency of the sound piece represented by the sound piece data before the first processing. Including processing to determine,
The speech synthesizer according to claim 2, 3, or 4. Sound piece storage means for storing sound piece data representing a sound piece in association with a phonetic character representing a reading of the sound piece data or a phonetic character representing an ID reading identifying the sound piece data;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Dividing means for dividing, among the sound piece data selected by the selecting means, sound piece data whose data amount exceeds a predetermined amount into a plurality of sound piece data whose data amount is equal to or less than the predetermined amount;
Temporary storage temporarily stored for the piece of piece data selected by the selection unit and having the data amount equal to or less than the predetermined amount or the piece of piece data obtained as a result of division by the dividing unit. Storage means;
Processing means for performing predetermined processing on the sound piece data temporarily stored by the temporary storage means;
Replaying means for playing back the sound represented by the sound piece data processed by the processing means,
A speech synthesizer characterized by the above. The dividing means divides sound piece data whose data amount exceeds a predetermined amount at a position corresponding to a substantially silent section of sound pieces represented by the sound piece data.
The speech synthesizer according to claim 8. The speech piece data representing the speech piece and having the data amount equal to or smaller than the predetermined amount is associated with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID for identifying the speech piece data. Remember,
The ideogram or ID supplied from the outside is converted into a phonogram string representing the reading of the ideogram or ID,
Select the speech piece data associated with the phonetic character string that matches the phonetic character string obtained as a result of the conversion,
Of the selected piece data, the portion below the predetermined amount is temporarily stored,
Apply predetermined processing to the temporarily stored sound piece data,
Play the sound represented by the processed piece data,
A speech synthesis method characterized by the above. Storing the speech piece data representing the speech piece in association with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID identifying the speech piece data;
The ideogram or ID supplied from the outside is converted into a phonogram string representing the reading of the ideogram or ID,
Select the speech piece data associated with the phonetic character string that matches the phonetic character string obtained as a result of the conversion,
Of the selected sound piece data, the sound piece data whose data amount exceeds a predetermined amount is divided into a plurality of sound piece data whose data amount is equal to or less than the predetermined amount,
Of the selected piece of piece data whose data amount is less than or equal to the predetermined amount, or of the piece of piece data obtained as a result of the division, temporarily store the amount less than or equal to the predetermined amount,
Apply predetermined processing to the temporarily stored sound piece data,
Play the sound represented by the processed piece data,
A speech synthesis method characterized by the above. Computer
The speech piece data representing the speech piece and having the data amount equal to or smaller than the predetermined amount is associated with the phonogram representing the reading of the speech piece data or the phonogram representing the reading of the ID for identifying the speech piece data. Sound piece storage means for storing;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Temporary storage means for temporarily storing less than the predetermined amount of the selected sound piece data;
Processing means for performing predetermined processing on the sound piece data stored in the temporary storage means;
Playback means for playing back the sound represented by the sound piece data processed by the processing means;
Program to make it function. Computer
Sound piece storage means for storing sound piece data representing a sound piece in association with a phonetic character representing a reading of the sound piece data or a phonetic character representing an ID reading identifying the sound piece data;
A morpheme analyzer that converts an ideogram or ID supplied from the outside into a phonogram string representing the reading of the ideogram or ID;
Selection means for selecting speech piece data associated with a phonetic character string that matches a phonetic character string obtained as a result of conversion by the morphological analysis unit;
Dividing means for dividing, among the sound piece data selected by the selecting means, sound piece data whose data amount exceeds a predetermined amount into a plurality of sound piece data whose data amount is equal to or less than the predetermined amount;
Temporary storage temporarily stored for the piece of piece data selected by the selection unit and having the data amount equal to or less than the predetermined amount or the piece of piece data obtained as a result of division by the dividing unit. Storage means;
Processing means for performing predetermined processing on the sound piece data stored in the temporary storage means;
Playback means for playing back the sound represented by the sound piece data processed by the processing means;
Program to make it function.

Images