JP2003308083A - Voice synthesizing processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To conduct a voice synthesizing process at a terminal which has small scale hardware resources while maintaining reasonable quality for synthesized voice. <P>SOLUTION: A voice synthesizing processor is capable of communicating with a server 10 in which voice piece data are accumulated and character strings are processed for voice synthesis. The processor determines voice pieces required to output character strings with synthesized voice and transmits a transmission request to the server for the data of the determined voice pieces. The server transmits the voice piece data in response to the transmission request and the processor receives the voice piece data and generates synthesized voice data of the character strings based on the received voice piece data. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to speech synthesis technology.

[0002]

2. Description of the Related Art In recent years, in the field of voice synthesis, a waveform editing / connection type voice synthesis technique has become mainstream, for example, as described in Japanese Patent Laid-Open No. 10-49193. This speech synthesis technology uses a synthesis unit dictionary in which waveform data of a voice unit is stored, and connects waveform data of voice units stored in the synthesis unit dictionary to create waveform data of a synthesized voice. Is.

Further, since this synthesis segment dictionary often requires a relatively large storage capacity, for example, as described in Japanese Patent Application Laid-Open No. 11-95796, speech synthesis processing can be performed with less resources. In order to realize it, a technique for making the synthesis element dictionary compact is also proposed.

A technique for outputting a synthetic voice in a portable terminal such as a portable telephone has also been proposed. Since the mobile terminal has small hardware resources, in order to reduce the load on the mobile terminal and output the synthesized speech, the server on the network performs speech synthesis processing. A system has also been proposed in which data is transmitted to a mobile terminal and synthetic voice is output from the mobile terminal.

[0005]

However, in the case where a terminal having a small hardware resource such as a portable terminal outputs synthetic speech, all the speech synthesis processing should be performed by the server on the network as described above. In that case, it is possible that a plurality of mobile terminals request voice synthesis processing to the server, which increases the computational load of the server, and the number of installed servers must be increased. Becomes

In a terminal such as a portable terminal,
Hardware resources are not always sufficient to perform all speech synthesis processing, and in particular, there are many cases where sufficient memory cannot be secured for accumulating a synthesis segment dictionary. in this case,
It is conceivable to make the synthesis unit dictionary compact as described in Japanese Patent Laid-Open No. 11-95796, but the quality deterioration of the synthesized speech accompanying this is inevitable.

Therefore, an object of the present invention is to provide a technique capable of performing speech synthesis processing while maintaining the quality of synthesized speech with a terminal having small hardware resources.

[0008]

According to the present invention, there is provided a voice synthesizing processing device which is communicably connected to a server storing voice segment data, and which performs voice synthesizing processing of a character string, the character string Deciding means for deciding a speech segment necessary for outputting as a synthetic speech, transmitting means for transmitting a transmission request for the data of the speech segment decided by the deciding means to the server, and corresponding to the transmission request. A voice that includes a receiving unit that receives the data of the voice unit transmitted from the server, and a creating unit that creates the synthetic voice data of the character string based on the received data of the voice unit. A synthesis processing device is provided.

Further, according to the present invention, there is provided a program for causing a computer communicatively connected to a server storing voice segment data to perform a voice synthesis process of a character string, the character string being synthesized voice. A determining step of determining a speech segment necessary for outputting, a transmitting step of transmitting a transmission request of the data of the speech segment determined by the determining step to the server, and corresponding to the transmission request, A receiving step of receiving the data of the voice unit transmitted from the server,
There is provided a program for causing the computer to execute a creating step of creating synthetic voice data of the character string based on the received data of the voice unit.

According to the present invention, a server communicatively connected to a voice synthesis processing device for performing voice synthesis processing of a character string, and storage means for storing voice segment data,
Receiving means for receiving a request for transmitting the data of the voice unit from the voice synthesis processing device, and transmission for transmitting the data of the voice unit to the voice synthesis processing device in response to the received transmission request. A server is provided.

Further, according to the present invention, there is provided a data providing method using a server, which is communicably connected to a voice synthesis processing device for performing voice synthesis processing of a character string, and stores voice segment data. A receiving step of receiving a transmission request for the data of the voice unit from the voice synthesis processing device, and the server performing the voice synthesis of the data of the voice unit in response to the transmission request received by the server. A data providing method including a transmitting step of transmitting to a processing device.

Further, according to the present invention, a server accumulating voice segment data and a server communicably connected to the server,
A voice synthesis processing system comprising: a voice synthesis processing device for performing voice synthesis processing of a character string, wherein the voice synthesis processing device determines a voice unit necessary for outputting the character string in a synthesized voice. Deciding means, transmitting means for transmitting to the server a transmission request for the data of the speech unit determined by the deciding means, and receiving data of the speech segment transmitted from the server in response to the transmission request There is provided a speech synthesis processing system comprising: a receiving unit that performs the above; and a creating unit that creates the synthesized voice data of the character string based on the received data of the voice unit.

[0013]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment><SystemConfiguration> FIG. 1 is a system diagram of a speech synthesis processing system 1 according to an embodiment of the present invention. The speech synthesis processing system 1 includes a mobile phone 10 as a speech synthesis processing device that performs speech synthesis processing on a character string, a server 20 that stores a speech unit dictionary in which voice unit data is recorded,
, And these are configured to be communicable via the base station 2 and the network 3 represented by the Internet. First, the configuration of the server 20 will be described with reference to FIG.

The CPU 21 controls the entire server 20, and particularly executes the processing described later in this embodiment. R
The AM 22 is a memory used as a work area for the CPU 21. The ROM 23 is a memory that stores fixed data such as a control program executed by the CPU 21 and data used for processing the program.
In the case of the present embodiment, a program indicating a communication procedure with the mobile phone 10, a search program for the voice unit dictionary 24, and the like are stored in the ROM 23.

The speech unit dictionary 24 is dictionary data in which waveform data of each speech unit is recorded, and is stored in a storage device such as a hard disk or MO disk inside or outside the server 20. FIG. 7 is a diagram showing an example of data recorded in the speech unit dictionary 24.

In the example of FIG. 7, the unit of the speech unit is dipho.
An example is shown in which digital waveform data quantized in three stages of high, medium, and low as a fundamental frequency is recorded in units of ne. In the present embodiment, description will be given on the premise that the unit of the speech unit is the diphone unit, but the unit is not limited to this. There are various possible ways of constructing the speech unit dictionary 24, and the unit of the speech unit may be a phoneme unit, a triphone unit, or the like. Also, different types of units may be mixed. Further, it is not always necessary to classify by the fundamental frequency, and only one fundamental frequency data may be stored for each speech unit. Further, in addition to the fundamental frequency, it may be classified by prosody information (time length, power) or the like.

Next, the network interface 25
Is an interface for performing communication processing with other terminals via the network 3, whereby the server 20 can receive and transmit information to and from the mobile phone 10.

Next, the configuration of the mobile phone 10 will be described. FIG. 2 is a block diagram showing the hardware configuration of the mobile phone 10. In the present embodiment, the mobile phone 10 will be described as an example of the voice synthesis processing device, but a mobile terminal such as a PDA or a personal computer may be used. In addition, in the present embodiment, a case where wireless communication is performed by the mobile phone 10 will be described.
It goes without saying that wired communication may be performed.

The CPU 11 controls the entire mobile phone 10, and particularly executes a voice synthesis process described later in this embodiment. The RAM 12 is a memory used as a work area for the CPU 1. The ROM 13 has a CPU
11 is a memory that stores fixed data such as a control program executed by 11 and data used for processing the program. A program for voice synthesis processing and data of various dictionaries necessary for the processing can be stored in the ROM 13. The flash memory 14 is a memory that stores e-mails and other information received by the mobile phone 10. A part of the programs and data stored in the ROM 13 may be stored in the flash memory 14.

The interface 15a functions as an interface between the CPU 1 and the operation button 15. The operation button 15 is used by the user to give various instructions to the mobile phone 10, and is composed of a key switch or the like. The display 16 is composed of, for example, a liquid crystal display device or the like, and displays information to the user, and C is displayed via the display driver 16a.
The display is controlled by the PU 11.

The communication device 17 is a device for performing communication processing with other terminals via the base station 2 and the network 3, and for example, an RF circuit for performing wireless communication with the base station 2 or the like. It has an electronic circuit. This allows
The mobile phone 10 receives information from the server 20,
It becomes possible to send.

The A / D converter 18a is a circuit for converting an analog signal into a digital signal. In the present embodiment, since the analog signal of voice input from the microphone 18 is converted into a digital signal and output to the CPU 11. Used for.

The D / A converter 19a is a circuit for converting a digital signal into an analog signal, and in the present embodiment, it is C
It is used, for example, to convert digital format synthetic voice data output from the PU 11 into an analog signal. The speaker 19 outputs the analog signal output from the D / A converter 19a as sound, and is, for example, a headphone or the like. The D / A converter 19a and the speaker 1
It goes without saying that an amplifier circuit or the like may be provided between the switch 9 and the switch 9.

<System Processing> Next, the processing in the speech synthesis processing system 1 having the above-mentioned configuration will be described with reference to FIG. FIG. 3 is a flowchart showing the processing of the mobile phone 10 and the server 20.

In step S301, the portable telephone 10
At, it is determined whether there is a sentence to be read. Examples of the sentence to be read include a sentence of an email received by the mobile phone 10, a sentence of a home page provided on the network 3 acquired by the mobile phone 10, and the like. If there is a sentence to be read, the process proceeds to S302, and if not, the process ends.

In step S302, the mobile phone 10 performs a voice segment determination process. Here, the speech unit necessary to output the character string included in the sentence to be read as synthetic speech is determined. FIG. 4 is a flowchart showing the speech unit determination process.

At S401, one sentence is cut out from the sentence to be read. Here, “Good morning.” Shown in FIG.
An example is given in which a sentence consisting of the character string is cut out.
In S402, a language analysis process is performed on the extracted one sentence. Here, language analysis is performed on the cut-out one sentence to identify readings, accents, etc. necessary for speech synthesis,
save. FIG. 8 (b) is a reading (diphone series) created for “Good morning.”, “-.O”,
"Oh", "ha", "ay", "yo", "oo", "o."
-"Has been created.

In step S403, prosodic information such as fundamental frequency, phoneme time length, and power required for speech synthesis is estimated using the language analysis result in step S402, and created and stored. Figure 8
(C) shows an example of prosody information, and each diphon
The time length (broken line) and the fundamental frequency (solid line) for e are shown.

In step S404, the types of speech units necessary to create the waveform data of the synthesized speech of one sentence cut out based on the language analysis result in step S402 and the prosody information in step S403 are listed, and the list is created. ·save. Figure 8
(C) shows a list of speech units created for "Good morning." The diphone and the fundamental frequency (3
Quantization) is shown in the stage.

Next, returning to FIG. 3, in S303, the portable telephone 10 transmits the speech unit list created in S404 to the server 20 and transfers the corresponding speech unit data. Request.

In S311, the server 20 receives the transfer request from the portable telephone 10, and in response to this, S31
In step 2, the voice unit dictionary 24 is searched to extract the requested voice unit data. In S313, the voice unit data extracted by the server 20 is transmitted to the mobile phone 10.

In S304, the portable telephone 10 receives the data of the voice unit transmitted from the server 20, and S30
At 5, the synthesized speech data of the cut-out sentence is created based on the received speech unit data. In this step S305, for example, based on the prosody information created in step S403, the received voice segment data is edited and connected (waveform connection) to create synthetic voice data. After that, the voice based on the synthesized voice data is output from the speaker 19 in real time or when the user requests. S
After the process of 305, the process returns to S301 and the same process is performed for the remaining sentences of the sentence.

As described above, in the present embodiment, while the voice synthesis processing is executed on the portable telephone 10 side, the speech unit dictionary 24 used at that time is accumulated on the server 10 side and appropriately used, Even if the mobile phone 10 has a small hardware resource, especially a memory, the voice synthesis process can be performed, and the calculation load of the server 10 can be reduced. Furthermore, since the voice unit dictionary can be centrally managed and the voice unit dictionary 24 does not necessarily have to be made compact, the deterioration of sound quality can be avoided.

<Second Embodiment> In the first embodiment, if there are overlapping types of speech units in the speech unit list created in S404, the duplication is deleted and the speech unit list is deleted. It is also possible to create a list and transmit a voice segment data transfer request to the server 10 based on the created list. In this case, the location of the deleted voice segment may be checked and stored, and the data of the same voice segment may be embedded in the location of the deleted voice segment when creating the synthesized voice data in S306.

By creating the speech unit list in this way, the amount of communication information is reduced and higher speed communication is realized. FIG. 9 is an example showing a situation in which duplication is deleted and a speech unit list is created. Synthetic element (di
phone: / a. (i /, fundamental frequency: middle) appears and overlaps in the second and fifth positions, so one is deleted from the list to create a more compact list.

<Third Embodiment> In the first embodiment,
The speech synthesis processing is performed in units of one sentence, and the speech unit list is created in units of one sentence. However, the list is not limited to this.
The unit may be shorter than a sentence, for example, a unit of a fixed number of units, a unit of a pause phrase, a unit of a phrase, or a unit of an accent phrase. Alternatively, the unit may be a unit longer than one sentence, a unit of two sentences, a unit of paragraphs, or an entire sentence.

In the second embodiment, the case of deleting the overlapping speech units from the list has been described. However, the larger the number of speech units is, the larger the number of overlapping speech units is. Conceivable. Therefore, the second
In the case of the embodiment, when the speech unit list is created in a unit longer than one sentence and the speech unit data is requested to the server 10, the amount of communication information can be more effectively reduced.

On the other hand, when the capacity of the communication line is small,
By requesting the voice unit data from the server 10 in small units in units shorter than one sentence, it is possible to realize voice synthesis with an emphasis on real-time property.

<Fourth Embodiment> In the first embodiment, the voice unit data received by the mobile phone 10 from the server 10 is stored in the mobile phone 10 for reuse, and is stored in the mobile phone 10. It is also possible to hold a simple speech unit dictionary (hereinafter referred to as a sub-speech unit dictionary) in 10. In this case, the data of the sub-speech unit dictionary is, for example, the flash memory 1 of the mobile phone 10.
4 and the configuration thereof can be the same as that of the speech unit dictionary of the server 10 described with reference to FIG. 7 except that the storage amount is different.

FIG. 5 is a flowchart showing the processing of S404 in this case, and FIG. 6 is a flowchart showing the processing of S306 in this case.

Referring to FIG. 5, in S501, a list of speech units is listed and the list is stored. In S502, the sub-speech unit dictionary stored in the mobile phone 10 is referred to. In S503, in the sub-speech unit dictionary,
It is determined whether or not the speech units listed in S501 are registered. If no one is registered, the process ends, and if any one is registered, the process proceeds to S504.

In S504, it is determined whether or not all of the listed speech units are registered in the sub-speech unit dictionary. If all of them are registered, there is no need to request the voice unit data from the server 10, so the process proceeds to S305, and the synthesis voice data is created without accessing the server 10. If all are not registered, the speech units registered in the sub-speech unit dictionary are deleted from the list, only the necessary speech units are listed, and the process ends.

Referring to FIG. 6, in step S601, the server 1
It is determined whether or not the voice unit data from 0 is received.
When it is received, it means that the voice unit data which is not registered in the sub voice unit dictionary is received. Therefore, the process proceeds to S602, is registered in the sub voice unit dictionary, and the process proceeds to S603. On the other hand, if the voice unit data from the server 10 is not received, that is, if the answer in S504 is Yes, the process proceeds to S603.

In S603, S5 is selected from the sub-speech unit dictionary.
The voice unit data corresponding to the list of voice units stored in 01 is read out and connected to create synthesized voice data, and the process ends.

By doing so, the portable telephone 10
The number of times of communication between the server and the server 20 and the amount of communication information can be reduced. In the case of the present embodiment, the capacity of the sub-speech element dictionary may increase with the frequency of speech synthesis. However, the sub-speech element dictionary may be appropriately selected according to the storage capacity of the mobile phone 10. You can also delete the data. In this case, for example, when the capacity of the sub-speech unit dictionary exceeds a certain capacity, it is possible to erase the speech unit data that has been used less frequently in the past. In this case, it is possible to calculate the threshold value of the capacity, the current capacity, and the frequency of use of each voice segment data, and erase them according to the result.

<Fifth Embodiment> In the above embodiment, when deciding a speech unit, the prosody information (fundamental frequency) is used to distinguish the speech unit. However, the present invention is not limited to this, and the prosodic information is used. Alternatively, the speech unit (only the diphone) is determined, and along with this, the prosody information is transmitted from the mobile phone 10 to the server 20, and the server 20 side selects the speech unit distinguished by the prosody information. Good.

<Sixth Embodiment> In the above-described embodiment, each process is realized by software, but the present invention is not limited to this, and it may be realized by a unique circuit that performs the same operation.

Another object of the present invention is to supply a software program for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads and executes the program. Needless to say, it can be achieved by things.

In this case, the program itself realizes the functions of the above-described embodiment, and the program, the storage medium storing the program, or the program product constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instructions of the program code. It goes without saying that a case where some or all of the actual processing is performed and the functions of the above-described embodiments are realized by the processing is also included.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, based on the instruction of the program code, Needless to say, this also includes a case where a CPU or the like included in the function expansion card or the function expansion unit performs a part or all of the actual processing and the processing realizes the functions of the above-described embodiments.

[0052]

As described above, according to the present invention,
A terminal with small hardware resources can perform speech synthesis processing while maintaining the quality of synthesized speech.

[Brief description of drawings]

FIG. 1 is a system diagram of a speech synthesis processing system 1 according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a hardware configuration of the mobile phone 10.

FIG. 3 is a flowchart showing processing of the mobile phone 10 and the server 20.

FIG. 4 is a flowchart showing a speech unit determination process.

FIG. 5 is a flowchart showing the processing of S404 in the fourth embodiment.

FIG. 6 is a flowchart showing a process of S306 in the fourth embodiment.

FIG. 7 is a diagram showing an example of data recorded in a speech unit dictionary 24.

FIG. 8 is a diagram showing a process in which a sentence is subjected to voice synthesis processing.

FIG. 9 is an example showing a state of creating a speech unit list by deleting duplication.

Claims (10)

[Claims] 1. A speech synthesis processing device capable of communicating with a server storing speech segment data and performing speech synthesis processing of a character string, the speech element being necessary for outputting the character string as synthesized speech. Determining means for determining a piece, transmitting means for transmitting to the server a transmission request for the data of the speech element determined by the determining means, and a speech element transmitted from the server in response to the transmission request A voice synthesis processing apparatus comprising: a receiving unit that receives the data of 1. and a creating unit that creates the synthetic voice data of the character string based on the received data of the voice unit. 2. The speech synthesis processing apparatus according to claim 1, wherein the determining unit creates a list of the determined speech units, and the transmitting unit transmits the list to the server. . 3. The determining unit creates the list in which the duplicated speech units are deleted when the determined speech units include the duplicated speech units. The speech synthesis processing apparatus according to claim 2. 4. A storage unit for storing the data of the speech unit received by the receiving unit, wherein the determining unit stores the data of the speech unit stored in the storage unit in the determined speech unit. When there is voice segment data, the list is created by deleting the voice segment, and the creating unit creates the character string based on the data of the voice unit stored in the storage unit. 3. The speech synthesis processing apparatus according to claim 2, wherein the synthesized speech data of 1 is created. 5. The deciding means, a linguistic analyzing means for performing linguistic analysis of the character string, a prosody information creating means for creating prosody information of the character string,
The speech element necessary for outputting the character string as synthesized speech is determined based on the analysis result by the language analysis unit and the prosody information. Speech synthesis processor. 6. The voice synthesis processing apparatus according to claim 1, further comprising an output unit that outputs a synthetic voice based on the synthetic voice data created by the creating unit. 7. A program for causing a computer capable of communicating with a server, which stores speech segment data, to execute a speech synthesis process of a character string, the speech segment being necessary for outputting a character string as synthetic speech. And a transmission step of transmitting to the server a transmission request for the data of the speech segment determined by the determination step, in response to the transmission request, of the speech segment transmitted from the server A program for causing the computer to execute a receiving step of receiving data, and a creating step of creating synthetic voice data of the character string based on the received data of the voice unit. 8. A server communicable with a voice synthesis processing device for performing voice synthesis processing of a character string, comprising: storage means for accumulating voice segment data; A server comprising: a reception unit that receives a data transmission request; and a transmission unit that transmits the data of the speech unit to the speech synthesis processing device in response to the received transmission request. 9. A data providing method using a server, which is communicatively connected to a voice synthesis processing device for voice-synthesizing a character string, and stores voice segment data, the server performing the voice synthesis process. A receiving step of receiving a transmission request for the data of the speech unit from the device, and the server corresponding to the received transmission request,
And a step of transmitting the data of the voice unit to the voice synthesis processing device. 10. A speech synthesis processing system comprising: a server accumulating speech segment data; and a speech synthesis processing device capable of communicating with the server and performing speech synthesis processing of a character string, the speech synthesis processing system comprising: A synthesizing device determines a speech unit required to output the character string as a synthesized voice, and a transmission request for transmitting the data of the speech unit determined by the deciding unit to the server. Means, receiving means for receiving the data of the voice unit transmitted from the server in response to the transmission request, and creating synthetic voice data of the character string based on the received data of the voice unit. A speech synthesis processing system including: