JPH10222343A - Information communication system, information processor, information communicating method and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To keep high quality in voice synthesis by obtaining optimum correspondence relation without fixing the correspondence relation of speech information with tone information. SOLUTION: In a host device 1, DB11 permits one of largeness of voice and the height of it or both of them to correspond to the time difference, one of voice largeness and voice height or both of them are permitted not to rely on time difference between phonemes and file information including speech information constituted by a discreteness so as to have a correlative level is stored. In a terminal equipment 2, a tone storage part 21 stores tone data expressing acoustic parameters at every speech segment unit of the phonemes by tone kind. The host device 1 transfers file information to the terminal equipment 2 in accordance with the terminal equipment 2 and the terminal equipment 2 reads tone data designated by transferred speech information inside file information from a tone storage part 21 and executes voice synthesis based on the tone data and speech information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information communication system and method for reproducing media information such as voice by performing information communication between apparatuses via a communication network such as the Internet, and a communication network such as the Internet. TECHNICAL FIELD The present invention relates to an information processing apparatus and method for creating and editing information for reproducing media information such as audio by performing information communication between apparatuses via the apparatus.

[0002]

2. Description of the Related Art In the Internet, which has been remarkably developed in recent years, in order to transmit audio from a server to a client, audio is converted into waveform data (.wav or .au).
The technique of compressing and transferring in the form of is applied.

[0003] In the Internet, download of a homepage with a large amount of transfer tends to be avoided, and it is necessary to transfer waveform data having a large data size with a small amount of transfer in order to spread voice communication. Becomes

As a technique for solving the problem of the transfer amount in voice communication, there is a technique disclosed in Japanese Patent Publication No. 5-52520, for example. According to this publication, a technique is disclosed in which voice is divided into sound source information and vocal tract information corresponding to the sound source information, and the sound source information and the vocal tract information corresponding to each other are synthesized during voice synthesis. ing.

[0005]

However, since the Internet is a communication network used by an unspecified number of users, a client usually accesses arbitrarily sound source information, that is, utterance information from a server. This is a usage form that captures At this time, it is unclear whether the vocal tract information, that is, the timbre information prepared in the client matches the utterance information arbitrarily accessed.

Therefore, if the speaker on which the timbre information is based and the speaker on which the utterance information is based are the same person, and the voice conditions when creating the timbre information and the utterance information are the same, There is no problem with the reproducibility of the voice by voice synthesis, but if the person or condition is different, the amplitude is specified as the absolute amplitude level, and the pitch of the voice is specified as the absolute pitch frequency. However, there is a fear that the original amplitude pattern of the voice color information is not reflected, and the voice color information is reproduced with an inappropriate voice at the time of voice synthesis.

SUMMARY OF THE INVENTION An object of the present invention is to provide an information communication system capable of maintaining a high quality of speech synthesis by obtaining an optimum correspondence without fixing a correspondence between speech information and voice information. And

It is another object of the present invention to provide an information processing apparatus capable of easily creating and editing information for maintaining high quality of speech synthesis by the information communication system.

Another object of the present invention is to provide an information communication method capable of maintaining a high quality of speech synthesis by obtaining an optimum correspondence without fixing a correspondence between speech information and voice color information. For other purposes.

It is another object of the present invention to provide an information processing method capable of easily creating and editing information for maintaining high quality of speech synthesis by the information communication method.

[0011]

Means for Solving the Problems The above-mentioned problems are solved,
To achieve the object, an information communication system according to the first aspect of the present invention connects a first communication device and a second communication device to a communication network, and the first communication device and the second communication device via the communication network. In the information communication system for performing information communication between the first and second communication devices, the first communication device associates one or both of the loudness and the loudness of the voice with a time difference between the loudness and the loudness thereof, and File information storage means for storing file information including utterance information obtained by discriminating one or both of them so as to have a relative level without depending on a time difference between phonemes, and a second communication device The utterance information stored in the file information storage means in response to the
First communication means for transferring to a communication device, wherein the second communication device stores voice data representing voice parameters for each unit of unit such as phoneme for each voice type, and first communication. A second communication unit for requesting the device to transfer the file information stored in the file information storage unit, and thereafter receiving the file information transferred by the first communication unit; Selecting means for selecting one voice data from a plurality of types of voice data stored in the voice storage means based on the utterance information of the file information, and a voice volume and a voice height included in the utterance information. Expansion means for expanding a continuous prosody pattern in the time axis direction based on one or both of them and the time difference, and a prosody pattern expanded by the expansion means and a selection means. Characterized in that has a sound reproducing means for generating a speech waveform, the based on the selected the tone of voice data.

According to the first aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device makes the voice volume and voice pitch independent of phonemes. By developing a temporally continuous prosody pattern and generating a voice waveform based on the prosody pattern and voice data selected based on the utterance information, it is suitable without being limited to a specific voice. It is possible to reproduce voices with different timbres, and there is no shift in voice pitch pattern during waveform synthesis. In this way, the optimum correspondence between utterance information and timbre information can be obtained without fixing the correspondence. This makes it possible to maintain high quality of speech synthesis.

An information communication system according to a second aspect of the present invention connects the first communication device and the second communication device to a communication network,
In the information communication system for performing information communication between the first communication device and the second communication device via the communication network, the first communication device includes one or both of a loudness and a high pitch of the voice, Match the time difference with the type of voice,
A file for storing file information including utterance information obtained by discriminating one or both of a voice volume and a voice pitch so as to have a relative level without depending on a time difference between phonemes. An information storage unit; and a first communication unit configured to transfer file information stored in the file information storage unit to the second communication device in response to a request from the second communication device. Voice data storage means for storing voice data representing acoustic parameters for each unit such as phoneme for each type, and requesting the first communication device to transfer the file information stored in the file information storage means. Second communication means for receiving the file information transferred by the first communication means; and voice data corresponding to the type of voice in the utterance information of the file information received by the second communication means. Selection means for selecting from among a plurality of types of voice data stored in the color storage means, and either one or both of the loudness and pitch of the voice included in the utterance information and the time difference based on the both. Expansion means for expanding a prosody pattern continuous in the time axis direction, and audio reproduction means for generating an audio waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the selection means It is characterized by.

According to the second aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and in the second communication device, the loudness and pitch of the voice independent of the phoneme. In order to generate a speech waveform based on the prosody pattern that is temporally continuous and generate the voice waveform based on the prosody pattern and the timbre data selected by the information indicating the type of timbre in the utterance information, The voice can be reproduced with the optimum voice directly specified from a plurality of voices without limitation, and there is no shift in the voice pitch pattern during waveform synthesis. Even if the correspondence with the information is not fixed, it is possible to maintain high quality of speech synthesis by obtaining the optimum correspondence.

An information communication system according to a third aspect of the present invention connects the first communication device and the second communication device to a communication network,
In the information communication system for performing information communication between the first communication device and the second communication device via the communication network, the first communication device includes one or both of a loudness and a high pitch of the voice, Make the time difference correspond to the voice attribute,
A file for storing file information including utterance information obtained by discriminating one or both of a voice volume and a voice pitch so as to have a relative level without depending on a time difference between phonemes. An information storage unit; and a first communication unit configured to transfer file information stored in the file information storage unit to the second communication device in response to a request from the second communication device. Voice color storage means for storing voice data representing acoustic parameters for each unit such as phonemes and information indicating the attributes of the voice in association with each other, and file information storage means for the first communication device in the file information storage means Requesting the transfer of file information that has been
A second communication unit for receiving the file information transferred by the communication unit; and information indicating a voice attribute of the utterance information in the file information received by the second communication unit and stored in the voice storage unit. Collation means for comparing the information indicating the attributes of various voices to determine the similarity of the timbre, and a plurality of types of timbre data stored in the timbre storage means based on the similarity determined by the collation means. Selecting means for selecting voice data having the highest similarity, and a prosody pattern continuous in the time axis direction based on one or both of the loudness and loudness of the voice included in the utterance information and the time difference thereof Expansion means, and sound reproduction means for generating an audio waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the selection means. And it features.

According to the third aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and in the second communication device, the loudness and pitch of the voice independent of the phoneme. Now, a temporally continuous prosody pattern is developed, and a voice waveform is generated based on the prosody pattern and voice data selected by the similarity based on information indicating a voice attribute in the utterance information. Voice reproduction with the highest similarity can be performed without using inappropriate voices, and there is no shift in voice pitch pattern during waveform synthesis. Thus, correspondence between speech information and voice information Even if the relationship is not fixed, it is possible to maintain the high quality of speech synthesis by obtaining the optimal correspondence.

An information communication system according to a fourth aspect of the present invention connects the first communication device and the second communication device to a communication network,
In the information communication system for performing information communication between the first communication device and the second communication device via the communication network, the first communication device includes one or both of a loudness and a high pitch of the voice, The time difference, the type of timbre, and the attribute of the timbre are associated with each other, and one or both of the loudness and the pitch of the voice do not depend on the time difference between phonemes,
And file information storage means for storing file information including utterance information discretely having a relative level, and file information stored in the file information storage means in response to a request from the second communication device. First communication means for transferring to the communication device, the second communication device, for each type of timbre, voice data representing an acoustic parameter for each unit such as a phoneme and information indicating the attribute of the voice. Requesting the first communication device to transfer the file information stored in the file information storage unit, and receiving the file information transferred by the first communication unit thereafter; Second communication means, and search means for searching for the type of voice in the utterance information of the file information received by the second communication means from the various types of voice stored in the voice storage means. When the type of voice in the utterance information can be obtained by the search of the search means, the voice data corresponding to the obtained voice type is selected from the various voice data stored in the voice storage means. When the type of voice in the utterance information cannot be obtained by the first selection unit and the search unit, the information indicating the attribute of the voice in the utterance information stored in the utterance information storage unit and the voice Matching means for comparing the information indicating the attributes of various timbres stored in the storage means to obtain similarities of timbres; and a plurality of types stored in the timbre storage means based on the similarities determined by the matching means. A second selecting means for selecting the voice data having the highest similarity from the voice data of the first and second voices, based on one or both of the voice loudness and the voice pitch included in the utterance information and the time difference thereof. Expansion means for expanding a prosody pattern continuous in the interaxial direction, and audio reproduction means for generating an audio waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the first or second selection means And having the following.

According to the fourth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the voice volume and the pitch of the voice independent of the phoneme in the second communication device. Then, a temporally continuous prosody pattern is developed, and a voice waveform is generated based on the prosody pattern and the voice data selected by the information indicating the type and attribute of the voice in the utterance information. Even if there is no specified voice, the voice can be reproduced with the highest similarity without using an inappropriate voice, and there is no shift in the voice pitch pattern during waveform synthesis. Even if the correspondence between the utterance information and the timbre information is not fixed, it is possible to maintain the high quality of speech synthesis by obtaining the optimum correspondence.

An information communication system according to a fifth aspect of the present invention connects the first communication device and the second communication device to a communication network,
In an information communication system for performing information communication between a first communication device and a second communication device via the communication network, the first communication device includes a file storing file information including utterance information including phonemes and prosody as information. An information storage unit; and a first communication unit configured to transfer utterance information stored in the file information storage unit to the second communication device in response to a request from the second communication device. Voice data storage means for storing voice data representing acoustic parameters for each unit such as phoneme for each type, and requesting the first communication device to transfer the file information stored in the file information storage means. Second communication means for receiving the file information transferred by the first communication means, and voice information stored in the timbre storage means based on speech information in the file information received by the second communication means. Selecting means for selecting one voice data from several kinds of voice data, developing means for developing a continuous prosodic pattern in the time axis direction based on speech information, and prosody pattern and selecting means developed by the developing means And a sound reproducing means for generating a sound waveform based on the voice data selected by (1).

According to the fifth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device uses the utterance information in the file information in time. Since a continuous prosody pattern is developed and a voice waveform is generated based on the prosody pattern and the voice data selected based on the utterance information, the voice is reproduced with a suitable voice without being limited to a specific voice. And the voice pitch pattern does not deviate during waveform synthesis. In this way, it is possible to obtain the optimal correspondence without fixing the correspondence between speech information and voice color information. It is possible to maintain high quality.

An information communication system according to a sixth aspect of the present invention connects the first communication device and the second communication device to a communication network,
In the information communication system for performing information communication between the first communication device and the second communication device via the communication network, the first communication device includes file information including speech information including phonemes, prosody, and timbre type as information. And a first communication unit for transferring file information stored in the file information storage unit to the second communication device in response to a request from the second communication device. Requests voice color storage means for storing voice data representing acoustic parameters for each segment unit such as phoneme for each voice type, and requests the first communication device to transfer the file information stored in the file information storage means. Then, the second communication means for receiving the file information transferred by the first communication means, and corresponding to the type of voice in the speech information in the file information received by the second communication means Selecting means for selecting voice data from a plurality of types of voice data stored in the voice memory means; developing means for developing a continuous prosodic pattern in the time axis direction based on speech information; Voice reproducing means for generating a voice waveform based on the selected prosody pattern and the timbre data selected by the selecting means.

According to the sixth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continuously transmits the utterance information in the file information. Is developed based on the prosody pattern and the voice data selected based on the information indicating the type of voice in the utterance information, so that the voice is not limited to a specific voice. The voice can be reproduced with the optimum voice directly specified from a plurality of voices, and there is no shift in the voice pitch pattern during waveform synthesis. Thus, the correspondence between speech information and voice information Even if is not fixed, it is possible to maintain high quality of speech synthesis by obtaining an optimal correspondence.

An information communication system according to a seventh aspect of the present invention connects the first communication device and the second communication device to a communication network,
In an information communication system for performing information communication between a first communication device and a second communication device via the communication network, the first communication device includes file information including speech information including phonemes, prosody, and voice attributes as information. And a first communication unit for transferring file information stored in the file information storage unit to the second communication device in response to a request from the second communication device. Is a timbre storage means for storing, in association with each type of timbre, timbre data representing an acoustic parameter for each unit such as a phoneme and information indicating an attribute of the timbre, and file information for the first communication device. A second communication unit for requesting the transfer of the file information stored in the storage unit and subsequently receiving the file information transferred by the first communication unit; and a file received by the second communication unit. Matching means for comparing the information indicating the voice attributes of the utterance information in the report with the information indicating the attributes of various voices stored in the voice storage means; Selecting means for selecting voice data having the highest similarity from among a plurality of types of voice data stored in the voice memory based on the obtained similarity; and prosodic patterns continuous in the time axis direction based on the utterance information. It is characterized by having expansion means for expanding, and audio reproduction means for generating an audio waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the selection means.

According to the seventh aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device uses the utterance information in the file information in time. Since a continuous prosodic pattern is developed and a voice waveform is generated based on the prosodic pattern and voice data selected by similarity based on information indicating the voice attribute in the utterance information, an inappropriate voice is used. The voice reproduction with the highest similarity can be performed without performing the voice synthesis, and the voice pitch pattern does not shift during the waveform synthesis. Thus, the correspondence between the utterance information and the voice color information is not fixed. Even so, it is possible to maintain high quality of speech synthesis by obtaining an optimal correspondence.

An information communication system according to an eighth aspect of the present invention connects the first communication device and the second communication device to a communication network,
In the information communication system for performing information communication between the first communication device and the second communication device via the communication network, the first communication device includes utterance information including phonemes, prosody, timbre type, and vocal attribute as information. File information storage means for storing file information including: and first communication means for transferring file information stored in the file information storage means to the second communication device in response to a request from the second communication device, The second communication device is
Voice color storage means for storing, in association with each type of voice, voice data representing an acoustic parameter for each unit such as a phoneme and information indicating the attribute of the voice, and file information storage means for the first communication device. A second communication unit for requesting the transfer of the file information stored in the first communication unit, and thereafter receiving the file information transferred by the first communication unit; and speech information in the file information received by the second communication unit. Search means for searching for the type of timbre among the timbres stored in the timbre storage means.If the type of utterance in the utterance information can be obtained by the search of the search means, it can be obtained. First selecting means for selecting voice data corresponding to the selected voice type from the various voice data stored in the voice color storage means, and obtaining the voice type in the utterance information by searching the search means. If not, the information indicating the voice attribute of the utterance information stored in the utterance information storage means is compared with the information indicating the various timbre attributes stored in the timbre storage means. Matching means for determining the similarity, and second selecting means for selecting voice data having the highest similarity from among a plurality of types of voice data stored in the voice memory based on the similarity determined by the matching means. ,
Expansion means for expanding a continuous prosody pattern in the time axis direction based on the utterance information; and a speech waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the first or second selection means. And sound reproducing means for generating.

According to this invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally uses the utterance information in the file information. Since a continuous prosody pattern is developed and a voice waveform is generated based on the prosody pattern and voice data selected by information indicating the type and attribute of the voice in the utterance information, the directly specified voice is used. Even if there is no voice, the voice can be reproduced with the highest similarity without using an inappropriate voice, and there is no shift in the voice pitch pattern during waveform synthesis. Even if the correspondence with the information is not fixed, it is possible to maintain high quality of speech synthesis by obtaining the optimum correspondence.

According to a ninth aspect of the present invention, in the information communication system according to any one of the third, fourth, seventh, and eighth aspects, the information indicating the attribute is determined based on gender, age, and voice pitch. , Intelligibility, and naturalness, or a combination of two or more of them, the collation target of the attribute of the utterance information storage means and the attribute of the timbre storage means is parameterized, whereby It is possible to easily select a voice.

[0028] According to a tenth aspect of the present invention, in the information communication system according to any one of the first to eighth aspects, the file information storage means includes a first information indicating a reference of a voice pitch.
The information is stored in the utterance information, the timbre storage means stores the second information indicating the criterion of the voice pitch in the timbre data, and the voice reproducing means stores the criterion of the voice pitch based on the first information. Is shifted based on the voice pitch criterion according to the second information, thereby determining the voice pitch criterion during voice reproduction.

According to the tenth aspect of the invention, the reference of the voice pitch of the speech information storage means is shifted by the reference of the voice pitch of the timbre storage means at the time of voice reproduction.
The pitch of an individual voice changes relatively according to the shifted voice pitch criterion regardless of the time segment of the phoneme, so that the voice pitch criterion approaches the timbre side, so that the voice quality Can be further improved.

An information communication system according to an eleventh aspect of the present invention is the information communication system according to any one of the first to eighth aspects, wherein the file information storage means indicates the first reference of the voice pitch.
Information is included in the utterance information, and the voice reproducing means has input means for arbitrarily inputting second information indicating a reference of voice pitch, and input means for inputting a reference of voice pitch based on the first information. And determining the reference of the voice pitch at the time of the sound reproduction by shifting based on the reference of the voice pitch based on the second information input by (1).

According to the eleventh aspect of the present invention, the reference of the voice pitch of the utterance information storage means is shifted by an arbitrary reference of the voice pitch during the reproduction of the voice. Is relatively changed irrespective of the time division of the phoneme according to the shifted voice pitch criterion. Therefore, it is possible to process the timbre such as approaching the intended voice quality according to the shift amount.

According to a twelfth aspect of the present invention, in the information communication system according to the tenth or eleventh aspect, the criterion of the voice pitch based on the first and second information is determined based on the average voice frequency and the maximum frequency. , Or the minimum frequency, so that it becomes easier to take a reference for the voice pitch.

According to a thirteenth aspect of the present invention, in the information communication system according to any one of the first to eighth aspects, voice data is read from a recording medium in the second communication device and stored in voice color storage means. With this configuration, it is possible to give variations to the types of timbres through the recording medium, and it is possible to apply an optimum timbre at the time of voice reproduction.

An information communication system according to a fourteenth aspect of the present invention is the information communication system according to any one of the first to eighth aspects, wherein the second communication device receives voice color data from an external device via a communication line. Since the timbre data is stored in the timbre storage means, variations in timbre type can be given through a communication line, and an optimum timbre can be applied at the time of voice reproduction.

According to a fifteenth aspect of the present invention, in the information communication system according to any one of the first to eighth aspects, an operation based on other information of the file information and an operation performed by the sound reproducing means are included in the utterance information. Since the sound reproducing means operates in synchronization with the operation of the other information in the file information according to the control information included in the utterance information during the sound reproduction, including the control information for synchronizing the It is possible to enhance the power of expression by fusing the expression.

In the information communication system according to the sixteenth aspect of the present invention, in the invention according to the fifteenth aspect, the other information is image information, music information, or the like. It is possible to enhance expressive power.

According to a seventeenth aspect of the present invention, there is provided an information processing apparatus comprising:
An information processing apparatus for creating and editing utterance information used in the information communication system according to any one of claims 1 to 8, wherein the voice input means inputs a natural voice, and the natural information input by the voice input means. Requesting the first communication device to register file information including the utterance information generated by the generating unit, the generating unit generating the utterance information based on the voice;
The file information including the utterance information created thereafter is stored in the first
Registration transfer means for transferring to the communication device and registering it in the file information storage means of the first communication device.

According to the seventeenth aspect of the invention, based on the input natural speech, one or both of the voice volume and the voice pitch are not dependent on the time difference between phonemes, and Since the utterance information is created by being discrete so as to have a relative level, the utterance information is transferred to the first communication device and registered in the file information storage means. It is possible to give loudness and voice pitch.

An information processing apparatus according to claim 18 is
An information processing apparatus for creating and editing utterance information for use in the information communication system according to claim 10 or 11, wherein the creating means transmits the first information indicating a reference of voice pitch. Since the speech information is created so as to be included in the speech information, it is possible to provide a reference for the voice pitch in the speech information.

The information processing apparatus according to the nineteenth aspect of the present invention
According to the seventeenth aspect of the present invention, since the creating means includes the changing means for arbitrarily changing each information, it is possible to change the information for improving the quality of the voice.

According to a twentieth aspect of the present invention, there is provided an information processing apparatus comprising:
18. An information processing apparatus for creating and editing utterance information used in the information communication system according to claim 15 or 16, wherein the creating means creates control information when the utterance information is created. , So
Information for synchronizing the operation of speech synthesis with the operation based on other information can be provided in the speech information.

According to a twenty-first aspect of the present invention, there is provided an information communication method comprising:
Connecting a first communication device and a second communication device to a communication network;
In the communication device, one or both of the loudness of the voice and the pitch of the voice are associated with the time difference, and either the loudness of the voice and the pitch of the voice or both are determined by the time difference between the phonemes. And file information including utterance information that is discrete so as to have a relative level is stored in the file information storage unit in advance, and in the second communication device, phonemes such as phonemes are classified by color type. Voice data representing acoustic parameters for each unit is stored in the voice storage unit in advance, and is stored in the file information storage unit by performing information communication between the first communication device and the second communication device via a communication network. An information communication method for synthesizing voice based on utterance information in file information and voice data stored in a voice storage unit, the method comprising responding to a request from a second communication device to a first communication device. A transfer step of transferring the utterance information stored in the file information storage unit to the second communication device; and storing the utterance information in the voice storage unit based on the utterance information of the file information transferred by the transfer step in the second communication device. Selecting one voice data from a plurality of types of voice data being performed, and the second communication device performs one or both of the voice loudness and the voice pitch included in the utterance information, or both of them. A developing step of developing a continuous prosody pattern in the time axis direction based on the time difference, and generating a speech waveform based on the prosody pattern developed in the developing step and the timbre data selected in the selecting step in the second communication device And a sound reproducing step.

According to the twenty-first aspect of the present invention, file information including speech information is transferred from the first communication device to the second communication device. By developing a prosody pattern that is temporally continuous and generating a voice waveform based on the prosody pattern and voice data selected based on the utterance information, it is suitable without being limited to a specific voice. It is possible to reproduce voices with different timbres, and there is no shift in voice pitch pattern during waveform synthesis. In this way, the optimum correspondence between utterance information and timbre information can be obtained without fixing the correspondence. This makes it possible to maintain high quality of speech synthesis.

The information communication method according to the invention of claim 22 is as follows:
Connecting a first communication device and a second communication device to a communication network;
In the communication device, one or both of the loudness and the loudness of the voice, the time difference thereof, and the type of the timbre are associated with each other, and either or both of the loudness and the loudness of the voice are mixed. Is independent of the time difference between phonemes,
In addition, file information including utterance information that is discrete so as to have a relative level is stored in the file information storage unit in advance, and in the first communication device, for each type of voice, each unit of phoneme or the like is divided. Voice data representing sound parameters is stored in the voice storage unit in advance, and information communication is performed between the first communication device and the second communication device via a communication network, so that the file information stored in the file information storage unit An information communication method for synthesizing voice based on the utterance information and voice color data stored in a voice color storage unit, wherein the information is stored in file information storage means in response to a request from a first communication device to a second communication device. Transfer means for transferring the received file information to the second communication device, and corresponding to the type of voice in the utterance information of the file information transferred by the transfer process in the second communication device. A selection step of selecting voice data from a plurality of types of voice data stored in the voice storage unit; and a second communication device that selects one or both of a voice volume and a voice pitch included in the utterance information. A developing step of developing a continuous prosody pattern in the time axis direction based on both of them and a time difference between the two, and a voice based on the prosody pattern developed in the developing step and the timbre data selected in the selecting step in the second communication device. And a sound reproducing step of generating a waveform.

According to the twenty-second aspect of the present invention, file information including speech information is transferred from the first communication device to the second communication device. Then, a temporally continuous prosody pattern is developed, and the voice waveform is generated based on the prosody pattern and the voice data selected by the information indicating the type of the voice in the utterance information. The voice can be reproduced with the optimum voice directly specified from a plurality of types of voices without being limited to the above, and there is no shift in the voice pitch pattern during waveform synthesis. Even if the correspondence with the timbre information is not fixed, it is possible to maintain high quality of speech synthesis by obtaining the optimum correspondence.

An information communication method according to a twenty-third aspect of the present invention
Connecting a first communication device and a second communication device to a communication network;
In the communication device, one or both of the loudness and the loudness of the voice, the time difference thereof, and the attribute of the timbre are associated with each other, and either or both of the loudness and the loudness of the voice are mixed. Is independent of the time difference between phonemes,
In addition, file information including utterance information that is discrete so as to have a relative level is stored in the file information storage unit in advance, and in the second communication device, for each voice type, for each unit of phoneme or the like, The voice data representing the acoustic parameters of the voice and the information indicating the attributes of the voice are stored in advance in the voice storage unit, and information communication is performed between the first communication device and the second communication device via a communication network. An information communication method for synthesizing voice based on utterance information of the file information stored in the file information storage unit and voice data stored in the voice color storage unit, wherein the second communication with the first communication device is performed. A transfer step of transferring the file information stored in the file information storage unit to the second communication apparatus in response to a request from the communication apparatus, and a file information transferred by the transfer step in the second communication apparatus A collation step of collating information indicating the attribute of the timbre in the utterance information within the utterance information with information indicating the attributes of various timbres stored in the timbre storage unit to obtain a similarity of the timbre; A selecting step of selecting voice data having the highest similarity from among a plurality of types of voice data stored in the voice memory based on the similarity obtained in the step; A development step of developing a continuous prosody pattern in the time axis direction based on one or both of the loudness and pitch of the voice and a time difference between the two, and a prosody developed by the development step in the second communication device A sound reproduction step of generating a sound waveform based on the pattern and the voice data selected in the selection step.

According to the twenty-third aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device. By developing a temporally continuous prosody pattern, a voice waveform was generated based on the prosody pattern and voice data selected by similarity based on information indicating a voice attribute in the utterance information. Voice reproduction with the highest similarity can be performed without using inappropriate voices, and there is no shift in voice pitch pattern during waveform synthesis. Thus, correspondence between speech information and voice information Even if the relationship is not fixed, it is possible to maintain the high quality of speech synthesis by obtaining the optimal correspondence.

An information communication method according to a twenty-fourth aspect of the present invention,
Connecting a first communication device and a second communication device to a communication network;
In the communication device, one or both of the loudness and the loudness of the voice, the time difference thereof, the type of the timbre, and the attribute of the timbre are associated with each other, and one of the loudness and the loudness of the voice is set. On the other hand, file information including utterance information obtained by discriminating one or both of them without depending on the time difference between phonemes and having a relative level is stored in the file information storage unit in advance, In a communication device,
For each type of timbre, timbre data representing acoustic parameters for each unit such as a phoneme and information indicating the attribute of the timbre are stored in advance in a timbre storage unit, and the first communication is performed via a communication network. Information communication for performing voice communication based on utterance information of file information stored in a file information storage unit and voice data stored in a voice color storage unit by performing information communication between the device and the second communication device In the method, the file information stored in the file information storage unit is transferred to the second communication device in response to a request from the second communication device to the first communication device, and the file information is transferred by the transfer process in the second communication device. A search step of searching for the type of voice in the utterance information of the file information obtained from the various types of voice stored in the voice storage unit, and a search performed by the search step in the second communication device. A first selecting step of, when the type of voice in the information has been obtained, selecting voice data corresponding to the obtained voice type from among various types of voice data stored in the voice storage unit; If the type of voice in the utterance information cannot be obtained by the search in the search step in the second communication device, information indicating the attribute of the voice in the utterance information stored in the file information storage unit and voice color storage A collation step of comparing the information indicating the attributes of various timbres stored in the section to obtain similarities of timbres; and a timbre storage step stored in the timbre storage step based on the similarities determined by the collation steps in the second communication device. A second selecting step of selecting voice data having the highest similarity from the plurality of types of voice data,
A developing step of developing a continuous prosody pattern in the time axis direction based on one or both of the voice volume and voice pitch included in the utterance information in the second communication device; And a voice reproducing step of generating a voice waveform based on the prosody pattern expanded in the expanding step and the timbre data selected in the first or second selecting step in the communication device.

According to the twenty-fourth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device. Then, a time-continuous prosody pattern was developed, and a voice waveform was generated based on the prosody pattern and voice data selected based on information indicating the type and attribute of the voice in the speech information. Even if there is no specified voice, the voice can be reproduced with the highest similarity without using an inappropriate voice, and there is no shift in the voice pitch pattern during waveform synthesis. Even if the correspondence between the utterance information and the timbre information is not fixed, it is possible to maintain the high quality of speech synthesis by obtaining the optimum correspondence.

[0050] The information communication method according to the invention of claim 25 is characterized in that:
Connecting a first communication device and a second communication device to a communication network;
In the communication device, file information including utterance information including phonemes and prosody as information is stored in the file information storage unit in advance, and in the second communication device, the sound for each unit of phoneme or the like is classified by color type. Voice data representing parameters is stored in the voice storage unit in advance, and information communication is performed between the first communication device and the second communication device via the communication network, so that the file information stored in the file information storage unit is stored. An information communication method for synthesizing voice based on utterance information in a voice and voice color data stored in a voice color storage unit, wherein the voice communication is stored in the file information storage unit in response to a request from a second communication device to a first communication device. Transferring the received utterance information to the second communication device, and writing the utterance information in the voice storage unit based on the utterance information in the file information transferred by the transfer process in the second communication device. A selecting step of selecting one voice data from a plurality of types of voice data, a developing step of developing a continuous prosody pattern in the time axis direction based on the utterance information in the second communication device; The apparatus includes a sound reproducing step of generating a sound waveform based on the prosody pattern expanded in the expanding step and the timbre data selected in the selecting step.

According to the twenty-fifth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally uses the utterance information of the file information. Since a continuous prosody pattern is developed and a voice waveform is generated based on the prosody pattern and the voice data selected based on the utterance information, the voice is reproduced with a suitable voice without being limited to a specific voice. Can be
In addition, there is no shift in the voice pitch pattern during waveform synthesis, and thus high optimal speech synthesis quality can be obtained by obtaining the optimal correspondence without fixing the correspondence between speech information and voice color information. It is possible to maintain

According to a twenty-sixth aspect of the present invention, there is provided an information communication method comprising:
Connecting a first communication device and a second communication device to a communication network;
In the communication device, file information including utterance information including phonemes, prosody, and timbre types as information is stored in a file information storage unit in advance, and in the second communication device, segments such as phonemes are classified by timbre type. Voice data representing acoustic parameters for each unit is stored in the voice storage unit in advance,
By performing information communication between the first communication device and the second communication device via the communication network, the utterance information of the file information stored in the file information storage unit and the voice data stored in the voice color storage unit A communication method for synthesizing speech based on the first communication device, wherein a transfer step of transferring file information stored in the file information storage means to the second communication device in response to a request from the second communication device to the first communication device; (2) a selecting step of selecting voice data corresponding to a voice type in the utterance information of the file information transferred in the transfer step in the communication device from a plurality of types of voice data stored in the voice storage unit; And a developing step of developing a continuous prosody pattern in the time axis direction based on the utterance information in the second communication device, and a prosody pattern developed by the developing step in the second communication device Characterized in that it includes a sound reproducing step of generating a speech waveform, the based on the selection process tone of voice data selected by.

According to the twenty-sixth aspect, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally uses the utterance information of the file information. Since a continuous prosody pattern is developed and a voice waveform is generated based on the prosody pattern and voice data selected by information indicating the type of voice in the speech information, the process is not limited to a specific voice. Can also play audio with the optimal voice specified directly from multiple types of voices,
In addition, there is no shift in the voice pitch pattern during waveform synthesis, and thus high optimal speech synthesis quality can be obtained by obtaining the optimal correspondence without fixing the correspondence between speech information and voice color information. It is possible to maintain

An information communication method according to a twenty-seventh aspect of the present invention
Connecting a first communication device and a second communication device to a communication network;
In the communication device, file information including utterance information including phonemes, prosody, and voice attributes as information is stored in the file information storage unit in advance, and in the second communication device, elements such as phonemes are classified by voice type. Voice data representing an acoustic parameter for each unit and information indicating an attribute of the voice are stored in a voice storage unit in advance in association with each other, and information is transmitted between the first communication device and the second communication device via a communication network. An information communication method for performing voice communication based on utterance information of file information stored in a file information storage unit and voice data stored in a voice color storage unit by performing communication, comprising: a first communication device; A transfer step of transferring the file information stored in the file information storage unit to the second communication device in response to a request from the second communication device for the second communication device, and a transfer process in the second communication device. A collation step of collating information indicating a voice attribute in the utterance information of the file information with information indicating various voice attributes stored in the voice storage unit to obtain a similarity of the voice; A selecting step of selecting voice data having the highest similarity from a plurality of types of voice data stored in the voice storage unit based on the similarity obtained by the matching step in the device; Developing a prosody pattern continuous in the time axis direction based on the prosody pattern, and generating a speech waveform based on the prosody pattern expanded by the expansion step and the timbre data selected by the selection step in the second communication device And a regeneration step.

According to the twenty-seventh aspect, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device uses the utterance information of the file information in time. Uses an unsuitable timbre because it develops a continuous prosody pattern and generates an audio waveform based on the rhythm pattern and timbre data selected by similarity based on information indicating the timbre attribute in the utterance information. The voice reproduction with the highest similarity can be performed without performing the voice synthesis, and the voice pitch pattern does not shift during the waveform synthesis. Thus, the correspondence between the utterance information and the voice color information is not fixed. Even so, it is possible to maintain high quality of speech synthesis by obtaining an optimal correspondence.

According to a twenty-eighth aspect of the present invention, there is provided an information communication method comprising:
Connecting a first communication device and a second communication device to a communication network;
In the communication device, file information including speech information including phonemes, prosody, voice type, and voice attribute as information is stored in the file information storage unit in advance, and in the second communication device, for each voice type, Voice data representing an acoustic parameter for each unit such as a phoneme and information indicating an attribute of the voice are stored in advance in the voice storage unit in association with each other,
By performing information communication between the first communication device and the second communication device via the communication network, the utterance information of the file information stored in the file information storage unit and the voice data stored in the voice color storage unit A communication method for synthesizing voice based on the first communication device, wherein a transfer step of transferring file information stored in the file information storage unit to the second communication device in response to a request from the second communication device to the first communication device; (2) a retrieval step of retrieving the type of voice in the utterance information of the file information transferred by the transmission step in the communication apparatus from the various types of voice stored in the voice storage unit; and a search in the second communication apparatus. If the type of voice in the utterance information can be obtained by the search for the process, the voice data corresponding to the obtained voice type is selected from among the various voice data stored in the voice storage unit. If the type of vocal tone in the utterance information cannot be obtained by the first selection step to be selected and the search step in the second communication device, the timbre in the utterance information stored in the file information storage unit And the information indicating the attributes of various timbres stored in the timbre storage unit to determine the similarity of the timbre, and the similarity determined by the matching process in the second communication device. Second selecting the voice data having the highest similarity from the plurality of types of voice data stored in the voice memory based on the second voice data;
A selecting step, a developing step of developing a continuous prosody pattern in the time axis direction based on the utterance information in the second communication device, and a prosody pattern developed by the developing step in the second communication device and the first or second selecting step And a voice reproduction step of generating a voice waveform based on the voice color data selected by (1).

According to the twenty-eighth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device uses the utterance information in the file information in time. Since the continuous prosody pattern is developed and the voice waveform is generated based on the prosody pattern and the voice data selected by the information indicating the type and attribute of the voice in the utterance information, the directly specified voice is used. Even if there is no voice, the voice can be reproduced with the highest similarity without using an inappropriate voice, and there is no shift in the voice pitch pattern during waveform synthesis. Even if the correspondence with the information is not fixed, it is possible to maintain high quality of speech synthesis by obtaining the optimum correspondence.

An information communication method according to a twenty-ninth aspect of the present invention is characterized in that:
In the invention according to any one of claims 23, 24, 27, and 28, the information indicating the attribute is any one of gender, age, voice pitch criterion, intelligibility, and naturalness, Alternatively, since the two or more steps are combined, the collation target between the attribute of the utterance information storage unit and the attribute of the timbre storage unit is parameterized, thereby making it easy to select the timbre.

According to a thirtieth aspect of the present invention, there is provided an information communication method comprising:
The invention according to any one of claims 21 to 28, wherein the file information storage unit stores first information indicating a reference of voice pitch in the utterance information, and the voice color storage unit stores the voice pitch. Is stored in the timbre data in the voice data, and the voice reproduction step shifts the voice pitch standard according to the first information by the voice pitch standard according to the second information. It is characterized in that a criterion for the pitch of the time is determined.

According to the thirtieth aspect, the step of shifting the reference of the voice pitch of the utterance information storage means at the time of voice reproduction is performed according to the reference of the voice pitch of the timbre storage means.
The pitch of an individual voice changes relatively according to the shifted voice pitch criterion regardless of the time segment of the phoneme, so that the voice pitch criterion approaches the timbre side, so that the voice quality Can be further improved.

An information communication method according to a thirty-first aspect of the present invention comprises:
In the invention according to any one of claims 21 to 28, the file information storage unit stores the first information indicating the reference of the voice pitch included in the utterance information, and the voice reproducing step includes the step of reproducing the voice pitch. And an input step of arbitrarily inputting second information indicating the reference of (i), and shifting the reference of the voice pitch by the first information by the reference of the voice pitch by the second information input by the input step. Is used to determine a criterion for the pitch of voice at the time of voice reproduction.

According to the thirty-first aspect of the present invention, the step of shifting the reference of the voice pitch of the utterance information storage means at the time of voice reproduction by an arbitrary reference of the voice pitch is performed. Is relatively changed irrespective of the time division of the phoneme according to the shifted voice pitch criterion. Therefore, it is possible to process the timbre such as approaching the intended voice quality according to the shift amount.

The information communication method according to the invention of claim 32 is:
31. The invention according to claim 30 or 31, wherein the reference of the voice pitch based on the first and second information is an average frequency of the voice pitch,
Since the maximum frequency or the minimum frequency is set, it becomes easy to set a reference for the voice pitch.

According to a thirty-third aspect of the present invention, there is provided an information communication method comprising:
In the invention according to any one of claims 21 to 28, in the second communication device, the voice data is read from the recording medium and stored in the voice storage unit, so that the voice type is varied through the recording medium. This makes it possible to apply an optimal voice tone during voice reproduction.

The information communication method according to the invention of claim 34, comprises:
In the invention according to any one of claims 21 to 28, since the second communication device receives voice data from an external device via a communication line and stores the voice data in the voice storage unit, Variations can be given to the types of voices through a communication line, and it is possible to apply an optimum voice at the time of voice reproduction.

An information communication method according to claim 35 is characterized in that:
In the invention according to any one of claims 21 to 28, the utterance information includes control information for synchronizing an operation based on other information in the file information with an operation based on the audio reproduction step, and the audio reproduction step includes the audio reproduction step. Sometimes the operation is performed in synchronization with the operation of other information in the file information according to the control information included in the utterance information, so the expression of voice and other media can be fused to enhance the expressive power It is.

[0067] The information communication method according to claim 36 is:
The invention according to claim 35, wherein the other information is image information,
Since the music information or the like is used, it is possible to enhance the expressive power by fusing the expression of the voice with the image, the music, and the like.

The information processing method according to claim 37 is characterized in that:
An information processing method for creating and editing utterance information used in the information communication method according to any one of claims 21 to 28 by a third communication device connected to a communication network, wherein a natural sound is input. Requesting the first communication device to register file information including the utterance information created in the creating step, and a creation step of creating the utterance information based on the natural speech input in the speech input step; Transferring the file information including the utterance information created in the first communication device to the first communication device and registering the file information in the file information storage unit of the first communication device.

According to the thirty-seventh aspect, based on the input natural speech, one or both of the loudness and the pitch of the voice do not depend on the time difference between phonemes, and Since the utterance information is created by discretely generating the utterance information so as to have a relative level, the utterance information is transferred to the first communication device and registered in the file information storage means. It is possible to give loudness and voice pitch.

An information processing method according to a thirty-eighth aspect of the present invention
37. The information processing method according to claim 37, wherein the utterance information used in the information communication method according to claim 30 or 31 is created and edited, wherein the creation step includes: Since the step of creating the speech information is included in the speech information, it is possible to provide a reference for the voice pitch in the speech information.

The information processing method according to claim 39 is
In the invention according to claim 37, since the creation step includes a change step of arbitrarily changing each piece of information, it is possible to change information for improving the voice quality.

The information processing method according to claim 40 is:
37. An information processing method according to claim 37, wherein the utterance information used in the information communication method according to claim 35 or 36 is created and edited, wherein control information is generated when the creation step creates the utterance information. In the utterance information, information for synchronizing the operation of speech synthesis with the operation based on other information can be provided.

[0073]

Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The embodiments described below exemplify the Internet.

FIG. 1 is a configuration diagram showing an embodiment of the information communication system according to the present invention. This information communication system connects a host device 1 (a first communication device according to claims 1 to 8) and a plurality of terminal devices to a communication network NET such as an ISDN network, and communicates with the host device 1 via the communication network NET. In this configuration, information communication is performed between the terminal devices. FIG. 1 shows a terminal device 2 (second embodiment of claims 1 to 8) as a representative of a plurality of terminal devices.
Communication device) as an example.

The host device 1 includes a communication unit 10 (first communication means of claims 1 to 8) connected to the communication network NET, a database (hereinafter referred to as DB) 11 (file information storage of claims 1 to 8). Means), a control unit 12 (first communication means of claims 1 to 8, registration means of claim 9) and the like.

The communication unit 10 is a unit for controlling information communication (including voice communication) with the terminal device 2 via the communication network NET, and the DB 11 is file information including speech information created by the terminal device 2 and its own device. Is a memory for registering a file in units of files. The control unit 12 receives a file in response to a file registration request from the terminal device 2 and registers it in the DB 11, and stores necessary file information in response to a request from the terminal device 2.
Control such as reading from B11 and transferring to terminal device 2 is performed.

The above-mentioned utterance information is obtained by associating one or both of the loudness of the voice and the pitch of the voice with the time difference and the type of the timbre. One or both of them is information that is discrete so as to have a relative level without depending on the time difference between phonemes.

The terminal device 2 is connected to a communication unit 20 (second communication means of claims 1 to 8;
Registration / transfer unit), voice storage unit 21 (voice storage unit of claims 1 to 8), application storage unit 22, speaker 23 (voice playback unit of claims 1 to 8), control unit 24 (claims 1 to 8). The second communication means, the selection means, the development means, and the sound reproduction means, the creation means and the registration transfer means of claim 17), the display unit 25 (the creation means of claim 17), and the like.

The communication section 20 is a unit for controlling information communication (including voice communication) with the host device 1 via the communication network NET, and the voice storage section 21 is a memory for storing voice data. The timbre data is data representing acoustic parameters for each unit of unit such as phonemes for each type of timbre.

The application storage section 22 has a sound processing PM (program memory) 221.
Communication network NET, FD (floppy disk), CD
(Compact disk)-Add a program for this narration processing PM221 through a recording medium such as a ROM,
Operations such as change and deletion are possible.

This narration processing PM 221 is performed as shown in FIG.
6, file reproduction processing according to the flowcharts shown in FIGS. 17 and 18, speech information creation processing according to the flowchart shown in FIG. 23, new creation processing according to the flowchart shown in FIG. 24, and FIG. Playback processing according to the flowchart shown in FIG. 34, editing processing according to the flowchart shown in FIG. 34,
A program for executing a file registration process and the like according to the flowchart shown in FIG. 35 is stored.

The file transfer process shown in FIG. 16 means that the terminal device 2 requests the host device 1 for file information including desired utterance information, receives the file information transferred from the host device 1, and reproduces the sound. This is the process of executing the output process.

The reproduction process shown in FIGS. 17 and 18 is a process specifically showing the operation of audio reproduction in the file transfer process.

The utterance information creation processing shown in FIG. 23 is processing including new creation, editing, and file registration of utterance information indicating discrete prosody, which does not include voice data based on natural speech.

The new creation process of FIG. 24 is a process specifically showing a new creation operation in the above utterance information creation process.

The interrupt reproduction process of FIG. 25 is a process specifically showing a sound reproduction operation when a reproduction request is made during the operation of the above-mentioned new creation processing or editing processing.

The editing process in FIG. 34 is a process specifically showing the editing operation in the above-mentioned utterance information creating process, and the editing target is a file (utterance information) that has already been created.

The file registration process shown in FIG. 35 is a process specifically showing a file registration operation in the utterance information creation process. That is, the file registration process is a process in which the terminal device 2 requests the host device 1 to register desired file information, and transfers the file information to the host device 1 for registration.

The speaker 23 is a sound output unit that outputs a synthesized voice or the like reproduced by synthesizing the waveform of the utterance information and the voice color data in the above-described reproduction processing or interrupt reproduction processing.

The display unit 25 displays a file of speech information,
LCD and C to form a display screen during processing such as transfer and registration
A display unit such as an RT.

Next, the manner in which the host device 1 manages file information will be described in detail.

FIG. 2 is a diagram showing an example of a memory configuration of the DB 11 of the host device 1.

As shown in FIG. 2, the DB 11 stores file information including speech information in association with files A, B, C,. For example, the file information of file A is speech information (header information HDRA and speech information PRS).
A), image information IMGA, and program information PROA are stored in association with each other. Similarly, file information of file B is utterance information (header information HDRB and utterance information P
RSB), image information IMGB, program information PROB
And file information of the file C is utterance information (header information HDRC and utterance information PRSC),
The image information IMGC and the program information PROC are stored in association with each other. Since the Internet is taken as an example of the system, the file information A, B, C
…, The program information PROA, PROB, PR
OC ... is information described in the HTML language, and creates a homepage or the like.

FIG. 3 is a diagram showing an example of header information in the speech information, FIG. 4 is a diagram showing a configuration example of speech information in the speech information, and FIG. 5 is a diagram showing a configuration example of a speech event in the speech information. FIG. 6 and FIG. 6 are diagrams for explaining the contents of the velocity level, and FIG. 7 is a diagram showing a configuration example of a control event in the utterance information.

Here, the utterance information of the file A will be described as an example. FIG. 3 shows header information HDR of file A.
A is shown. The header information HDRA includes a phoneme group PG, a language code LG, a time resolution TD, voice color designation data VP, pitch reference data PB, and volume reference data V
B.

The phoneme group PG and the language code LG are data for specifying a phoneme group and a language code of a phoneme unit 42 (see FIG. 8), which will be described later. The data specifies a phoneme table used for speech synthesis. You.

The time resolution data TD is data for specifying a basic unit time of a time difference between phonemes. The voice designation data VP is used for voice synthesis 211 described later (FIG.
) File is specified (selected), and the data specifies the type of timbre used at the time of speech synthesis, that is, timbre data.

The pitch reference data PB is data for defining a reference voice pitch (pitch frequency). Although an average pitch is used as an example of the pitch standard, another standard such as a maximum frequency or a minimum frequency of the pitch may be applied in addition to the average pitch. At the time of synthesizing the audio waveform, for example, the pitch can be changed in the range of one octave up and down with reference to the pitch based on the pitch reference data PB.

The volume reference data VB is data for specifying the reference of the overall volume.

FIG. 4 shows utterance information PRSA of file A.
It is shown. The utterance information PRSA is the time difference data D
It has a structure in which T and event data (the utterance event PE or the control event CE) are alternately associated with each other, and does not depend on the time difference between phonemes.

Time difference data DT is data for specifying a time difference between event data. The unit of the time difference indicated by the time difference data DT is specified by the time resolution TD of the header information in the speech information.

The utterance event PE in the event data
Is data composed of a phoneme to be uttered, a voice pitch relatively designating a voice pitch, a velocity relatively designating a voice strength, and the like.

Control event CE in event data
Is data designated for performing a volume change or the like in the middle as control other than the parameters designated by the above-mentioned utterance event PE.

Next, the utterance event PE will be described in detail with reference to FIGS.

The utterance event PE is, as shown in FIG.
There are three types of phoneme event PE1, pitch event PE2, and velocity event PE3.

The phoneme event PE1 has a structure in which identification information P1, voice intensity (velocity), and phoneme code PH are associated with each other, and is an event for specifying a phoneme at the same time as specifying a phoneme.

The identification information P1 added to the head of the phoneme event PE1 indicates that the event type is the phoneme event PE1 in the utterance event PE.

The voice intensity VL is data for specifying the volume (velocity), and is specified as a sensible voice intensity.

When this voice intensity VL is divided into, for example, 3 bits of 8 values, and each value is associated with a tone symbol, as shown in FIG. 6, the values “0” and “1” are obtained. ... The value “7” corresponds to silence, pianissimo (ppp)... Fortissimo (fff), respectively.

Since the value of the actual voice strength VL and the physical voice strength depend on voice data in voice synthesis, for example, the VL of the vowel "A" and the vowel "A" All of the values may be set to standard values, and if the standard values are used, the vowel “A” can be made larger in physical voice strength than the vowel “A” by the timbre data. In general, the average amplitude power of the vowel "A" is larger than the amplitude power of the vowel "A".

The phoneme code PH is data for specifying a phoneme code in each of the phoneme tables (see FIGS. 10, 11, and 12). In this embodiment, the phoneme code is 1-byte data.

The pitch event PE2 has a structure in which the identification information P2 is associated with the voice pitch (pitch) PT, and is an event for designating the voice pitch at an arbitrary time. In the pitch event PE2, the pitch of the voice can be specified independently of the phonemes (without depending on the time difference between the phonemes), and the pitch of the voice can be specified at a fine time interval within a time segment of one phoneme. . These designations and their operations are essential conditions for generating high-quality prosody.

The identification information P2 added to the head of the pitch event PE2 indicates that the event type is a pitch event in the utterance event PE.

The voice pitch PT does not indicate the absolute voice pitch, but is data specified relative to the reference (center) with respect to the pitch reference indicated by the pitch reference data PB in the header information. is there.

When the voice pitch PT is 1-byte data, the value is specified in a range of one octave up and down on the basis of the pitch, represented by gradations of 0 to 255. If the pitch PT of the voice is defined by, for example, a pitch frequency f [Hz], the following expression (1) is obtained.

That is, f = PBV · ((PT / 256) ² + 0.5 · (PT / 256) +0.5) (1) Here, PBV is a pitch reference value (unit: Hz) specified by the pitch reference data PB.

Conversely, the value of the pitch reference PT can be obtained from the pitch frequency f according to the following equation (2). This equation (2) is as follows.

That is, PT = 64 · (√ ((16 · f / PBV) −7) −1) (2)

The velocity event PE3 has the identification information P
3 is an event that has a structure in which the voice strength VL is associated with the voice strength VL, and specifies the voice strength at an arbitrary point in time. In the velocity event PE3, the strength of the voice can be specified independently of the phonemes (independent of the time difference between the phonemes), and the strength of the voice can be specified at a fine time interval within the time division of one phoneme. . These designations and their operations are essential conditions for generating high-quality prosody.

The voice strength VL is basically specified for each phoneme. However, when the voice strength (velocity) is changed in the middle of one phoneme such as a long sound, the voice strength VL is independent of the phoneme. , The velocity event PE3 is additionally specified at any time.

Next, the control event CE will be described in detail with reference to FIG.

The control event CE is a volume event CE1.
(See FIG. 7A) and pitch reference event CE2 (see FIG. 7A).
(See (B)).

The volume event CE1 has a structure in which the identification information C1 is associated with the volume data VBC, and specifies that the volume reference data VB specified by the header information HDRA is changed on the way.

That is, this is an event used when operating the overall volume level up or down, and the volume reference is determined by the header information HDRA until the volume is designated by the next volume event CE1 in the time axis direction. The specified volume reference data VB is replaced with the specified volume data VBC.

The identification information C1 added to the head of the volume event CE1 indicates the volume which is the type of the control event.

The pitch reference event CE2 includes the identification information C
2 has a structure in which pitch reference data PBC is associated with the pitch reference data PBC, and is an event specified when the pitch exceeds the range of the voice pitch that can be specified by the pitch reference data PB specified by the header information HDRA.

That is, this is an event used when the overall pitch reference is operated up and down, and the pitch reference is set in the header information until the next pitch reference event CE2 specifies the pitch reference in the time axis direction. The pitch reference data PB specified by HDRA is replaced with the specified pitch reference data PBC. Thereafter, the pitch of the voice changes within a range of one octave above and below the pitch reference data PBC.

Next, the terminal device 2 will be described in detail. FIG.
FIG. 2 is a block diagram showing an internal configuration of the terminal device 2.

The terminal device 2 comprises a control unit 24, a key input unit 2
9 (input means of claim 11, creation means of claim 17, change means of claim 19), application storage unit 22,
Voice storage unit 21, DB 26, original waveform storage unit 27, microphone 28 (voice input means of claim 17), speaker 23, display unit 25, interface (I / F) 30, FD drive 31, CD-ROM drive 32, A unit such as the communication unit 20 is provided.

The control unit 24 is a central processing unit that controls each unit connected to the bus BS. The control unit 24 detects the key operation of the key input unit 2, executes an application, adds / deletes information relating to timbre, phoneme, and utterance, creates / transmits utterance information, stores original waveform data, and stores various display screens. Control operations such as forming.

The control unit 24 includes a CPU 241 and a ROM
242, a RAM 243, and the like. The CPU 241 operates according to the OS program stored in the ROM 242 and the application program (such as the sound processing PM (program memory) 31) stored in the application storage unit 22.

The ROM 242 is a recording medium storing an OS (operating system) program and the like.
The RAM 243 is a memory used as a work area for the various programs described above, and is also used when temporarily storing transmission / reception data.

A key input unit 29 is an input device such as various keys or a mouse for causing the control unit 24 to detect, as a key signal, an instruction such as file creation / transmission / reception / filing relating to speech information and file transmission / reception / filing of the voice storage unit 21. Is provided.

The application storage unit 22 is a recording medium that stores an application program such as a narration process PM221. The application storage unit 22 can perform operations such as addition, change, and deletion of the program of the narration processing PM 221 through another recording medium such as a communication network NET, an FD (floppy disk), and a CD (compact disk) -ROM. It is.

This narration processing PM 221 is performed as shown in FIG.
6, file reproduction processing according to the flowcharts shown in FIGS. 17 and 18, speech information creation processing according to the flowchart shown in FIG. 23, new creation processing according to the flowchart shown in FIG. 24, and FIG. Playback processing according to the flowchart shown in FIG. 34, editing processing according to the flowchart shown in FIG. 34,
A program for executing a file registration process and the like according to the flowchart shown in FIG. 35 is stored.

The file transfer process shown in FIG.
Requests the host device 1 for desired file information (including utterance information, image information, and the like), receives the file information transferred from the host device 1, and executes reproduction of voice, image, and the like. .

The reproduction process shown in FIGS. 17 and 18 is a process specifically showing a reproduction operation of voice and image in the file transfer process.

The utterance information creation processing shown in FIG. 23 includes new creation, editing, and creation of utterance information (see FIGS. 3 to 7) based on natural speech that does not include voice data representing acoustic parameters for each unit such as phonemes. And filing.

The new creation process in FIG. 24 is a process specifically showing a new creation operation in the utterance information creation process.

The interrupt reproduction process shown in FIG. 25 is a process specifically showing an audio reproduction operation when a reproduction request is made during the operation of the above-mentioned new creation processing or editing processing.

The editing process in FIG. 34 is a process specifically showing the editing operation in the above-mentioned utterance information creating process, and the editing target is the utterance information of a file that has already been created.

The file registration process shown in FIG. 35 is a process in which the terminal device 2 requests the host device 1 to register desired file information and transfers it to the host device 1.

The timbre storage unit 21 is a recording medium for storing timbre data indicating various timbres, and includes a timbre unit 211 and a phoneme unit 212. The timbre section 211 stores vocal data representing acoustic parameters for each unit such as phoneme for each type of timbre in a selectable manner (see FIG. 9).
Stores a phoneme table in which phonemes and phoneme codes are associated with each phoneme group to which each language belongs (see FIGS. 10 to 13).

Both the voice part 211 and the phoneme part 212
It is possible to add voice data or a phoneme table through a recording medium such as the communication line LN, FD, CD-ROM or the like, or delete it by operating the key input unit 29.

The DB 26 stores utterance information in file units. The utterance information includes utterance information composed of discrete phonemes and discrete prosody information (phoneme group, time difference between utterances and utterance control, pitch of voice, and loudness of voice). This is information including header information to be defined (language, time resolution, voice color designation, pitch reference indicating a reference voice volume, volume reference indicating a reference volume).

At the time of voice reproduction, the voice can be reproduced by developing a discrete prosody into a continuous prosody pattern based on the utterance information and synthesizing a waveform with voice data indicating the voice color of the voice based on the header information. .

The original waveform storage section 27 is a recording medium for storing natural speech for creating a file of speech information in the form of waveform data. The microphone 28 is a voice input unit for inputting natural voice by performing processing such as creating a file of speech information.

The speaker 23 is a sound output unit that outputs a sound such as a synthesized sound reproduced in the above-described reproduction processing or interrupt reproduction processing.

The display unit 25 is a display unit such as an LCD and a CRT which forms a display screen related to processing such as file creation / transmission / reception / filing of speech information.

The interface 30 is a unit for exchanging data between the bus BS and the FD drive 31 or the CD-ROM drive 32. The FD drive 31 performs an operation of reading and writing data by mounting a detachable FD 31a (recording medium). CD-ROM drive 3
Reference numeral 2 carries out an operation of reading data by mounting a removable CD-ROM 32a (recording medium).

If information such as voice data, a phoneme table, and an application program is stored in the FD 31a or the CD-ROM 32a, the stored contents of the voice color storage unit 21, the application storage unit 22, and the like can be updated. .

The communication section 20 is connected to the communication line LN.
Communication with an external device is performed via the communication line LN.

Next, the timbre storage unit 21 will be described in detail.
FIG. 9 is a diagram showing an example of a memory configuration of the voice color unit 211 of the voice color storage unit 21. The voice part 211, as shown in FIG.
Selection No. Voice data V corresponding to 1, 2,.
D1, VD2,... As the types of voices, voices of men, women, children, adults, husky, etc. are adopted. The voice color data VD1, VD2,... Respectively have pitch reference data PB indicating a reference of voice pitch.
1, PB2... Are included.

The voice data includes sound parameters for each synthesis unit (for example, CVC). As the acoustic parameters, LSP parameters, cepstrum, and one-pitch waveform data are suitable.

Next, the phoneme unit 212 will be described. FIG. 10 is a diagram showing a memory configuration example of the phoneme unit 212 of the timbre storage unit 21, FIG. 11 is a diagram showing a memory configuration example of the voiced phoneme table 33A of the Japanese phoneme table, and FIG. FIG. 13 is a diagram illustrating a memory configuration example of the voiced phoneme table 33B, and FIG. 13 is a diagram illustrating a correspondence relationship between phonemes and phoneme codes for each language code in the phoneme unit 212.

As shown in FIG. 10, the phoneme unit 212
It is a memory that stores a phoneme table 212A in which phoneme groups are made to correspond to language codes of languages such as English, German, and Japanese, and a phoneme table 212B that shows the correspondence between phonemes and phoneme codes for each phoneme group.

The language code is a code assigned to each language, and there is a one-to-one correspondence between the language and the language code.
For example, a language code "1" is given to English, a language code "2" is given to German, and a language code "3" is given to Japanese.

A phoneme group specifies a phoneme table corresponding to each language. For example, in the case of English and German, the address ADR1 in the phoneme table 212B is determined by the phoneme group.
Is specified, in which case the Latin phoneme table is used. In the case of Japanese, the address ADR2 in the phoneme table 212B is specified, and in this case, the Japanese phoneme table is used.

More specifically, in Latin languages such as English and German, phoneme levels are used as units of speech. That is, one type of phoneme code set corresponds to characters in a plurality of languages. On the other hand, in the case of a language such as Japanese, there is a one-to-one correspondence between phoneme codes and characters.

The phoneme table 212B is tabular data indicating the correspondence between phoneme codes and phonemes. The phoneme table 212B is provided for each phoneme group. For example, a Latin (English, German) phoneme table (Latin phoneme table) has an address ADR1 in a memory, and a Japanese phoneme table (Japanese phoneme table) has an address in a memory. A
It is stored in DR2.

For example, the phoneme table (the position of the address ADR2) corresponding to the Japanese language is shown in FIGS.
As shown in (1), it is composed of a voiced phoneme table 33A and a non-voiced phoneme table 33B.

The voiced phoneme table 33 shown in FIG.
In A, a voiced phoneme code and a voiced phoneme (character:
(Represented by character codes). The voiced phoneme code is composed of 1 byte. For example, the voiced phoneme character "A" has the voiced phoneme code 03.
h (h indicates a hexadecimal number) corresponds thereto.

A phoneme with a “.” Added to the upper right of the character in the line has a meaning that the voice of the character is a muddy sound. For example, voiced phoneme codes 13h to 17h correspond to the expressions of the nasal sounds of the characters "ka" to "ko".

The unvoiced phoneme table 33 shown in FIG.
In B, the unvoiced phoneme code and the unvoiced phoneme (character: represented by a character code) are associated with each other. In this embodiment, the non-voiced phoneme code is also composed of one byte, and for example, the non-voiced phonetic character "ka"("Uka")
Corresponds to the non-voiced phoneme code A0h. For phonemes to be unvoiced, “U” is added before the character.

For example, when the language code is "3", the Japanese phoneme table at address ADR2 is used. As a result, as in the example shown in FIG. 13, the characters "A", "F", and "F" are associated with the phoneme codes 03h, 09h, and 39h, respectively.

If the language is English or German, the Latin phoneme table at address ADR1 is used. Thus, as in the example shown in FIG. 13, the English phonemes (phonemes) “a” and “i” have phoneme codes 39h and 05, respectively.
h, while the German phoneme (a phoneme) "a",
“I” is associated with phoneme codes 39h and 05h, respectively.

As described above, common phonemic codes 39h and 05h are assigned to, for example, phonemes "a" and "i" common to English and German, as in the example shown in FIG.

Next, the DB 26 will be described. FIG.
FIG. 4 is a diagram illustrating an example of a memory configuration of a DB 26 of the terminal device 2.

As shown in FIG. 14, the DB 26 stores file information including speech information in association with the files A, D,. For example, the file information of the file A is information that has already been received from the host device 1, and includes speech information (header information HDRA and speech information PRSA) and image information I.
The MGA and the program information PROA are stored in association with each other. Similarly, the file information of the file D is stored in association with speech information (header information HDRD and speech information PRSD), image information IMGD, and program information PROD. Since the Internet is taken as an example of the system, in the file information A, D..., The program information PROA, PROD... Is information described in HTML language, and creates a homepage or the like.

Next, speech synthesis will be described. FIG.
FIG. 4 is a block diagram conceptually illustrating a sound reproduction process according to the embodiment.

The sound reproduction process is performed by the CPU 241 of the control unit 24.
This is the operation performed by That is, the CPU 241
Input utterance information sequentially and perform prosodic pattern development processing PR
1 and a composite waveform generation process PR2 to generate composite waveform data.

The prosody pattern development processing PR1 is executed by inputting the utterance information of the file information received from the host device 1 or the utterance information of the file information designated to be read from the DB 26, and executing the time difference data in the utterance event PE. A continuous prosody pattern is developed on the time axis from DT, voice pitch PT, and voice strength VL. The utterance event P
Since E has three types of event patterns as described above, the pitch and loudness of the voice are specified with a time difference independent of the phoneme.

In the timbre storage unit 21, the host device 1
Header information of the received file information, or D
The voice data is selected based on the phoneme group PG, voice color designation data VP, and pitch reference data PB specified by the header information of the file information stored in B26, and pitch shift data for determining the pitch value is supplied to the synthetic waveform generation process PR2. I do. The time difference, pitch, and velocity are respectively time resolution TD, pitch reference data PB, and volume reference data VB.
Is determined as a relative value with respect to.

The synthetic waveform generation processing PR2 is performed by using the phoneme code P
Based on H and the time difference data DT, a sequence and duration of each phoneme are obtained, and a process of expanding / contracting the length of the acoustic parameter of the corresponding synthesis unit from the timbre data based on the phoneme sequence is executed.

Thereafter, in the synthetic waveform generation processing PR2, voice synthesis is performed based on the acoustic parameters and the temporally continuous pitch and velocity patterns obtained by the prosody pattern development processing PR1 to obtain synthetic waveform data.

The actual physical pitch frequency is determined by the pattern obtained in the prosody pattern development process PR1 and the shift data.

The synthesized waveform data is converted from digital data to analog data by a D / A converter 34 not shown in FIG.

Next, the operation will be described.

First, file transfer will be described. FIG. 16 is a flowchart illustrating a file transfer process according to this embodiment, and FIGS. 17 and 18 are flowcharts illustrating a reproduction process according to this embodiment. FIG. 19 to FIG. 22 are diagrams showing state transitions according to operations on the display screen during the reproduction process.

In this file transfer, a process is executed in which the terminal device 2 downloads desired file information from the host device 1 and reproduces audio and images.

Specifically, at the time of communication between the host device 1 and the terminal device 2, first, in the terminal device 2, a desired file is selected by a key operation of the key input unit 29 (step T1). In the file selection in step T1, a list of selectable files is transferred from the host device 1 during communication, and the list is displayed on the display unit 30 on the terminal device 2 side.

Thereafter, transfer (download) of the file selected in step T1 is requested to the host device 1 (step T2). This request process operates according to the above-described file selection.

If any request is received from the terminal device 2, the host device 1 accepts the request (step H).
1), the content of the request is determined (step H2).

If the result of the determination is that a file transfer request has been obtained (step H3), the process proceeds to step H4 to execute the file transfer process. If not, the process proceeds to step H3. The processing shifts to other processing according to the determination result.

In the file transfer process in step H4, a process of reading a file requested by the terminal device 2 from the DB 11 and transferring the file to the terminal device 2 is executed. In this file transfer, as for the voice information, only the utterance information necessary for voice reproduction in the terminal device 2 is transferred. That is, this transfer is a file transfer with a small transfer amount that does not include voice data in the audio information.

Thereafter, the terminal device 2 completes the reception (download) of the desired file (step T).
3) The process shifts to step T4 to execute a reproduction process.

This playback process is a process for playing back audio and images based on file information downloaded from the host device 1. At the time of the reproduction process, if there is an event input and the event is the selection of another file (step T5), the process proceeds to step T2 and executes the above-described file transfer request again. If it is an instruction to end the operation (step T6), the process ends, or if the event is an instruction for another process, the process according to the instruction is executed.

Here, the reproduction process in step T4 will be described with reference to FIG. 15, FIG. 17, and FIG.

In this reproducing process, a reproducing operation is started from the transferred file information according to the program information. First, in step T401, image information in the file information is read, and an image (a picture of Awa Odori as an example) is displayed on the display unit 25 as shown in FIG. Since the file information includes utterance information, a narration control (hereinafter, referred to as NC) window 250 is displayed as shown in FIG.

The NC window 250 has a stop button 2
51, a play button 252, a pause button 253, and a fast forward button 254, and the display position can be arbitrarily moved by operating the key input unit 29.

A play button 252 is a software switch for instructing narration (synthesized voice obtained by generating a voice waveform based on speech information), and a fast-forward button 254 is an operation for fast-forwarding a narration reproduction position on an address. Is a soft switch for instructing the operation.

The stop button 251 is a software switch for instructing the stop of the narration playback or the stop of the fast-forward operation, which operates in response to the operation of the play button 252 or the fast-forward button 254.

The pause button 253 is a software switch for instructing the operation of temporarily stopping the narration reproduction position on the address during the narration reproduction.

In the following step T402, the utterance information in the file information is read and analyzed. In this case, first, the voice color designation data VP of the header information in the utterance information is referred to, and it is determined whether or not voice color designation is performed based on the voice color designation data VP (step T403).

If it is determined that there is a voice designation, the process proceeds to step T404. On the other hand, if it is determined that there is no voice designation, the process proceeds to step T406. I do.

In step T404, first, the voice specified by the voice specification data VP is retrieved from the voice section 211 of the voice storage section 21, and it is determined whether or not the voice is prepared in the voice section 211.

If the result of the determination that the designated voice tone is prepared is obtained, the process proceeds to step T
The process proceeds to step T4, and if it is determined that the voice designation is not prepared, the process proceeds to step T4.
Shift to 06.

In step T405, the timbre prepared in the timbre storage unit 21 is set as a timbre to be used for voice reproduction. Thereafter, the processing shifts to Step T407.

In step T406, the voice information is not specified in the header information, or the specified voice is not prepared in the voice color part 211. Therefore, the pitch reference data PB1 and PB1 are further determined based on the pitch reference data PB of the header information.
2 are determined to be close to the reference value, and the voice color corresponding to the latest pitch reference is set as the voice color used for voice reproduction. Thereafter, the processing shifts to Step T407.

Subsequently, in step T407, a process of setting the pitch of the voice at the time of speech synthesis by the key input unit 29 is performed. Note that this setting is optional and need not be set (in this case, the pitch reference of the voice part 211 is used). However, if it is set, the pitch reference data in the voice data is used instead of the pitch reference data in the voice data. The set value is adopted as the reference value.

Then, when the processing shifts to step T408, it waits for an event input. The events to be input include input of each button of the NC window 250, designation of another file, and end instruction.

For example, as shown in FIG. 20, when the cursor (not shown) on the display screen is moved to the position indicated by X1 and the play button 252 is operated, the processing is as follows.
The process shifts to step T410 to execute the speech synthesis process (corresponding to the narration playback process) described with reference to FIG. In the example of FIG. 20, a narration is reproduced from the speaker 23 in accordance with the picture of the Awa dance displayed on the display unit 25, such as "Awa dance in Tokushima is a world-famous dance."

If there is a deviation in the pitch reference between the utterance information and the timbre data during speech synthesis, pitch shift data indicating the shift amount is supplied from the timbre storage unit 21 to the synthesized waveform generation processing PR2. Is done. In the synthetic waveform generation processing PR2, the pitch reference is changed based on the pitch shift data. For this reason, the pitch of the voice changes to match the pitch of the voice on the timbre side.

The pitch shift will be specifically described. For example, using the average pitch frequency as a pitch reference, the average pitch frequency of speech information is 200 [Hz],
When the average pitch frequency of voice data is 230 [Hz], the voice pitch at the time of voice synthesis is changed to 230/20 as a whole.
Speech synthesis is performed at 0 times. As a result, it is possible to synthesize a voice having a voice pitch suitable for the voice color data, and the quality of the voice is improved.

It is to be noted that another expression such as a cycle in which the pitch reference is formed by frequency may be used.

As soon as speech synthesis starts in step T410, the process returns to step T408 and waits for the next event input.

In step T410, the sound reproduction is started, and at the stage where the narration is reproduced until "Awa Odori wa Tokushima", the pause button 253 at the position indicated by X2 is pressed as shown in FIG. If the operation has been performed (step T413), the process proceeds to step T414 to pause the reproduction of the narration at the reproduction position of “,”.

Thereafter, the process again shifts to step T408 to wait for the next event input. When the pause button 253 is operated or the play button 251 is operated again, the event input is performed in step T409. The narration is determined to be an event input for sound reproduction, and in the subsequent step T410, the narration is reproduced from the position following the pause position. That is, as shown in FIG.
The narration is played as "This is a world-famous dance."

[0209] The stop button 2
If 51 has been operated (step T411), the process proceeds to step T412 to stop the narration and return the next playback position to the beginning of the narration even if the narration is being played.

When the narration is reproduced or when the fast-forward button 254 is operated in the event waiting state (step T415), the process proceeds to step T416, and the narration being reproduced is advanced at a rapid pace.
Fast forwards the narration playback position on the memory count.

If an instruction to request other file information to the host device 1 or an instruction to end the present processing is input as an event while waiting for an event input, the processing is performed in this reproduction processing. Then, the process returns to the file transfer process (main process).

Next, file processing of the terminal device 2 will be described. FIG. 23 is a flowchart illustrating the utterance information creation processing according to this embodiment, FIG. 24 is a flowchart illustrating the new creation processing according to this embodiment, and FIG.
5 is a flowchart for explaining an interrupt reproduction process according to this embodiment, FIGS. 26 to 33 are diagrams showing a state transition of an operation screen at the time of a new creation process according to this embodiment, and FIG. 34 is a diagram according to this embodiment. It is a flowchart explaining an edit process.

[0213] This file processing includes speech information creation processing, interrupt reproduction processing, reproduction processing, and the like. The utterance information creating process includes a new creating process and an editing process.

In the speech information creating process shown in FIG. 23, first, a process is selected according to a key operation of the key input unit 29 (step S1). Then, the content of the selected process is determined, and if a determination result of new creation is obtained (step S2), the process proceeds to step S3 to execute a new creation process (see FIG. 24). If a result of the determination of editing is obtained (step S4), the process proceeds to step S5 to execute an editing process (see FIG. 29).

Then, a new creation process (step S3),
After any one of the editing processes (step S5) is completed, the process proceeds to step S6 to determine whether or not there is an end instruction. As a result, if the processing is completed, the processing is completed, and if not, the processing returns to step S1.

Next, the new creation processing will be described with reference to FIGS. In the new creation process, first, the header information and the utterance information constituting the speech information are initialized, and the creation screen used for creating the file is also initialized (step S101).

Next, a new natural sound is input using the microphone 28, or a file of the original sound information (waveform data) already registered in the original waveform storage unit 27 is opened (step S102). The original waveform is displayed on the creation screen (step S103). When a new natural voice is input, the input natural voice is analyzed, digitized by the D / A converter 34, and then displayed on the display unit 2.
5 is displayed as waveform data.

As shown in FIG. 26, the creation screen is displayed on the display unit 25 as a phoneme display window 25A, an original waveform display window 25B, a synthesized waveform display window 25C, a pitch display window 25D, a velocity display window 25E, and an original voice. It comprises a play / stop button 25F, a synthesized voice play / stop button 25G, a pitch reference setting scale 25H, and the like.

On this creation screen, the original waveform formed by voice input or file open is displayed in the original waveform display window 25B as shown in FIG.

In the following step S104, in order to set the time length of each phoneme with respect to the original waveform displayed in the original waveform display window 25B, a label for temporally dividing the phonemes is given by manual operation. For this operation, for example,
By operating the key input unit 29, the cursor on the display screen is moved into the synthesized waveform display window 25C located below the original waveform display window 25B, and a label can be assigned by specifying a label at a desired position. In this case, if an input device such as a mouse is used, the label position can be easily specified.

FIG. 27 shows a composite waveform display window 25C.
In the figure, an example in which 11 labels are given is shown. By this labeling, each label is also extended to the phonological display window 25A, the original waveform display window 25B, the pitch display window 25D, and the velocity display window 25E positioned above and below the synthesized waveform display window 25C. Correspondence of parameters is taken.

If the language of the input natural speech is, for example, Japanese, in a succeeding step S105, a Japanese phoneme (character) is input to the phoneme display window 25A. In this case, the key input unit 29 is provided in the same manner as the labeling.
The phoneme input is performed by manual operation using, and each phoneme is set in a space separated by a label in the phoneme display window 25A.

In FIG. 28, on the time axis, "Y"
"B", "U", "I", "De", "U"
An example is shown in which phonemes are input in the order of “S”, “,”, and “F”. Of the input phonemes, "Ushi" and "U
"" Indicates a non-voiced phoneme, and the other indicates a voiced phoneme.

In the following step S106, pitch analysis of the original waveform displayed on the original waveform display window 25B is performed.

In FIG. 29, the pitch pattern W1 of the original waveform displayed on the pitch display window 25D after the pitch analysis has been performed.
(Solid line in FIG. 29) and pitch pattern W of the synthesized waveform
2 (in FIG. 29, a broken line portion connected to a label position by a circle) is shown in a different color, for example.

At the following step S107, pitch adjustment is performed. This pitch adjustment includes adding a pitch label,
Included are operations of adding, moving (time axis direction and level direction), deleting, etc., with the movement and deletion in the time axis direction, respectively.

Specifically, the pitch adjustment is to manually set the pitch pattern W2 of the synthesized waveform by manually referring to the pitch pattern of the original waveform by the user. The pitch pattern W1 is fixed. The pitch pattern W2 of the synthesized waveform is specified by the point pitch of the label position on the time axis, and is obtained by interpolating linearly between labels having a time difference independent of the time division of phonemes.

In the adjustment of the pitch label, as shown in FIG. 30, it is possible to add a label between labels separating phonemes. Additional operation is the pitch display window 2
The label position may be directly specified in the 5D with a mouse or the like as indicated by D1, D3, D4, and D5. Since the pitch newly added in this way is connected to the adjacent pitch by a straight line, a desired pitch change can be given in one phoneme, and it is easy to process into the ideal prosody.

In the movement operation, the destination of the pitch label may be directly designated in the pitch display window 25D by a mouse or the like as indicated by D2. Even in the movement of the pitch label, the pitch is connected to the adjacent pitch by a straight line, so that a desired pitch change can be given in one phoneme, and it is easy to process into the ideal prosody.

[0230] Even if the pitch is deleted from the pitch label, the pitch is connected by a straight line to the adjacent pitch excluding the deleted pitch, so that a desired pitch change can be given in one phoneme. It is easy to process to the ideal prosody.

In this case, an utterance event PE1 is set.

In the following step S108, a composite waveform at the stage adjusted to the pitch is generated, and is displayed and formed on the composite waveform display window 25C, for example, as shown in FIG. At this time, since the velocity is not set, a plain velocity is displayed in the velocity display window 25E as shown in FIG.

At the stage where the synthesized waveform is displayed in step S108, the original voice and the synthesized voice can be compared and reproduced. At this stage, the type of voice to be synthesized is a default voice.

When the original sound is reproduced, the original sound reproduction / stop button 25F is operated, and when the reproduction is stopped, the original sound reproduction / stop button 25F is operated again. When the synthesized voice is reproduced, the synthesized voice reproduction / stop button 25G is operated, and when the reproduction is stopped, the synthesized voice reproduction / stop button 25G is operated again.

The above reproduction processing is executed as an interruption reproduction processing in a new creation processing or an editing processing described later. The details are the operation shown in FIG. That is,
In step S201, first, it is determined whether the reproduction target is the original sound or the synthesized sound in response to the operation of the original sound reproduction / stop button 25F or the synthesized sound reproduction / stop button 25G.

If the result of the determination is that of the original voice (step S202), the process proceeds to step S20.
3 and reproduce and output the original sound with the original waveform.
If a determination result of a synthesized voice is obtained (step S202), the process proceeds to step S204, where the synthesized voice is reproduced and output using the synthesized waveform. Thereafter, the process returns to the operation at the time of interruption of the new creation process.

Now, returning to the description of the new creation processing, in step S109, the velocity representing the volume of the phoneme is manually adjusted. This velocity adjustment is performed within a predetermined range of steps (for example, 16 steps) as shown in FIG.

As with the above-described pitch adjustment, this velocity adjustment does not depend on the time division between phonemes, but is finer than the time difference between phonemes on the time axis. Can be given.

For example, as shown in FIG. 33, the velocities E1 of the time division of the phoneme "ka" in the velocity display window 25E shown in FIG.
2 can be subdivided. This velocity adjustment is also set by operating the key input unit 29 in the velocity display window 25E as in the case of the pitch adjustment.

When the reproduction of the synthesized speech is operated again after the velocity adjustment, the intensity of the speech changes with a time difference that does not depend on the time difference between the phonemes, and the intonation can be added to the speech as compared with the plain velocity state. . The time division of the voice intensity (velocity) may be synchronized with the time division of the pitch label obtained by the pitch adjustment.

Thereafter, the end operation of the new creation process is determined in step S110, and if the end operation is executed, the process shifts to step S117 to execute the new filing process. In the new filing process, a file name is input, and a newly created file is stored in the DB 26 in correspondence with the file name. If the file name is "A", the speech information is stored in the form of header information HDRA and speech information PRSA as shown in FIG.

There is no ending operation in step S110, and the velocity is changed (step S111), the pitch is changed (step S112), the phoneme is changed (step S113), the label is changed (step S114), and the timbre setting is changed (step S114). If any one of the operations in step S115) is determined, the process shifts to a process according to each change request.

That is, if the velocity is changed (step S111), the process returns to step S109 to change the velocity value in phoneme units according to the manual operation. If the pitch is changed (step S11)
2), the process returns to step S107 to change (including addition and deletion) the pitch value in label units according to the manual operation.

If the phoneme is changed (step S1)
13), the process returns to step S105, and changes the phoneme according to the manual operation. If the label is to be changed (step S114), the process returns to step S104 to change the label according to a manual operation. In addition,
In the label change and the pitch change, the pitch pattern W2 of the synthesized waveform changes according to the pitch interval after the change.

If the tone setting is to be changed (step S115), the process proceeds to step S116 to change the setting to the desired tone type in accordance with the manual operation. When the synthesized voice is reproduced again by the change of the voice setting, the characteristics of the voice are changed. For example, even if the natural voice is a male voice, the voice can be changed to a female voice by changing the voice.

It should be noted that as long as no end operation is detected after the process of step S109 and no parameter change operation is detected, the process of returning from step S115 to step S110 is repeatedly executed.

Further, the change of each parameter only executes the change of the parameter designated to be changed. For example, when the processing in step S104 ends due to a label change, the processing in steps S105 to S109 is passed through, and the processing is restarted from step S110.

Next, the editing process will be described with reference to FIG. This editing process is a process for operating addition, change, and deletion of a parameter to a file that has already been created, and basically executes the same process as the change step of the new creation process.

That is, in this editing processing, first, in step S301, the file to be edited is stored in the DB.
A selection operation is performed with reference to the 26 file list. Then, a creation screen similar to the above-described new creation processing is displayed on the display unit 2.
5 is formed.

In this editing processing, the original synthesized waveform to be edited is now treated as the original waveform, and the original waveform is displayed and formed on the original waveform display window 25B.

At the following step S302, an editing operation is input. This input corresponds to the change operation of the above-described new creation processing.

By this editing operation, the label is changed (step S303), the phoneme is changed (step S305), the pitch is changed (step S307), the velocity is changed (step S309), and the tone setting is changed (step S3).
When any one of the operations in 11) is determined, the processing shifts to a processing corresponding to each change request.

That is, if the label is to be changed (step S303), the process shifts to step S304 to change the label according to a manual operation. It should be noted that the pitch pattern W2 of the synthesized waveform changes in accordance with the change in the label change and the pitch change in the editing process as well.

If the phoneme is to be changed (step S3
05), the process proceeds to step S306, and the phoneme is changed according to the manual operation. If the pitch is to be changed (step S307), the process proceeds to step S308 to change (including addition or deletion) the pitch value in label units in accordance with the manual operation.

If the velocity is to be changed (step S309), the process shifts to step S310 to change the velocity value in phoneme units according to the manual operation. If the voice setting is changed (step S31)
1), the process proceeds to step S312, and the setting is changed to a desired tone type according to the manual operation.

If the ending operation has been performed in the editing operation in step S302, the process shifts to step S313 to confirm the end of the operation, and further proceeds to step S31.
Move to 4. In this step S314, edit filing processing is executed, and at this time, registration as a new file or overwriting to an existing file can be arbitrarily selected.

After each parameter is changed, the process returns to step S302, and the parameter changing operation can be continued.

Next, file registration will be described. FIG. 35 is a flowchart for explaining a file registration process according to this embodiment.

In the file registration, a process is executed in which the terminal device 2 uploads desired file information to the host device 1 and registers utterance information.

Specifically, at the time of communication between the host device 1 and the terminal device 2, first, in the terminal device 2, a created file is selected by a key operation of the key input unit 29 (step T11). In the file selection in step T11, the files stored in the DB 26 may be displayed in a list and selected.

Thereafter, transfer (upload) of the file selected in step T11 is requested to the host device 1 (step T12). This request process operates according to the above-described file selection.

When any request is received from the terminal device 2, the host device 1 accepts the request (step H1 similar to the above-described file transfer) and determines the content of the request (similar to the above-described file transfer). Step H
2).

If the result of the determination is that a file registration request has been obtained (step H5), the processing shifts to step H6, where the terminal device 2 responds to the terminal device 2 with permission to accept the file registration request. If the result of the determination that the file registration request is not obtained in step H5,
The processing shifts to other processing according to the determination result.

In the terminal device 2, when the reception permission is obtained from the host device 1, the file information to be registered is stored in the DB 2.
6 and a process of transferring the file to the host device 1 is executed.

When the reception (download) of the file for which registration has been requested is completed (step H7), the host device 1 shifts the processing to step H8 and executes the registration processing in the DB 11.

As described above, when the file registration in the host device 1 is completed, the file registered in the DB 11 can be accessed from another terminal device connected to the communication network NET. Is applied.

As described above, according to the present embodiment, file information including utterance information is transferred from the host device 1 to the terminal device 2, and the voice volume and the voice volume independent of the phoneme are transferred to the terminal device 2. Since a prosody pattern that is temporally continuous in height is developed, and a voice waveform is generated based on the prosody pattern and voice data selected by information indicating the type of voice in the utterance information, a specific waveform is generated. The voice can be reproduced with the optimum voice directly specified from a plurality of types of voices without being limited to voices, and there is no shift in the voice pitch pattern during waveform synthesis. It is possible to maintain high quality of speech synthesis by obtaining an optimal correspondence without fixing the correspondence between the voice and the timbre information.

Also, since the reference of the voice pitch of the utterance information is shifted according to the reference of the voice pitch of the timbre section 211 at the time of voice reproduction, the pitch of each voice is independent of the time division of the phoneme. Since the pitch is relatively changed according to the shifted voice pitch criterion, and the voice pitch criterion approaches the timbre side, it is possible to further improve the voice quality.

Further, since the reference of the voice pitch of the utterance information is shifted by an arbitrary reference of the voice pitch during voice reproduction, the pitch of each voice can be shifted regardless of the time division of the phoneme. It changes relatively according to the reference of the pitch of the voice, and therefore, it is possible to process the timbre such as approaching the intended voice quality according to the shift amount.

Further, since the reference of the voice pitch is set to the average frequency, the maximum frequency, or the minimum frequency of the voice pitch, the reference of the voice pitch can be easily set.

Also, since the voice data is read from the recording medium in the terminal device 2 and stored in the voice section 211, a variation can be given to the type of voice through the recording medium, and the optimum voice can be applied at the time of voice reproduction. It is possible to do.

Further, since the terminal device 2 receives timbre data from an external device via the communication line LN and stores the timbre data in the timbre portion 211, the type of timbre is given variation through the communication line LN. This makes it possible to apply an optimal voice tone during voice reproduction.

In the terminal device 2, based on the input natural speech, one or both of the loudness and the loudness of the voice are not dependent on the time difference between phonemes, and are relative. Since the utterance information is discretely generated so as to have a level, the utterance information is transferred to the host device 1 and registered in the DB 11, so that the loudness and the pitch of the voice at any time independent of the time difference between the phonemes. It is possible to give

Also, when the utterance information is created, the utterance information is created by including the criterion of the voice pitch in the utterance information, so that the utterance information can be given a criterion of the voice pitch. .

Further, since each parameter is arbitrarily changed at the time of creating the utterance information, it is possible to change the information for improving the voice quality.

Next, a modified example of the above embodiment will be described.

In the first modification, since the new creation processing of the above-described embodiment is modified, the new creation processing will be described below.

FIG. 36 is a block diagram showing a main part according to a first modification of the present embodiment. The device according to this modification has a configuration in which a speech recognition unit 35 is added to the above-described terminal device 2 (see FIG. 8), and is coupled to a bus BS.

[0279] The voice recognition unit 35 is connected to the microphone 2
The voice recognition is performed based on the natural voice input by using the control unit 8, and the recognition result is supplied to the control unit 24. Control unit 24
Then, a process of converting the supplied recognition result into a character code (corresponding to the phoneme table described above) is executed.

Next, main operations of the modification will be described. FIG. 37 is a flowchart illustrating a new creation process according to the first modification.

In the new creation processing according to the first modification, as in step S101 (see FIG. 24), first, the header information and the utterance information constituting the utterance information are initialized and used for file creation. The creation screen to be created is also initialized (step S501).

Next, when a new natural voice is input using the microphone 28 (step S502), the original waveform is displayed in the original waveform display window 25B of the creation screen (step S503).

Note that the creation screen is the same as that of the above-described embodiment (see FIG. 17). On the display unit 25, the phoneme display window 25A, the original waveform display window 25B, the synthesized waveform display window 25C, the pitch display window 25D, It comprises a velocity display window 25E, an original sound reproduction / stop button 25F, a synthesized sound reproduction / stop button 25G, a pitch reference setting scale 25H, and the like.

In this modified example, the speech recognition unit 35 performs speech recognition based on the original waveform by speech input, and acquires phonemes collectively (step S503).

In the following step S504, phonemes are automatically assigned to the phoneme display window 25A based on the obtained phonemes and the original waveform, and a label is given at that time. In this case, the phoneme name (character) and the time interval (range on the time axis) between the phoneme are obtained.

Further, in step S505, the pitch (including the pitch reference) and velocity are extracted from the original waveform, and in step S506, the pitch and velocity extracted corresponding to the phoneme are displayed in the pitch display window 25D and the velocity display window 25E, respectively. Is displayed. In addition, there is a method of setting the pitch reference to, for example, twice the minimum value of the pitch frequency.

Thereafter, a speech waveform is generated based on each parameter and default timbre data, and is displayed on the synthesized waveform display window 25C (step S50).
7).

Thereafter, the end operation of the new creation process is determined in step S508, and if the end operation is executed, the process shifts to step S513 to execute the new filing process. In the new filing process, a file name is input, and a newly created file is stored in the DB 26 in correspondence with the file name.

In step S508, there is no ending operation, and if it is determined that an operation to change any of the parameters of velocity, pitch, phoneme, and label is determined (step S5).
09), the process proceeds to step S510, and a change process is performed on the parameter to be changed.

[0290] If it is determined in step S511 that the voice setting has been changed, the process proceeds to step S512, in which the voice setting is changed.

Note that while the end operation is not detected in step S508 and the parameter changing operation is not detected in step S509 or step S511, the processing of steps S508, S509, and S512 is repeatedly executed.

As in the first modification, even after the natural speech is input, the parameters up to the synthesized waveform are automatically obtained and then individual parameters are changed, as in the above-described embodiment. It is possible to realize practical speech synthesis that can maintain sound reproduction with high quality.

Also, as a second modification, the amplitude value of the original waveform is compared with the amplitude pattern of the synthesized waveform after the speech is synthesized once, and the velocity value is optimized so that the synthesized waveform matches the amplitude of the original waveform. Well, the quality of the voice can be further improved.

As a third modification, when the voice data specified by the utterance information does not exist in the voice part, a voice having a feature (voice attribute) similar to the feature (voice attribute) of the speech information is selected from the voice part. Voice synthesis may be performed.

Hereinafter, Modification 3 will be described in detail. FIG.
FIG. 8 is a diagram showing a configuration example of header information according to the third modification;
9 is a diagram showing a configuration example of a voice attribute in the header information shown in FIG. 38, FIG. 40 is a diagram showing a configuration example of a voice portion according to Modification 3, and FIG. 41 is a voice in the voice portion shown in FIG. FIG. 4 is a diagram illustrating a configuration example of an attribute.

In the third modification, as shown in FIGS. 38 and 40, voice attribute information having a common format is prepared in the header information in the speech information and the voice portion 213.

[0297] To the header information HDRX in the speech information, voice attribute information AT is added as a new parameter to the header information applied to the above-described embodiment.

As shown in FIG. 39, the voice attribute information AT has a structure in which gender data SX, age data AG, pitch reference PB, clarity CL, and naturalness NT are associated with each other.

Similarly, the voice color part 213 has the voice color attribute information ATn corresponding to the voice color data as a new parameter with respect to the voice color part 211 applied to the above embodiment.
(N is a natural number).

As shown in FIG. 41, the voice attribute information ATn has a structure in which gender data SXn, age data AGn, pitch reference PBn, clarity CLn, and naturalness NTn are associated with each other.

Each item of voice attribute is voice attribute information AT,
Gender: -1/1 (male / female) Age: 0 Pitch reference (average pitch): 100 to 300 [Hz] Clarity: 1 to 10 (increasing clarity as frequency increases) ) Naturalness: defined by 1 to 10 (naturalness increases as frequency increases). Note that the clarity and naturalness indicate a sensory level.

Next, main operations of the third modification will be described. FIG. 42 is a flowchart for explaining main processing of the new creation processing according to the third modification, and FIG.
Is a flowchart for explaining a reproduction process by the.

[0303] Since the entire flow of the new creation processing is the same as that of the new creation processing according to the above-described embodiment (see Fig. 24), only different parts will be described.

In the processing flow shown in FIG. 24, when the new creation is completed, the processing proceeds from step S110 to step S110.
Then, the process proceeds to step S118, and in this modified example 3, as shown in FIG. 42, the process proceeds to step S118 to execute voice attribute setting. After this, the processing executes the filing processing in step S117.

In step S118, the above-described voice attribute information AT is created and incorporated into the header information HDRX. Here, as an example, gender: 1 (female) Age: 25 (years) Pitch reference (average pitch): 200 [Hz] Clarity: 5 (normal) Naturalness: 5 (normal) is set in the voice attribute information AT. Shall be performed.

Next, the reproducing process will be described. Before this description, an example will be given of the contents of each item of the voice attribute information ATn of the voice part 213.

In the case of the voice attribute information AT1, for example, gender: -1 (male) Age: 35 (years) Pitch reference (average pitch): 140 [Hz] Clarity: 7 (slightly higher) Naturalness: 5 ( Normal).

In the case of the voice attribute information AT2, for example, sex: 1 (female) Age: 20 (years) Pitch reference (average pitch): 200 [Hz] Clarity: 5 (normal) Naturalness: 5 ( Normal).

The reproduction process shown in FIG. 43 has the same general flow as the reproduction process according to the above-described embodiment (see FIGS. 17 and 18), and therefore only different parts will be described.

[0310] If it is determined in step S402 that there is no designated voice tone, the process proceeds to step S407. In this step S407, a process of collating the voice attribute information AT of the utterance information with each voice attribute information ATn stored in the voice part 213 is executed.

In this collation, a difference between the values of the items to be collated is taken, weighted and squared, and the result of each item is added (Euclidean distance), or a weighted addition of the absolute value, etc. There is a way.

[0312] For example, a case where a method of calculating the Euclidean distance (DSn) is applied will be described. The weighting used at this time is, for example, gender: 20 age: 1 pitch reference (average pitch): 1 clarity: 5 naturalness: 3

Therefore, in the comparison between the voice attribute information AT and AT1, DS1 = (− 1−1) * 20) ² + ((35−25) * 1) ² + ((140−200) * 1) ² + ((7−5) * 5) ² + ((5−5) * 3) ² = 720, and in the comparison between the voice attribute information AT and AT2, DS2 = ((1-1) * 20) ² + ((20-25) * 1) ² + ((230-200) * 1) ² + ((4-5) * 5) ² + ((7-5) * 3) ² = 986.

Therefore, in step S408,
DS1 <DS2, and the timbre data VD1 stored corresponding to the short-distance timbre attribute information AT1 is selected as the type of timbre having the highest similarity of timbre attributes.

In the third modification, the timbre is selected by the timbre attribute after directly specifying the timbre type. However, the similarity is determined by using only the timbre attribute without directly specifying the timbre type. May be selected from the voice data.

According to the third modification, a prosody pattern that is temporally continuous with a voice volume and a voice pitch that does not depend on a phoneme is developed, and the prosody pattern and information indicating the attribute of the voice color in the speech information are used. Since the voice waveform is generated based on the voice data selected by the similarity, the voice can be reproduced in the voice having the highest similarity without using an inappropriate voice, and the voice is generated when the voice waveform is generated. There is no shift in the height pattern, which allows audio to be reproduced with high quality.

Further, a temporally continuous prosody pattern is developed based on the volume and pitch of the voice independent of the phoneme, and the prosody pattern and the timbre selected by the information indicating the type and attribute of the timbre in the speech information Since the voice waveform is generated based on the data, even if there is no directly specified voice, the voice with the highest similarity can be reproduced without using an inappropriate voice, and the voice waveform is generated. Occasionally, there is no shift in the pitch pattern of the voice, which makes it possible to reproduce the voice with high quality.

Next, a fourth modification will be described. The fourth modification is a variation of the control event used in the above-described embodiment.

The control event CE according to the fourth modification will be described in detail. FIG. 44 is a diagram illustrating a configuration example of a control event according to the fourth modification.

In the modification 4, a pause event CE3 and a completion event CE4 are newly added to the control event CE.

The pause event CE4 has a structure in which the identification information C3 and the pause event data PSE are associated with each other, and is an event for temporarily pausing narration reproduction at an arbitrary time.

That is, this pause event CE3 is
Like the other control events CE1, CE2, and CE4, this event can be incorporated into the utterance data, and the narration playback is paused when this event appears. The release of the pause state is performed in synchronization with an operation based on other information (such as image display).

[0323] The identification information C3 added to the head of the pause event CE3 indicates a pause that is a type of control event.

[0324] The completion event CE4 has a structure in which the identification information C4 and the completion event data COE are associated with each other, and informs an external upper application or the like of how far the narration has been reproduced.

That is, the completion event CE4 is an event that can be incorporated into the utterance data, like the other control events CE1, CE2, and CE3, and notifies the completion of the narration reproduction to an external upper application by the appearance of this event.

[0326] The identification information C4 added to the head of the completion event CE4 indicates completion, which is the type of control event.

[0327] Here, the reproduction process of Modification 4 will be described. FIG. 45 is a flowchart for explaining the reproduction process according to the fourth modification, and FIGS. 46 to 48 are diagrams showing the state transition of the display screen during the reproduction process.

In the description, in the program information, the image information is programmed in the step of displaying the first image and the second image in this order, and the utterance information reproduces the first narration when the display of the first image starts. Thereafter, the reproduction of the second narration is weighted by the completion event and the pause event, and the second narration is reproduced together with the start of the display of the second image, so that the image and the narration are synchronized.

In the fourth modification, as the reproduction is started, first, the first image (for example, a Japanese picture) is displayed as shown in FIG. 46 based on the image information in the file information (step T501). Subsequently, the utterance information in the file information is analyzed (step T502).

As shown in FIG. 46, based on the analysis result, the speaker 23 outputs the first message such as "Japan is an island country".
The reproduction of the narration is started (step T50)
3). Also in this case, similarly to the above-described embodiment, N
The C window 250 is displayed on the display unit 25 together with the image.

In the fourth modification, after the reproduction of the first narration is started, a completion event indicating the completion of the first narration or the above-described event (operation of NC window 250, request instruction of other file information, this processing) (Step T504, Step T504)
506)

[0332] If an event input is detected in step T506, the process proceeds to step T507. If it is determined in step T507 that the event is an event input for narration playback by operating the NC window 250 as in the above-described embodiment, the process further proceeds to step T508 to perform playback, stop, pause, or fast forward. Execute control. If the event is not an event input for narration playback, the process exits from this playback process and returns to the file transfer process (main process) shown in FIG.

If the end of the reproduction of the first narration is detected by the completion event in the utterance event of the speech information (step T504), a pause event following this completion event is detected (step T504).
505), at this timing, the second image (a picture of Mt. Fuji, for example) is displayed on the display unit 25 as shown in FIG. 47 (step T509).

At the timing of the start of the display of the second image, the reproduction of the second narration is started based on the utterance information (utterance data) as in "Mt. Fuji is the highest mountain in Japan" (see FIG. 48). The reproduction of the second image and the reproduction of the second narration are synchronized (steps T502 and T503).

[0335] In the above-described modification 4, an example in which the completion event and the pause event are used as a pair has been described. However, each event may be used independently.

That is, during the reproduction of the narration, the appearance of the completion event is passed to the upper application in the reproduction processing as the base point for knowing the reproduction position of the narration.
The higher-level application may synchronize with another operation. In this case, the completion event may be incorporated at an arbitrary point on the time axis of the narration playback (a point where synchronization with other operations is to be performed).

In the pause event, for example, by incorporating a pause event into the utterance data for each sentence of the narration, the release of the pause of the narration reproduction is synchronized with the operation of the key input unit 29 differently from the above-described image display. It can also be done.

As described above, according to the fourth modification, the utterance information includes the control event for synchronizing the operation based on the image information in the file information and the operation based on the audio reproduction, and the control event included in the utterance information includes the audio reproduction. , The operation is performed in synchronization with the operation of the image information in the file information, so that the expression of sound and other media can be fused to enhance the expressive power.

[0339] Note that music information or the like may be added to the file information in addition to the image information. This makes it possible to enhance the expressive power by combining expressions of music and the like in addition to the sound and the image. .

Since this control event is included in the utterance information when the utterance information is created, information for synchronizing the operation of speech synthesis with the operation based on other information can be given in the utterance information. is there.

By the way, in the above-described embodiment and each of the modified examples, the timbre data is selected by designating the voice pitch and the voice intensity that do not depend on the phoneme. Even if the pitch and strength of the voice do not depend on the phoneme, it is possible to select the optimum voice data for the speech information to be synthesized in the voice section 211 (voice section 213). At this level, audio can be reproduced with high quality.

[0342]

As described above, according to the first aspect of the present invention, file information including speech information is transferred from the first communication device to the second communication device, and the second communication device does not depend on phonemes. Since a temporally continuous prosody pattern is developed based on the loudness and pitch of the voice, and a voice waveform is generated based on the prosody pattern and voice data selected based on the utterance information, a specific voice color is generated. It is possible to reproduce the voice with a suitable voice color even if it is not limited to the above, and there is no deviation in the voice pitch pattern when generating the voice waveform, thus fixing the correspondence between the utterance information and the voice color information By obtaining the optimal correspondence without performing the above, it is possible to obtain an information communication system capable of maintaining high quality of speech synthesis.

According to the second aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device. In order to generate a speech waveform based on the prosody pattern that is temporally continuous and generate the voice waveform based on the prosody pattern and the timbre data selected by the information indicating the type of timbre in the utterance information, The voice can be reproduced with the optimum voice directly specified from a plurality of voices without limitation, and there is no shift in the voice pitch pattern when generating the voice waveform. There is an effect that an information communication system capable of maintaining a high quality of speech synthesis can be obtained by obtaining an optimum correspondence relationship without fixing a correspondence relationship between the voice and timbre information.

According to the third aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device. Since a temporally continuous prosody pattern is developed in step (1) and a voice waveform is generated based on the prosody pattern and voice data selected by similarity based on information indicating a voice attribute in the utterance information, The voice can be reproduced with the voice having the highest similarity without using any vocal timbre, and there is no shift in the voice pitch pattern when the voice waveform is generated. There is an effect that an information communication system capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence.

According to the fourth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device. Since a temporally continuous prosody pattern is developed in step (1), a speech waveform is generated based on the prosody pattern and voice data selected by information indicating the type and attribute of the voice in the utterance information. The voice can be reproduced with the highest similarity without using an unsuitable voice even if the voice specified in the above is not used, and there is no shift in the voice pitch pattern when generating the voice waveform. In addition, it is possible to obtain an information communication system capable of maintaining a high quality of speech synthesis by obtaining an optimum correspondence without fixing the correspondence between the utterance information and the voice color information.

According to the fifth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. Since the prosody pattern is developed and the voice waveform is generated based on the prosody pattern and the voice data selected based on the utterance information, the voice can be reproduced with a suitable voice without being limited to a specific voice. In addition, there is no shift in the pitch pattern of the voice at the time of generation of the voice waveform, and thus, it is possible to obtain the optimum correspondence without obtaining the correspondence between the utterance information and the voice color information. There is an effect that an information communication system capable of maintaining high quality of synthesis can be obtained.

[0347] According to the invention of claim 6, file information including speech information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the speech information in the file information. Is developed based on the prosody pattern and the voice data selected based on the information indicating the type of voice in the utterance information, so that the voice is not limited to a specific voice. The voice can be reproduced with the optimal voice directly specified from the plurality of voices, and the voice pitch pattern does not shift when the voice waveform is generated. There is an effect that an information communication system capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence.

[0348] According to the invention of claim 7, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. The prosody pattern is developed, and the voice waveform is generated based on the prosody pattern and the voice data selected by the similarity based on the information indicating the voice attribute in the utterance information. The voice can be reproduced with the highest similarity without any voice, and the voice pitch pattern does not shift when generating the voice waveform.
As described above, an advantageous effect is obtained in that an information communication system capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence between speech information and voice color information.

According to the eighth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. The voice waveform is generated based on the prosody pattern and the voice data selected by the information indicating the type and attribute of the voice in the utterance information. Without using an inappropriate voice tone, the voice can be reproduced with the highest similarity voice without using an inappropriate voice tone, and there is no shift in the voice pitch pattern when generating a voice waveform. There is an effect that an information communication system capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence with voice color information.

According to the ninth aspect of the present invention, the third aspect, the fourth aspect,
In the invention described in any one of the above items 7 and 8, the information indicating the attribute is represented by any one of gender, age, voice pitch criterion, intelligibility, and naturalness, or two or more thereof. Because of the combination, the collation target of the attribute of the utterance information storage means and the attribute of the timbre storage means is parameterized, thereby providing an information communication system capable of easily selecting the timbre. .

According to the tenth aspect, claims 1 to 8
In the invention described in any one of the above, since the reference of the voice pitch of the utterance information storage means is shifted by the reference of the voice pitch of the timbre storage means at the time of voice reproduction, the pitch of each voice Is relatively changed according to the shifted voice pitch criterion regardless of the time segment of the phoneme, so that the voice pitch criterion approaches the timbre side, so that the voice quality can be further improved. There is an effect that a possible information communication system is obtained.

According to the eleventh aspect, claims 1 to 8
In the invention according to any one of the above, since the reference of the voice pitch of the utterance information storage means is shifted by an arbitrary reference of the voice pitch at the time of voice reproduction, the pitch of each voice is Irrespective of the time segment of the voice, the pitch relatively changes according to the reference of the pitch of the shifted voice, and therefore, the effect that an information communication system capable of processing the voice color, such as approaching the intended voice quality according to the shift amount, is obtained. Play.

According to the twelfth aspect of the present invention, in the tenth or eleventh aspect, the criterion of the voice pitch based on the first and second information is determined based on an average frequency of the voice pitch, a maximum frequency,
Alternatively, since the frequency is set to the minimum frequency, it is possible to obtain an information communication system in which the reference of the voice pitch can be easily obtained.

According to the invention of claim 13, claims 1 to 8
In the invention according to any one of the above, the voice data is read from the recording medium in the second communication device and stored in the voice storage means, so that the voice type can be varied through the recording medium, There is an effect that an information communication system capable of applying an optimal voice during reproduction is obtained.

According to the fourteenth aspect, claims 1 to 8
In the invention described in any one of the above, the voice data is received from the external device via the communication line in the second communication device, and the voice data is stored in the voice storage means. Variations can be given to the types, and an information communication system capable of applying an optimal voice at the time of voice reproduction is obtained.

According to the invention of claim 15, claims 1 to 8
In the invention described in any one of the above, the utterance information includes control information for synchronizing the operation by the other information in the file information and the operation by the audio reproduction unit, and the audio reproduction unit is included in the utterance information at the time of audio reproduction. Since it operates in synchronization with the operation of the other information in the file information according to the control information to be transmitted, the information communication system that can enhance the expressive power by combining the expression of voice and other media is provided. The effect is obtained.

According to the sixteenth aspect, in the invention according to the fifteenth aspect, the other information is image information, music information, and the like. There is an effect that an information communication system that can be strengthened is obtained.

According to the seventeenth aspect, based on the input natural speech, one or both of the voice volume and the voice pitch can be determined independently of the time difference between phonemes and relative. The utterance information is created by discretely having a vocal level, and the utterance information is transferred to the first communication device and registered in the file information storage means. There is an effect that an information processing device capable of giving a loudness and a loud voice can be obtained.

According to an eighteenth aspect of the present invention, there is provided the information processing apparatus according to the seventeenth aspect, wherein the utterance information used in the information communication system according to the tenth or eleventh aspect is created and edited. Since the first information indicating the criterion of the voice pitch is included in the utterance information and created, the information processing device capable of providing the criterion of the voice pitch in the utterance information is obtained. Play.

According to the nineteenth aspect of the present invention, in the invention of the seventeenth aspect, since the creation means includes a change means for arbitrarily changing each information, the information change for improving the voice quality can be performed. There is an effect that a possible information processing device is obtained.

According to the twentieth aspect of the present invention, when the creating means creates the utterance information, the control information is included in the utterance information, so that the information for synchronizing the operation of speech synthesis with the operation based on other information is uttered. There is an effect that an information processing device that can be given in information can be obtained.

According to the twenty-first aspect of the present invention, file information including speech information is transferred from the first communication device to the second communication device. In the step of developing a temporally continuous prosody pattern and generating a voice waveform based on the prosody pattern and voice data selected based on the utterance information, it is suitable without being limited to a specific voice. Voice reproduction can be performed in voice color, and there is no shift in voice pitch pattern when generating a voice waveform. In this way, the optimum correspondence relationship between the utterance information and the voice color information is not fixed. Thus, there is an effect that an information communication method capable of maintaining high quality of speech synthesis can be obtained.

[0363] According to the invention of claim 22, file information including speech information is transferred from the first communication device to the second communication device. In the step of developing a temporally continuous prosody pattern and generating a voice waveform based on the prosody pattern and voice data selected by information indicating the type of voice in the speech information, a specific voice The voice can be reproduced with the optimum voice directly specified from a plurality of voices without limitation, and there is no shift in the voice pitch pattern when generating the voice waveform. There is an effect that an information communication method capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimum correspondence relationship without fixing a correspondence relationship between the voice and the timbre information.

According to the twenty-third aspect, the file information including the utterance information is transferred from the first communication device to the second communication device. In this process, a temporally continuous prosody pattern is developed, and a voice waveform is generated based on the prosody pattern and voice data selected by similarity based on information indicating a voice color attribute in the utterance information. The voice can be reproduced with the voice having the highest similarity without using any vocal timbre, and there is no shift in the voice pitch pattern when the voice waveform is generated. There is an effect that an information communication method capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence.

According to the twenty-fourth aspect, the file information including the utterance information is transferred from the first communication device to the second communication device. In this process, a temporally continuous prosody pattern is developed, and a voice waveform is generated based on the prosody pattern and voice data selected by information indicating the type and attribute of the voice in the speech information. The voice can be reproduced with the highest similarity without using an unsuitable voice even if the voice specified in the above is not used, and there is no shift in the voice pitch pattern when generating the voice waveform. In addition, it is possible to obtain an information communication method capable of maintaining a high quality of speech synthesis by obtaining an optimum correspondence without fixing the correspondence between the utterance information and the voice color information.

[0366] According to the twenty-fifth aspect, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. The prosody pattern is developed, and the voice waveform is generated based on the prosody pattern and the voice data selected based on the utterance information, so that the voice can be reproduced with a suitable voice without being limited to a specific voice. In addition, there is no shift in the pitch pattern of the voice at the time of generation of the voice waveform, and thus, it is possible to obtain the optimum correspondence without obtaining the correspondence between the utterance information and the voice color information. There is an effect that an information communication method capable of maintaining high quality of synthesis can be obtained.

According to the twenty-sixth aspect, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. The prosody pattern is developed, and the voice waveform is generated based on the prosody pattern and the voice data selected by the information indicating the type of voice in the utterance information. The voice can be reproduced with the optimal voice directly specified from the plurality of voices, and the voice pitch pattern does not shift when the voice waveform is generated. There is an effect that an information communication method capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence.

[0368] According to the twenty-seventh aspect, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. The prosody pattern is developed and a voice waveform is generated based on the prosody pattern and the voice data selected by the similarity based on the information indicating the voice attribute in the utterance information. The voice can be reproduced with the highest similarity without any voice, and the voice pitch pattern does not shift when generating the voice waveform.
As described above, an effect is obtained that an information communication method capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimum correspondence relationship without fixing the correspondence relationship between speech information and voice color information.

According to the twenty-eighth aspect of the present invention, the file information including the utterance information is transferred from the first communication device to the second communication device, and the second communication device temporally continues the utterance information in the file information. Is developed based on the prosodic pattern and the voice waveform selected based on information indicating the type and attribute of the voice in the utterance information. Without using an inappropriate voice tone, the voice can be reproduced with the highest similarity voice without using an inappropriate voice tone, and there is no shift in the voice pitch pattern when generating a voice waveform. There is an effect that an information communication method capable of maintaining high quality of speech synthesis can be obtained by obtaining an optimal correspondence without fixing the correspondence with voice color information.

According to the invention of claim 29, claim 23,
24. The invention according to any one of 24, 27, and 28, wherein the information indicating the attribute is any one of gender, age, voice pitch criterion, clarity, and naturalness, or two of them. Since the above-described combination process is performed, the collation target of the attribute of the utterance information storage unit and the attribute of the timbre storage unit is parameterized, whereby an information communication method that can easily select a timbre can be obtained. It works.

According to the thirtieth aspect, the twenty-first to twenty-first aspects are described.
In the invention described in any one of the twenty-eighth and thirteenth aspects, the step of shifting the reference of the voice pitch of the utterance information storage means during the voice reproduction by the reference of the voice pitch of the timbre storage means is performed. The magnitude changes relative to the shifted voice pitch irrespective of the time segment of the phoneme,
Since the criterion of the voice pitch approaches the timbre side, there is an effect that an information communication method capable of further improving voice quality is obtained.

According to the thirty-first aspect, the twenty-first to twenty-first aspects are described.
In the invention according to any one of the twenty-sixth to thirteenth aspects, the step of shifting the reference of the voice pitch of the utterance information storage means at the time of voice reproduction by an arbitrary reference of the voice pitch is performed. Regardless of the time segment of the phoneme, it changes relatively according to the reference of the pitch of the shifted voice, so that an information communication method capable of processing the voice color, such as approaching the intended voice quality according to the shift amount, is obtained. To play.

According to the invention of claim 32, in the invention of claim 30 or 31, the criterion of the voice pitch based on the first and second information is determined based on the average frequency of the voice pitch, the maximum frequency,
Alternatively, since the frequency is set to the minimum frequency, there is an effect that an information communication method in which a reference of voice pitch is easily obtained can be obtained.

According to the thirty-third aspect, the twenty-first to twenty-first aspects are described.
28, in the second communication device, the voice data is read from the recording medium in the second communication device and stored in the voice storage unit, so that the voice type can be varied through the recording medium, There is an effect that an information communication method capable of applying an optimum voice tone during voice reproduction is obtained.

According to the thirty-fourth aspect, the twenty-first to twenty-first aspects are described.
28. In the invention according to any one of the items 28, the voice data is received from the external device via the communication line in the second communication device, and the voice data is stored in the voice memory unit. And an information communication method capable of applying an optimum voice tone at the time of voice reproduction can be obtained.

According to the invention of claim 35, claims 21 to 21
28, the utterance information includes control information for synchronizing an operation based on the other information in the file information with an operation according to the audio reproduction step, and the audio reproduction step includes adding the utterance information during the audio reproduction. An information communication method capable of enhancing the expressive power by fusing the expression of voice and other media because the operation is performed in synchronization with the operation of other information in the file information according to the included control information. Is obtained.

[0377] The information communication method according to the invention of claim 36 is characterized in that:
The invention according to claim 35, wherein the other information is image information,
Since the music information or the like is used, there is an effect that an information communication method capable of enhancing the expressive power by fusing the expression of sound with media such as images and music is obtained.

[0378] According to the invention of claim 37, based on the input natural speech, one or both of the loudness and pitch of the voice can be determined independently of the time difference between phonemes and relative. Utterance information is created by discretely arranging the utterance information so as to have a typical level, and the utterance information is transferred to the first communication device and registered in the file information storage means. There is an effect that an information processing method capable of giving a loudness and a voice pitch can be obtained.

The information processing method according to claim 38 is:
37. The information processing method according to claim 37, wherein the utterance information used in the information communication method according to claim 30 or 31 is created and edited, wherein the creation step includes: Since the step of creating the utterance information is included, an information processing method capable of providing a reference of the voice pitch in the utterance information is obtained.

According to the thirty-ninth aspect of the present invention, in the invention according to the thirty-seventh aspect, since the creation step includes a change step of arbitrarily changing each information, information capable of changing information for improving voice quality is provided. There is an effect that a processing method can be obtained.

According to the forty-ninth aspect of the present invention, the creation step includes the step of including control information in the utterance information when the utterance information is created, so that the information for synchronizing the operation of speech synthesis with the operation based on other information is uttered. There is an effect that an information processing method that can be given in information is obtained.

[Brief description of the drawings]

FIG. 1 is a configuration diagram showing one embodiment of an information communication system according to the present invention.

FIG. 2 is a diagram illustrating an example of a memory configuration of a DB of the host device according to the embodiment;

FIG. 3 is a diagram showing an example of header information in speech information according to the embodiment;

FIG. 4 is a diagram showing a configuration example of utterance information in utterance information.

FIG. 5 is a diagram illustrating a configuration example of an utterance event in utterance information.

FIG. 6 is a diagram illustrating the contents of a velocity level.

FIG. 7 is a diagram illustrating a configuration example of a control event in utterance information.

FIG. 8 is a block diagram showing one embodiment of a terminal device according to the present invention.

FIG. 9 is a diagram showing an example of a memory configuration of a timbre section of a timbre storage section according to the embodiment.

FIG. 10 is a diagram showing an example of a memory configuration of a phoneme unit of the timbre storage unit according to the embodiment.

FIG. 11 is a diagram showing an example of a memory configuration of a voiced phoneme table of a Japanese phoneme table.

FIG. 12 is a diagram showing an example of a memory configuration of a non-voiced phoneme table of a Japanese phoneme table.

FIG. 13 is a diagram illustrating the correspondence between phonemes and phoneme codes for each language code in the phoneme unit.

FIG. 14 is a diagram showing an example of a memory configuration of a DB according to this embodiment.

FIG. 15 is a block diagram conceptually illustrating a sound reproduction process according to the embodiment.

FIG. 16 is a flowchart illustrating a file transfer process according to the embodiment.

FIG. 17 is a flowchart illustrating a reproduction process according to the embodiment.

FIG. 18 is a flowchart illustrating a reproducing process according to the embodiment.

FIG. 19 is a diagram showing one state transition of a display screen at the time of reproduction processing according to the embodiment.

FIG. 20 is a diagram showing another state transition of the display screen at the time of the reproduction process according to the embodiment.

FIG. 21 is a diagram showing still another state transition of the display screen at the time of the reproduction process according to the embodiment.

FIG. 22 is a diagram showing still another state transition of the display screen at the time of the reproduction process according to the embodiment.

FIG. 23 is a flowchart illustrating utterance information creation processing according to this embodiment.

FIG. 24 is a flowchart illustrating a new creation process according to the embodiment.

FIG. 25 is a flowchart illustrating an interrupt reproduction process according to the embodiment.

FIG. 26 is a diagram showing one state transition of an operation screen at the time of a new creation process according to the embodiment.

FIG. 27 is a diagram showing another state transition of the operation screen during a new creation process according to the embodiment.

FIG. 28 is a diagram showing still another state transition of the operation screen at the time of new creation processing according to the embodiment.

FIG. 29 is a diagram showing still another state transition of the operation screen at the time of a new creation process according to the embodiment.

FIG. 30 is a diagram showing still another state transition of the operation screen at the time of new creation processing according to the embodiment.

FIG. 31 is a diagram showing still another state transition of the operation screen at the time of new creation processing according to the embodiment.

FIG. 32 is a diagram showing still another state transition of the operation screen at the time of new creation processing according to the embodiment.

FIG. 33 is a diagram showing still another state transition of the operation screen at the time of new creation processing according to the embodiment.

FIG. 34 is a flowchart illustrating an editing process according to the embodiment.

FIG. 35 is a flowchart illustrating a file registration process according to this embodiment.

FIG. 36 is a block diagram showing a main part according to a first modification of the embodiment.

FIG. 37 is a flowchart illustrating a new creation process according to a first modification of the embodiment.

FIG. 38 is a diagram showing a configuration example of header information according to a third modification of the embodiment.

FIG. 39 is a diagram showing a configuration example of a voice attribute in the header information shown in FIG. 38;

FIG. 40 is a diagram showing a configuration example of a timbre portion according to a third modification of the embodiment.

FIG. 41 is a diagram showing a configuration example of a voice attribute in a voice portion shown in FIG. 40;

FIG. 42 is a flowchart illustrating main processing of a new creation processing according to a third modification of the embodiment.

FIG. 43 is a flowchart illustrating a reproducing process according to a third modification of the embodiment.

FIG. 44 is a diagram showing a configuration example of a control event according to a fourth modification of the embodiment.

FIG. 45 is a flowchart illustrating a reproduction process according to Modification 4 of the embodiment.

FIG. 46 is a diagram showing one state transition of a display screen at the time of a reproduction process according to Modification 4 of the embodiment.

FIG. 47 is a diagram showing another state transition of the display screen at the time of the reproduction process according to Modification 4 of the embodiment.

FIG. 48 is a diagram showing still another state transition of the display screen at the time of the reproduction process according to Modification 4 of the embodiment.

[Explanation of symbols]

 Reference Signs List 1 host device 2 terminal device 10 communication unit 11 DB 12 control unit 20 communication unit 21 voice color storage unit 22 application storage unit 23 speaker 24 control unit 25 display unit 26 DB 28 microphone 29 key input unit 31a FD 32a CD-ROM 35 voice recognition Part 211 voice part 212 phoneme part 213 phoneme part 241 CPU

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁶ Identification code FI G10L 5/04 G10L 5/04 E

Claims (40)

[Claims] 1. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, The first communication device associates one or both of the loudness and loudness of the voice with the time difference, and converts one or both of the loudness and the loudness of the voice between phonemes. File information storage means for storing file information including utterance information which is discrete so as to have a relative level without depending on the time difference between the file information storage means and the file information storage means in response to a request from the second communication device And a first communication means for transferring the utterance information stored in the second communication device to the second communication device, wherein the second communication device comprises: Memorize the voice Storage means; and a second communication for requesting the first communication device to transfer the file information stored in the file information storage means, and thereafter receiving the file information transferred by the first communication means. Means, selecting means for selecting one voice data from a plurality of types of voice data stored in the voice color storage means based on speech information in the file information received by the second communication means, Developing means for developing a continuous prosody pattern in the time axis direction based on one or both of the loudness and pitch of the voice included in the utterance information, or both, and the developing means; An audio reproduction means for generating an audio waveform based on the prosodic pattern and the timbre data selected by the selection means. Temu. 2. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, The first communication device associates one or both of the loudness and loudness of the voice with the time difference and the type of the timbre, and the one of the loudness and the loudness of the voice, or File information storage means for storing file information including utterance information which is not dependent on a time difference between phonemes and has a relative level, and which responds to a request of the second communication device. And a first communication unit for transferring the file information stored in the file information storage unit to the second communication device, wherein the second communication device is provided for each unit of phoneme or the like for each voice type. Voice data representing acoustic parameters Voice data storage means for storing data, and requesting the first communication device to transfer the file information stored in the file information storage means, and thereafter transmitting the file information transferred by the first communication means. A plurality of timbres stored in the timbre storage means, the timbre data corresponding to the type of timbre in the utterance information of the file information received by the second communicator. Selecting means for selecting from among the data, and developing a continuous prosody pattern in the time axis direction based on one or both of the voice loudness and the voice pitch included in the utterance information and the time difference thereof Expanding means; and audio reproducing means for generating an audio waveform based on the prosody pattern expanded by the expanding means and the timbre data selected by the selecting means. Information communication system, characterized in that the. 3. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, The first communication device associates one or both of the loudness and loudness of the voice with the time difference and the attribute of the timbre, and either the loudness and the loudness of the voice, or File information storage means for storing file information including utterance information which is not dependent on a time difference between phonemes and has a relative level, and which responds to a request of the second communication device. And a first communication unit for transferring the file information stored in the file information storage unit to the second communication device. The second communication device, for each voice type, for each unit of phoneme or the like Voice representing acoustic parameters Voice storage means for storing data and information indicating the voice attribute thereof in association with each other; and requesting the first communication device to transfer the file information stored in the file information storage means. Second communication means for receiving file information transferred by the first communication means; information indicating a voice attribute of speech information in the file information received by the second communication means; and the voice color storage means Matching means for comparing the information indicating the attributes of the various timbres stored in the timbre, and a plurality of types stored in the timbre storage means based on the similarity determined by the matching means. Selecting means for selecting the voice data having the highest similarity from the voice data of the utterance data, and either one or both of the loudness and the voice pitch included in the utterance information, Expansion means for expanding a continuous prosody pattern in the time axis direction based on the difference, and a voice for generating a voice waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the selection means An information communication system comprising: a reproducing unit. 4. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, The first communication device associates one or both of the loudness and the loudness of the voice with the time difference, the type of the timbre, and the attribute of the timbre. File information storage means for storing file information including utterance information obtained by discriminating one or both of them so as to have a relative level without depending on a time difference between phonemes; and And a first communication unit for transferring file information stored in the file information storage unit to the second communication device in response to a request from the device, wherein the second communication device includes a phoneme or the like for each voice type. Parameters of each unit Voice data storing means for storing voice data representing voice data and information indicating the attribute of the voice, and requesting the first communication device to transfer the file information stored in the file information storage means. And second communication means for receiving the file information transferred by the first communication means thereafter; and changing the type of voice in the speech information in the file information received by the second communication means to the voice color Searching means for searching from various types of voices stored in the storage means; if the type of voice in the utterance information can be obtained by the search of the searching means, the type of voice obtained corresponds to the obtained type of voice. First selecting means for selecting the timbre data to be performed from the various timbre data stored in the timbre storage means, and the type of the timbre in the utterance information can be obtained by searching by the searching means. If there is no vocal information, the information indicating the attribute of the timbre in the utterance information stored in the utterance information storage means is compared with the information indicating the attributes of various timbres stored in the timbre storage means. And a second unit for selecting voice data having the highest similarity from a plurality of types of voice data stored in the voice memory based on the similarity obtained by the matching unit. Selecting means; developing means for developing a continuous prosodic pattern in the time axis direction based on one or both of a voice volume and a voice pitch included in the utterance information, and a time difference thereof; Prosody pattern developed by the means and the first
Or an audio reproducing means for generating an audio waveform based on the timbre data selected by the second selecting means. 5. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, A first communication device for storing file information including utterance information including phonemes and prosody as information; utterance information stored in the file information storage device in response to a request from the second communication device; And a first communication unit for transferring voice data to the second communication device, wherein the second communication device stores voice data representing voice parameters for each unit of phoneme or the like for each voice type. A second communication device that requests the first communication device to transfer the file information stored in the file information storage unit, and thereafter receives the file information transferred by the first communication unit. Selecting means for selecting one voice data from a plurality of types of voice data stored in the voice memory based on speech information in the file information received by the second communication means; Expansion means for expanding a continuous prosody pattern in the time axis direction based on the utterance information; and audio reproduction for generating an audio waveform based on the prosody pattern expanded by the expansion means and the timbre data selected by the selection means. An information communication system comprising: 6. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, A first communication device, a file information storage unit that stores file information including speech information including a phoneme, a temperament, and a timbre type as information; and a file information storage unit that stores the file information in response to a request from the second communication device. First communication means for transferring the obtained file information to the second communication device, wherein the second communication device stores voice data representing an acoustic parameter for each unit of phoneme or the like for each type of voice. Requesting the first communication device to transfer the file information stored in the file information storage unit, and receiving the file information transferred by the first communication unit. A second communication unit, and voice data corresponding to the type of voice in the utterance information of the file information received by the second communication unit, of the plurality of types of voice data stored in the voice storage unit. Selecting means for selecting from among; developing means for developing a continuous prosody pattern in the time axis direction based on the utterance information; and prosody pattern developed by the developing means and voice data selected by the selecting means. An information communication system, comprising: a sound reproducing unit that generates a sound waveform based on the sound reproduction means. 7. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, A first communication device for storing file information including utterance information including phonemes, prosody, and voice attributes as information; and storing the file information in the file information storage device in response to a request from the second communication device. First communication means for transferring the obtained file information to the second communication device, wherein the second communication device comprises: for each type of timbre, voice data representing an acoustic parameter for each unit such as a phoneme; Voice color storage means for storing information indicating the voice color attributes in association with each other; requesting the first communication device to transfer the file information stored in the file information storage means; communication Second communication means for receiving the file information transferred by the step, information indicating a voice attribute of utterance information in the file information received by the second communication means, and stored in the voice storage means Matching means for comparing the information indicating the attributes of the various timbres to determine the similarity of the timbres; and a plurality of types of timbre data stored in the timbre storage means based on the similarity determined by the matching means. Selecting means for selecting voice data having the highest similarity from among the following; developing means for developing a continuous prosody pattern in the time axis direction based on the utterance information; and prosody pattern developed by the developing means and the selection. And a sound reproducing means for generating a sound waveform based on the voice color data selected by the means. 8. An information communication system for connecting a first communication device and a second communication device to a communication network and performing information communication between the first communication device and the second communication device via the communication network, A first communication device, file information storage means for storing file information including utterance information including a phoneme, a prosody, a timbre type, and a timbre attribute as information; and the file information in response to a request from the second communication device. And a first communication unit for transferring file information stored in a storage unit to the second communication device, wherein the second communication device is configured to determine, for each type of voice, an acoustic parameter for each unit of phoneme or the like. Voice data storage means for storing the voice data representing the voice and information indicating the attribute of the voice in association with each other; and requesting the first communication device to transfer the file information stored in the file information storage means. To A second communication unit for receiving the file information transferred by the first communication unit; and a voice type in the utterance information in the file information received by the second communication unit, stored in the voice color storage unit. Search means for searching from the various types of timbres that have been obtained.If the type of timbre in the utterance information can be obtained by the search of the search means, voice data corresponding to the obtained timbre type is obtained. A first selection unit for selecting from among various voice data stored in the voice storage unit; and a case where a type of voice in the speech information cannot be obtained by the search by the search unit. Matching means for comparing the information indicating the attributes of the timbres in the speech information stored in the storage means with the information indicating the attributes of various timbres stored in the timbre storage means to determine the similarity of the timbres Second selecting means for selecting voice data having the highest similarity from a plurality of types of timbre data stored in the timbre storage means based on the similarity determined by the matching means; and the utterance information. Expansion means for expanding a continuous prosody pattern in the time axis direction based on the prosody pattern;
Or an audio reproducing means for generating an audio waveform based on the timbre data selected by the second selecting means. 9. The information indicating the attribute is any one of gender, age, voice pitch criterion, intelligibility, and naturalness,
The information communication system according to any one of claims 3, 4, 7, and 8, wherein the information communication system is a combination of two or more thereof. 10. The file information storage means stores first information indicating a criterion of a voice pitch in the speech information,
The timbre storage means stores second information indicating a criterion of a voice pitch in the timbre data, and the voice reproducing means determines a criterion of a voice pitch based on the first information based on the second information. The information communication system according to any one of claims 1 to 8, wherein a reference for a voice pitch at the time of voice reproduction is determined by shifting according to a reference for a voice pitch. 11. The file information storage means stores first information indicating a criterion of a voice pitch in the utterance information, and stores the first information.
The voice reproducing means has an input means for arbitrarily inputting second information indicating a reference of a voice pitch, and a second information inputted by the input means based on the voice information based on the first information. The information communication system according to any one of claims 1 to 8, wherein a reference of a voice pitch at the time of voice reproduction is determined by shifting according to a voice pitch standard. 12. The information according to claim 10, wherein the criterion of the voice pitch according to the first and second information is an average frequency, a maximum frequency, or a minimum frequency of the voice pitch. Communications system. 13. The second communication device further includes a removable recording medium storing voice data, reads voice data from the recording medium, and stores the read voice data in the voice storage means. The information communication system according to any one of claims 1 to 8, wherein 14. The voice communication apparatus according to claim 1, wherein said second communication device receives voice data from an external device via a communication line, and stores the voice data in said voice storage means.
9. The information communication system according to any one of items 8. 15. The utterance information includes control information for synchronizing an operation based on other information in the file information with an operation performed by the audio reproducing unit, and the audio reproducing unit is included in the utterance information during audio reproduction. 9. The information communication system according to claim 1, wherein the information communication system operates in synchronization with the operation of other information in the file information according to the control information. 16. The information communication system according to claim 15, wherein said other information is image information, music information and the like. 17. An information processing apparatus for creating and editing utterance information used in the information communication system according to any one of claims 1 to 8, wherein: a voice input means for inputting a natural voice; Creating means for creating the utterance information based on natural speech input by means, and requesting the first communication device to register file information including the utterance information created by the creating means; An information processing apparatus, comprising: a registration transfer unit that transfers file information including generated utterance information to the first communication device and registers the file information in the file information storage unit of the first communication device. 18. An information processing apparatus for creating and editing utterance information for use in the information communication system according to claim 10 or 11, wherein said creating means uses the utterance as first information indicating a reference of a voice pitch. The information processing apparatus according to claim 17, wherein the information processing apparatus is created by including the information. 19. The information processing apparatus according to claim 17, wherein said creating means includes a changing means for arbitrarily changing said information. 20. An information processing apparatus for creating and editing utterance information used in the information communication system according to claim 15 or 16, wherein said creating means converts said control information into said utterance when creating said utterance information. 18. The method according to claim 17, wherein the information is included in the information.
An information processing apparatus according to claim 1. 21. A first communication device and a second communication device are connected to a communication network, and in the first communication device, one or both of a voice volume and a voice pitch and a time difference between the two are determined. File information including utterance information obtained by discriminating one or both of the loudness and the pitch of the voice so as to have a relative level without depending on the time difference between phonemes. In the second communication device, voice data representing acoustic parameters for each unit of phoneme or the like for each color type is stored in advance in the file information storage unit, and the voice data is stored in the voice storage unit in advance. By performing information communication between the first communication device and the second communication device via a network, the utterance information of the file information stored in the file information storage unit and the voice stored in the voice storage unit Data and An information communication method for synthesizing speech based on: a transfer that transfers speech information stored in the file information storage unit to the second communication device in response to a request from the second communication device to the first communication device And selecting one voice data from a plurality of types of voice data stored in the voice color storage unit based on speech information in the file information transferred in the transfer step in the second communication device. Selecting, and developing a prosody pattern that is continuous in the time axis direction in the second communication device based on one or both of the voice volume and the voice pitch included in the utterance information and the time difference between the two. And a tone pattern based on the prosody pattern developed in the second communication device in the development step and the timbre data selected in the selection step. An audio communication method, comprising: a voice reproduction step of generating a voice waveform. 22. A first communication device and a second communication device are connected to a communication network, and in the first communication device, one or both of a loudness and a loudness of a voice and a time difference between the two. Utterances that correspond to the timbre type, and that one or both of the voice volume and voice pitch are discrete so as to have a relative level without depending on the time difference between phonemes. The file information including the information is stored in the file information storage unit in advance, and in the first communication device, the timbre data representing the acoustic parameters for each unit of phoneme or the like for each voice type is stored in the timbre storage unit in advance. In addition, by performing information communication between the first communication device and the second communication device via the communication network, the utterance information of the file information stored in the file information storage unit and the voice color storage unit Remembered in An information communication method for performing voice synthesis based on voice data that is stored in said file information storage means in response to a request from said second communication device to said first communication device. Transfer means for transferring to the device; and voice data corresponding to the type of voice in the utterance information of the file information transferred in the transfer step in the second communication device is stored in the voice storage unit. A selecting step of selecting from among a plurality of types of voice data; and a time based on one or both of a loudness and a loudness of a voice included in the utterance information in the second communication device, and a time difference between the two. An expansion step of expanding a prosody pattern that is continuous in the axial direction; and a prosody pattern expanded by the expansion step and selected by the selection step in the second communication device. An audio reproduction step of generating an audio waveform based on the determined timbre data. 23. A first communication device and a second communication device connected to a communication network, wherein at the first communication device, one or both of a voice volume and a voice pitch, and a time difference between the two. , Utterances that correspond to the attributes of the timbre, and that one or both of the voice volume and voice pitch are discrete so as to have a relative level without depending on the time difference between phonemes. The file information including the information is stored in the file information storage unit in advance, and the second communication device indicates, for each type of voice, voice data representing an acoustic parameter for each unit such as a phoneme and the attribute of the voice. The information is stored in the voice storage unit in advance in association with the information, and is stored in the file information storage unit by performing information communication between the first communication device and the second communication device via the communication network. File information An information communication method for performing voice synthesis based on the utterance information and voice color data stored in the voice color storage unit, wherein the file information storage unit is responsive to a request from the second communication device to the first communication device. Transferring the file information stored in the second communication device to the second communication device; and information indicating a voice attribute of the utterance information in the file information transferred by the transfer process in the second communication device. A collation step of collating information indicating the attributes of various timbres stored in the timbre storage unit to obtain similarities of timbres; and A selecting step of selecting voice data having the highest similarity from a plurality of types of voice data stored in the voice storage unit; and the second communication device includes the voice data in the speech information. A developing step of developing a prosody pattern that is continuous in the time axis direction based on one or both of the loudness and pitch of the voice, and a time difference between the two, and the developing step is performed by the developing step in the second communication device. An audio reproduction step of generating an audio waveform based on the prosody pattern and the timbre data selected in the selection step. 24. A first communication device and a second communication device connected to a communication network, wherein at the first communication device, one or both of a voice volume and a voice pitch, and a time difference between the two. , The type of voice and the attribute of voice are associated with each other, and one or both of the voice volume and the voice pitch are not dependent on the time difference between phonemes and have a relative level. File information including discrete utterance information is stored in the file information storage unit in advance, and in the second communication device, for each type of timbre, voice data representing sound parameters for each unit such as a phoneme and The information indicating the attribute of the timbre is stored in the timbre storage unit in advance in association with the information, and the first communication device and the second communication device are connected via the communication network.
An information communication method for performing voice communication based on utterance information of file information stored in the file information storage unit and voice data stored in the voice color storage unit by performing information communication between communication devices. Transferring a file information stored in the file information storage unit to the second communication device in response to a request from the second communication device to the first communication device; and transferring the file information in the second communication device. Searching for the type of voice in the utterance information of the file information transferred by the method from the various types of voice stored in the voice storage unit; and searching in the searching step in the second communication device. When the voice type of the utterance information can be obtained by the above, voice data corresponding to the obtained voice type is stored in the voice storage unit. A first selecting step of selecting from among various voice data; and if the type of voice in the utterance information cannot be obtained by the search in the search step in the second communication device, the file information storage unit A collation step of collating information indicating a voice attribute of the stored utterance information with information indicating various voice attributes stored in the voice storage unit to obtain a similarity of the voice; A second selecting step of selecting, from the plurality of types of timbre data stored in the timbre storage step, the timbre data having the highest similarity in the communication device based on the similarity obtained in the matching step; (2) In the communication device, a prosody pattern continuous in a time axis direction is developed based on one or both of a voice volume and a voice pitch included in the speech information and a time difference between the two. A voice reproduction step of generating a voice waveform based on the prosody pattern expanded by the expansion step in the second communication device and the timbre data selected by the first or second selection step. An information communication method, comprising: 25. A first communication device and a second communication device are connected to a communication network, and in the first communication device, file information including speech information including phonemes and prosody as information is stored in a file information storage unit in advance. In the second communication device, the timbre data representing the acoustic parameters of each unit such as phoneme for each color type is stored in advance in the timbre storage unit, and the second communication device stores the timbre data via the communication network. By performing information communication between the first communication device and the second communication device, a voice is generated based on the utterance information of the file information stored in the file information storage unit and the voice data stored in the voice color storage unit. An information communication method for combining, wherein a transfer step of transferring utterance information stored in the file information storage unit to the second communication device in response to a request from the second communication device to the first communication device. Selecting one voice data from a plurality of types of voice data stored in the voice memory based on speech information in the file information transferred in the transfer step in the second communication device; A developing step of developing a prosody pattern continuous in the time axis direction based on the utterance information in the second communication device; and a prosody pattern developed in the second communication device by the developing step and the selection step. An audio reproduction step of generating an audio waveform based on the selected timbre data. 26. A first communication device and a second communication device connected to a communication network, wherein the first communication device converts file information including speech information including phonemes, prosody, and timbre types as information. Store in advance in the storage unit,
In the second communication device, voice data representing an acoustic parameter for each segment unit such as a phoneme for each type of voice is stored in a voice storage unit in advance, and the first communication device and the second communication device are connected to each other via the communication network. An information communication method for performing voice communication based on utterance information of file information stored in the file information storage unit and voice data stored in the voice color storage unit by performing information communication between communication devices. A transfer step of transferring file information stored in the file information storage unit to the second communication device in response to a request from the second communication device to the first communication device; Voice data corresponding to the type of voice in the utterance information of the file information transferred in the transfer step is output from the plurality of types of voice data stored in the voice storage unit. A selecting step to determine; a developing step of developing a continuous prosody pattern in the time axis direction based on the utterance information in the second communication device; and a prosody pattern developed by the developing step in the second communication device. An audio reproduction step of generating an audio waveform based on the voice color data selected in the selection step. 27. A first communication device and a second communication device connected to a communication network, wherein the first communication device converts file information including speech information including phonemes, prosody, and voice attributes as file information. Store in advance in the storage unit,
In the second communication device, for each type of voice, voice data representing an acoustic parameter for each unit such as a phoneme and information indicating an attribute of the voice are stored in a voice storage unit in advance, and the communication is performed. By performing information communication between the first communication device and the second communication device via a network, the utterance information of the file information stored in the file information storage unit and the voice stored in the voice storage unit An information communication method for performing voice synthesis on the basis of data, wherein the file information stored in the file information storage unit is transferred to the second communication device in response to a request from the second communication device to the first communication device. A transfer step, and information indicating a voice attribute of speech information in the utterance information of the file information transferred by the transfer step in the second communication device, and each of the information stored in the voice color storage unit. A collation step of collating information indicating the attribute of the timbre to determine the similarity of the timbre; and a plurality of types stored in the timbre storage unit based on the similarity determined by the collation step in the second communication device. Selecting the voice data having the highest similarity from the voice data of the first and second voice data; and developing the prosody pattern continuous in the time axis direction based on the utterance information in the second communication device; An information communication method, comprising: in a communication device, a voice reproduction step of generating a voice waveform based on the prosody pattern expanded in the expansion step and the timbre data selected in the selection step. 28. A file containing speech information including, as information, a phoneme, a prosody, a type of voice, and an attribute of voice, wherein the first communication device and the second communication device are connected to a communication network. The information is stored in the file information storage unit in advance, and in the second communication device, for each type of timbre, timbre data representing an acoustic parameter for each unit such as a phoneme and information indicating an attribute of the timbre are stored. Correspondingly stored in the voice storage unit in advance, and the first
By performing information communication between the communication device and the second communication device, voice synthesis is performed based on utterance information of the file information stored in the file information storage unit and voice data stored in the voice color storage unit. An information communication method for transferring file information stored in the file information storage unit to the second communication device in response to a request from the second communication device to the first communication device; (2) a search step of searching for the type of voice in the utterance information of the file information transferred in the transfer step from the various voice types stored in the voice color storage unit in the communication device; When the type of voice in the utterance information can be obtained by the search in the search step, the voice data corresponding to the obtained voice type is stored in the voice storage unit. A first selecting step of selecting from among various stored voice data; and a step of selecting the voice information in the utterance information by the second communication device in the search in the search step. A collation step of collating information indicating the attribute of the timbre of the utterance information stored in the information storage unit with information indicating the attributes of various timbres stored in the timbre storage unit to determine the similarity of the timbres; A second selecting step of selecting, from the plurality of types of timbre data stored in the timbre storage unit, voice data having the highest similarity in the second communication device based on the similarity obtained in the matching step; An expansion step of expanding a prosody pattern continuous in a time axis direction based on the utterance information in the second communication device; and an expansion step of expanding the prosody pattern in the second communication device. Information communication method, which comprises a sound reproduction step of generating a speech waveform based on the tone of voice data selected by said prosodic pattern first or second selection step. 29. The information indicating the attribute is any one of gender, age, voice pitch criterion, clarity, and naturalness, or a combination of two or more thereof. The information communication method according to any one of claims 23, 24, 27, and 28. 30. The file information storage unit stores first information indicating a criterion of a voice pitch in the utterance information, and the timbre storage unit stores the second information indicating a criterion of a voice pitch. The voice reproduction step is performed by shifting the reference of the voice pitch according to the first information by the reference of the voice pitch according to the second information. 22. A criterion for determining the criterion is determined.
29. The information communication method according to any one of -28. 31. The file information storage unit stores first information indicating a reference of a voice pitch in the speech information, and stores the second information indicating a reference of a voice pitch in the speech information. An input step of arbitrarily inputting, wherein a voice pitch at the time of voice reproduction is shifted by shifting a voice pitch standard according to the first information according to the voice pitch standard according to the second information input at the input step. The information communication method according to any one of claims 21 to 28, wherein a reference for the height of the information is determined. 32. The information according to claim 30, wherein the criterion of the voice pitch according to the first and second information is an average frequency, a maximum frequency, or a minimum frequency of the voice pitch. Communication method. 33. The second communication device, further comprising connecting a detachable recording medium storing voice data, reading voice data from the recording medium, and storing the read voice data in the voice storage unit. The information communication method according to any one of claims 21 to 28. 34. The apparatus according to claim 21, wherein the second communication device receives voice data from an external device via a communication line and stores the voice data in the voice storage unit.
28. The information communication method according to any one of 28. 35. The utterance information includes control information for synchronizing an operation based on other information in the file information with an operation according to the audio reproduction step, and the audio reproduction step is included in the utterance information during audio reproduction. The information communication method according to any one of claims 21 to 28, wherein the information communication method operates in synchronization with the operation of other information in the file information according to the control information. 36. The information communication method according to claim 35, wherein the other information is image information, music information, and the like. 37. An information processing method for creating and editing utterance information to be used in the information communication method according to claim 21 by a third communication device connected to the communication network. A voice inputting step of inputting a voice; a creating step of creating the utterance information based on the natural voice input in the voice inputting step; and the utterance information created by the creating step for the first communication device. A registration and transfer step of requesting registration of file information including the utterance information, transferring the file information including the generated utterance information to the first communication device, and registering the file information in the file information storage unit of the first communication device; An information processing method comprising: 38. An information processing method for creating and editing utterance information used in the information communication method according to claim 30 or 31, wherein the creating step includes first utterance indicating a criterion of a voice pitch being said utterance. 38. The information processing method according to claim 37, wherein the information is created by being included in the information. 39. The information processing method according to claim 37, wherein the creating step includes a changing step of arbitrarily changing the information. 40. An information processing method for creating and editing utterance information used in the information communication method according to claim 35 or 36, wherein the creating step includes transmitting the control information when creating the utterance information. 38. The information included in the information.
The information processing method described.