JPH1165597A - Voice compositing device, outputting device of voice compositing and cg synthesis, and conversation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice composition device, an outputting device of voice synthesis and CG synthesis and a conversation device which outputs composied voice having natural phoneme continuing time length and pause length and naturally moving CG images. SOLUTION: The voice compositing device is shown in the figure. A language processing means 31 generates synthesized sentences to be synthesized from voices an outputted. The composited sentences are converted to phonemic symbol sequences added with pronunciation information by a prosodic processing means 32 to be sent to a rhythm generating means 33. The means 33 retrieves the pitch patterns which correspond to the syllables to be intended for voice synthesis from the pitch pattern data base section 34 which stores pitch patterns depending on the type of accent to generate rhythms by controlling the obtained pitch patterns. And as for the outputting device of voice composites and CG synthesis a CG control means is added which synchronizes CG with voices and controls. Furthermore as for the conversation device a rhythm detecting means for detecting the rhythms of a user and using the detected information for rhythm generation is added.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech synthesizer in which a human and a computer interact with each other through voice or gestures, a speech synthesis and CG synthesis output device, and a dialogue device.

[0002]

2. Description of the Related Art Conventionally, as an interface between a human and a computer, a voice synthesizer which responds by voice has been considered. This is to realize a dialogue with a human by recognizing a voice uttered by a human, changing an internal state of the system according to the voice, outputting a predetermined voice. Further, in order to facilitate a dialogue with a computer, an output synthesizing device (equivalent to a voice synthesizing and CG synthesizing output device of the present application) in which animation or the like responds to an input voice has been proposed.

In the above-described speech synthesizer, when synthesizing speech, the quality of a prosody control rule such as the duration of a phoneme affects the naturalness of the synthesized speech. For example, IEICE Transactions Vol. J67-A, No.
7 (1984) describes a method of determining the duration of a phoneme by performing expansion and contraction on the average duration of the phoneme inherently in consideration of various conditions. Further, Japanese Patent Application Laid-Open No. 61-32096 describes a method of expanding and contracting the duration of each phoneme at a standard utterance tempo by using different ratios for consonants and vowels. Further, Japanese Patent Application Laid-Open No. 3-89299 describes a method of determining the duration of a phoneme using a different constant table for each utterance tempo. However, in the above-described speech synthesis method and apparatus, the rhythm of the synthesized speech and the tempo of the utterance are controlled by changing the duration time with the rhythm of the phoneme as a unit of control. Since the control is not performed in units of phonemes, an unnatural rhythm may occur.

In Japanese Patent Application Laid-Open No. Hei 6-222793, a sentence to be output is decomposed into vowel-consonant-vowel chains, and the duration from the center of the preceding vowel to the center of the succeeding vowel at the normal speech tempo of each chain is calculated. It describes a method of calculating as a unit of rhythm and expanding and contracting the length between vowel centers according to the utterance tempo. However, such as strange intonation
Unnaturalness was a problem.

[0005]

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems. The rhythm unit is defined as a start point of an utterance output from a computer, a start point of a descending pitch, or a hand of a CG person. By setting the periodic length of the swing and head swing and controlling these features to appear in a complementary and isochronous manner, a natural phoneme duration and pause length close to the actual utterance were obtained. A speech synthesizer capable of outputting synthesized speech or a CG image that moves naturally,
The purpose of the present invention is to provide a speech synthesis and CG synthesis output device and a dialogue device.

[0006]

According to a first aspect of the present invention, there is provided a language processing means for generating a synthesized sentence for synthesizing speech, and information on accent, pause, and unvoiced vowels in the synthesized sentence generated by the language processing means. Prosody processing means for generating a phoneme symbol string by adding pronunciation information including: a pitch pattern database section storing a pitch pattern classified by accent type in speech as a database; and a phoneme symbol string obtained by the prosody processing means. Rhythm generating means for searching the pitch pattern database section for a pitch pattern corresponding to each of the included phrases, controlling the obtained pitch pattern and generating a rhythm of the synthesized voice, and outputting the rhythm generating means. It is characterized by synthesizing speech based on It becomes, as a result, in which naturalness of synthetic speech is to be improved.

According to a second aspect of the present invention, in the first aspect of the present invention, the control of the pitch pattern is performed by changing a phoneme duration time and a pause length so that a predetermined characteristic amount of the pitch pattern is output at an equal time. It is characterized by performing control,
A rhythmic synthesized speech is obtained.

[0008] A third aspect of the present invention is characterized in that, in the second aspect of the present invention, a starting point of a speech to be synthesized and a starting point of a descending pitch are used as the predetermined feature amount, and a more rhythmical synthesized speech is obtained. A specific rhythm generating means can be provided.

The invention of claim 4 is the invention according to any one of claims 1 to 3, characterized in that the rhythm generating means has a fluctuation in the cycle of the generated rhythm, and is not mechanical but more natural. A rhythm can be generated.

According to a fifth aspect of the present invention, in any one of the first to fourth aspects of the present invention, the rhythm generating means has a function of modulating a rhythm to be generated in accordance with the content of a voice to be synthesized. It is intended to give emotion to computer speech and create a more intimate synthesized speech.

According to a sixth aspect of the present invention, in the voice synthesizing apparatus according to any one of the first to fifth aspects, CG control means for controlling the CG so that the CG person operates in synchronization with the synthesized voice is provided. The synthesized CG is output together with the synthesized voice based on the output of the CG control means.
The movement of a person can be obtained.

According to a seventh aspect of the present invention, there is provided an audio synthesizing apparatus according to any one of the first to fifth aspects, wherein input means for inputting a user's utterance and a rhythm of the user's utterance input from the input means are detected. Rhythm detecting means for controlling the pitch pattern using the detection result of the rhythm detecting means, so that the rhythm of the user and the computer cooperate to realize a dialog with a sense of rhythm. It was done.

According to an eighth aspect of the present invention, there is provided the speech synthesis and CG synthesis output device according to the sixth aspect, wherein input means for inputting a user's utterance and movement, and rhythm of the user's utterance and movement input from the input means Rhythm detecting means for detecting the pitch, and controlling the pitch pattern using the detection result of the rhythm detecting means, whereby the rhythm of the user and the computer cooperate to realize a rhythmic dialogue. It is like that.

According to a ninth aspect of the present invention, in the seventh or eighth aspect of the present invention, at least one of the operation modes of the voice power, the voice pitch, and the hand movement generated by the user is selected from the operation modes. A rhythm of the utterance or a rhythm of the utterance and the movement is detected from a recognition result, wherein a rhythm of the user and a computer cooperate to realize a rhythm-like dialogue. Can be detected more accurately.

According to a tenth aspect of the present invention, there is provided a computer-readable recording medium in which a program for causing the voice synthesizing apparatus according to any one of the first to fifth aspects to function is recorded.

According to an eleventh aspect of the present invention, there is provided a computer-readable recording medium on which a program for causing the voice synthesizing and CG synthesizing output device according to the sixth aspect to function is recorded.

According to a twelfth aspect of the present invention, there is provided a computer-readable recording medium storing a program for causing the interactive device according to any one of the seventh to ninth aspects to function.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION

(Embodiment 1) FIG. 1 is a block diagram for explaining a first embodiment of the present invention, and shows a basic configuration example of a speech synthesizer. Hereinafter, an embodiment of the present invention according to the basic configuration shown in FIG. 1 will be described. The voice of the user input to the input unit 10 is analyzed by the analysis unit 20 and input to the response generation unit 30. In the response generation unit 30, a synthesized sentence that is synthesized and output by the computer is generated by the language processing unit 31 and sent to the prosody processing unit 32. Prosody processing means 32
Then, the sent synthesized sentence is converted into a phoneme symbol string to which pronunciation information such as accent information, pause, and vowel devoicing necessary for speech synthesis processing is added, and sent to the rhythm generating means 33. The rhythm generating means 33 stores the pitch pattern database 3 storing pitch patterns of the accent type.
4, a pitch pattern corresponding to a phrase to be subjected to speech synthesis is searched from the pitch pattern database 34, the obtained pitch pattern is controlled, a rhythm is generated and output from the output unit 40. FIG. 2A shows an example of a synthesized sentence generated by the language processing unit 31, and FIG. 2B shows an example of a phoneme symbol string generated by the prosody processing unit 32.

Next, the rhythm generating means 33 and the pitch pattern database 34 which are features of the present invention will be described. In the pitch pattern database 34,
The pitch of a natural utterance is analyzed in advance, and the pitch is normalized to make a database as a pitch pattern.

FIG. 3 is a diagram showing an example of a 4-mora pitch pattern database. Where the accent type N
Represents the position of the accent nucleus. The accent is a mora just before the frequency drops when the accent is applied (the starting point of the drop pitch). That is, type 1 means that the accent nucleus is the first mora, type 2 means that the accent nucleus is the second mora, type 3 means that the accent nucleus is the third mora, and type 0 Indicates that there is no accent nucleus. Although FIG. 3 shows an example of four moras, one accent type N has different patterns of the number M of moras.

The rhythm generating means 33 determines 1 from the number of mora and the accent type of the phrase to be synthesized from the plurality of normalized pitch patterns stored as described above.
Search for one pitch pattern. In the analysis of the dialogue between humans, it has been found that in a smooth dialogue, the start point of the utterance, the descending pitch of the utterance, and the wave of the hand complementarily appear almost at the same time. Therefore, the rhythm generating means 33
Controls such that these feature values occur complementarily and isochronously. In the following, the pitch pattern database 34
By expanding and contracting the phoneme duration and pause length so that the appearance interval between the utterance start point and the accent nucleus (falling pitch start point) is complementary and isochronous to the pitch pattern retrieved from An example of generating a rhythm will be described.

FIG. 4 shows a pitch pattern for each phrase searched from the pitch pattern database 34 when the computer reads, "This electronic notebook can easily make a memo just by inputting with a pen." This is an example. FIG. 5 shows an example of a pitch pattern before generating a rhythm by controlling the retrieved pitch pattern.
Is an example of a pitch pattern after processing by the rhythm generating means 33. The computer shows, "This electronic organizer can easily make notes just by inputting with a pen." The vertical axis is pitch (Hz), the horizontal axis is time (frame = 1)
/ 30 seconds). In FIG. 6 where the rhythm generation processing has been performed,
Utterance start time T0 and descent pitch start times T1, T
The duration of the phoneme and the length of the pause are expanded and contracted so that the appearance interval P of 2, T3.

More specifically, the rhythm generating means 33 determines, for the pitch pattern extracted by the search, the utterance start point and the appearance interval P of each accent nucleus (falling pitch start point).
Is calculated such that P = np (sec.), And the duration of the phoneme (or mora) and pause between each accent nucleus (the starting point of the descending pitch) is controlled to control the start time of the accent nucleus. In this way, it is possible to generate a rhythm in which the accent nucleus appears at the same time. Here, p is, for example,
A constant such as 0.8, which can be changed according to the internal state of the computer.

Further, n is n = 1 when the number M of moras sandwiched by the appearance interval P satisfies, for example, 3 <M ≦ 10, and n = 0.5 when M ≦ 2. And M
> 10, n = 2, and in the pitch pattern obtained from the database, when adjacent accent nuclei are too close (M ≦ 2) or when a phrase without an accent nucleus continues (M > 10), the intervals of appearance of accent nuclei are held at multiples of p so that the rhythm can be maintained.

By giving 1 / f fluctuation to the value of p, it is possible to generate a non-mechanical, more natural rhythm. Also, for example, in the case of urgent contents, the value of p is shortened, or when there is an unclear point in the response content, and in the case of uncertainty, the value of p is extended, and By modulating the value of p according to, it is possible to give emotion to the sound of the computer,
It is possible to create a more familiar synthesized speech.

(Embodiment 2) FIG. 7 is a block diagram for explaining a second embodiment of the present invention.
1 shows an example of a basic configuration of a G synthesis output device. In the configuration of the present embodiment, as shown in the figure, in addition to the configuration of the speech synthesizer of the first embodiment, a CG control unit 35 for controlling the operation of the output CG person is added. In the following, C
An operation example in the case of controlling the head movement of the G person will be described.
In analyzing dialogue between humans, it has been found that the rhythm in the dialogue can be extracted from voice power, voice pitch, hand waving, and head movement. In a smooth dialogue, the vertical movement of the head tends to be synchronized with the starting point of the utterance and the descending pitch of the utterance. It is known to appear.

FIG. 8 is a view for explaining an example of the processing in the CG control means 35 shown in FIG. In FIG.
The time change of the sound when the user uttered, "You can easily create a memo just by inputting with this pen," and the movement of the head of the CG image synchronized with the sound are shown. The vertical axis represents the magnitude of the movement, and the horizontal axis represents time (frame). As shown in the drawing, the utterance start time T0 and the start times T1, T2, T3... Of the descending pitch and the start times Tcg1, Tcg2.
The appearance time of the head movement of the CG person is controlled so as to appear complementary and isochronously.

(Embodiment 3) FIG. 9 is a block diagram for explaining a third embodiment of the present invention, and shows a basic configuration example of a dialogue apparatus. In analyzing dialogue between humans, it has been found that the rhythm of talking while thinking and the rhythm of talking when eagerly talking change. In the present embodiment, the rhythm of the computer is changed in accordance with the rhythm of the user's dialogue, and a rhythmical dialogue between the user and the computer is realized.

The configuration of this embodiment is as shown in FIG.
In addition to the configuration of the speech synthesizer of the first embodiment, a rhythm detecting unit 50 for detecting a rhythm of a user's conversation is added. Hereinafter, an embodiment of the present invention will be described assuming that the computer is provided with means for recognizing the voice pitch of input data as voice, and voice output means for outputting synthesized voice as output means.

First, the voice pitch of the input data is A / D converted, and the voice pitch level is recognized for each predetermined processing unit (frame: 1 frame is 1/30 second). Is detected and sent to the rhythm detecting means 50. In the rhythm detecting means 50,
A cycle (rhythm) is detected from the falling pitch of the input user's voice, and the rhythm of the user's dialogue is detected together with time information.

FIG. 10 is a view for explaining an example of the processing in the rhythm detecting means shown in FIG. FIG. 10 shows a time change of the voice when the user utters "This is the form when reproduced in a computer." The vertical axis is pitch (Hz) and the horizontal axis is time (frame). The utterance start time T0 and the descending pitch start times T1, T2, T3 at regular intervals.
, The period of the appearance of the descending pitch is obtained by autocorrelation. The rhythm generating unit 33 determines the appearance interval P of the accent nucleus (the starting point of the downward pitch) of the utterance output from the computer according to the rhythm of the user detected by the rhythm detecting unit 50 as described above. .

[0032]

【The invention's effect】

Advantageous Effects of the Invention: A pitch pattern corresponding to a phrase to be subjected to speech synthesis is searched from a pitch pattern database unit, and the obtained pitch pattern is controlled to generate a rhythm, so that a pitch pattern close to natural speech is obtained. Can be obtained, and as a result, the naturalness of the synthesized speech is improved.

Effect of Claim 2 In addition to the effect of Claim 1, by controlling the phoneme duration time and pause length so that a constant feature of the pitch pattern appears complementarily and isochronously, A rhythmic synthesized voice is obtained.

The effect of the third aspect: In addition to the effect of the second aspect, the unit of the rhythm is the interval between the start point of the output voice and the start point of the descending pitch, and these feature amounts are complementary and isochronous. By controlling to appear, a specific rhythm generating means for obtaining a more rhythmic synthesized voice can be provided.

According to the fourth aspect of the invention, in addition to the effect of any one of the first to third aspects, the rhythm generating means generates a non-mechanical, more natural rhythm by making the rhythm cycle fluctuate. Becomes possible.

Effect of Claim 5: In addition to the effect of any one of Claims 1 to 3, in addition to the effect of the rhythm generating means, the rhythm is modulated according to the content of the synthesized voice to give emotion to the sound of the computer. And can produce synthesized speech that is more familiar.

Effect of Claim 6 In addition to the effect of any one of Claims 1 to 5, in addition to the effects of the output CG person's hand movements and head movements synchronized with the output voice. By performing control so that they appear in a complementary and isochronous manner, a natural CG person's movement can be obtained.

Effects of Claims 7 and 8: Claims 1 to 6
In addition to any one of the effects, the rhythm of the user is detected,
By controlling the output rhythm based on the detection result, the rhythm of the user and the computer cooperate, and a rhythmic dialogue is realized.

According to the ninth aspect, in addition to the effects of the seventh and eighth aspects, the rhythm of the user is detected by at least one of an operation mode of a voice power, a voice pitch, a hand movement and a head movement generated by the user. By recognizing the operation states of the above types of operation modes, the rhythm of the user and the computer cooperate, and the rhythm of the user for realizing a rhythmic dialog can be detected more accurately. .

According to the tenth aspect, it is possible to provide a computer-readable recording medium in which a program for causing a speech synthesizer to function is recorded.

According to the eleventh aspect, it is possible to provide a computer-readable recording medium in which a program for causing a speech synthesis and CG synthesis output device to function is recorded.

According to the twelfth aspect, it is possible to provide a computer-readable recording medium in which a program for causing the interactive device to function is recorded.

[Brief description of the drawings]

FIG. 1 is a block diagram for explaining a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of a synthesized sentence generated by a language processing unit shown in FIG. 1 and an example of a phoneme symbol string generated by a prosody processing unit.

FIG. 3 is a diagram showing an example of a 4-mora pitch pattern database.

FIG. 4 is a diagram showing an example of a pitch pattern for each clause retrieved from the pitch pattern database shown in FIG. 1;

FIG. 5 is a diagram showing an example of a pitch pattern before rhythm generation by the rhythm generation means shown in FIG. 1;

FIG. 6 is a diagram showing an example of a pitch pattern after processing by the rhythm generating means shown in FIG. 1;

FIG. 7 is a block diagram for explaining a second embodiment of the present invention, showing a basic configuration example of a speech synthesis and CG synthesis output device.

FIG. 8 is a diagram for explaining an example of processing in the CG control means shown in FIG.

FIG. 9 is a block diagram for explaining a third embodiment of the present invention, showing a basic configuration example of a dialogue device.

FIG. 10 is a diagram for explaining an example of a process in a rhythm detecting unit shown in FIG. 9;

[Explanation of symbols]

10 input unit, 20 analysis unit, 30 response generation unit, 31
... Language processing means, 32 ... Prosody processing means, 33 ... Rhythm generation means, 34 ... Pitch pattern database, 35 ... C
G control means, 40 ... output unit, 50 ... rhythm detection means.

Claims (12)

[Claims] 1. A language processing means for generating a synthesized sentence for synthesizing a speech, and phonetic symbol strings are added to the synthesized text generated by the language processing means by adding pronunciation information including accent, pause, and vowel unvoiced information. Prosody processing means to be generated, a pitch pattern database section storing a pitch pattern classified by accent type in speech as a database, and a pitch pattern corresponding to each of the phrases included in the phoneme symbol string obtained by the prosody processing means. A rhythm generating unit that searches the pitch pattern database unit and controls the obtained pitch pattern to generate a rhythm of the synthesized voice, and synthesizes a voice based on an output of the rhythm generating unit. Speech synthesizer. 2. The method according to claim 1, wherein the control of the pitch pattern includes controlling a phoneme duration time and a pause length so that a predetermined characteristic amount of the pitch pattern is output at the same time. Voice synthesizer. 3. The speech synthesizer according to claim 2, wherein a start point of a speech to be synthesized and a start point of a descending pitch are used as the predetermined feature amount. 4. The speech synthesizer according to claim 1, wherein said rhythm generating means has a fluctuation in a cycle of the generated rhythm. 5. The speech synthesizer according to claim 1, wherein said rhythm generating means has a function of modulating a rhythm to be generated in accordance with the content of the voice to be synthesized. 6. A voice synthesizing apparatus according to claim 1, further comprising CG control means for controlling CG so that a CG person operates in synchronization with the synthesized voice, and an output of said CG control means. A speech synthesis and CG synthesis output device, which outputs a synthesized CG together with a synthesized voice based on the following. 7. An input means for inputting a user's utterance, and a rhythm detecting means for detecting a rhythm of the user's utterance input from the input means, to the sound synthesizing apparatus according to any one of claims 1 to 5. And controlling the pitch pattern using a detection result of the rhythm detecting means. 8. An input means for inputting a user's utterance and movement to the speech synthesis and CG synthesis output device according to claim 6, and a rhythm detection for detecting a rhythm of the user's utterance and movement input from said input means. Means for controlling the pitch pattern using the detection result of the rhythm detecting means. 9. An operation state of at least one of an operation mode of a voice power, a voice pitch, and a hand movement generated by a user is recognized, and a rhythm or utterance of the utterance is recognized from a recognition result. The utterance and movement rhythm are detected.
Or the interactive device according to 8. 10. A computer-readable recording medium on which a program for causing the speech synthesizing apparatus according to claim 1 to function is recorded. 11. A computer-readable recording medium on which a program for causing the speech synthesis and CG synthesis output device according to claim 6 to function is recorded. 12. A computer-readable recording medium in which a program for causing the interactive device according to claim 7 to function is recorded.