JP6232892B2 - Speech synthesis apparatus and program - Google Patents

Description

  The present invention relates to a speech synthesizer and a program.

In recent years, the following have been proposed as speech synthesis techniques. That is, by synthesizing and outputting speech corresponding to the user's speech tone and voice quality, a technique for sounding more humanly (see, for example, Patent Document 1), analyzing the user's speech, A technique for diagnosing a health condition or the like (see, for example, Patent Document 2) has been proposed.
In addition, a voice dialogue system that recognizes a voice input by a user and outputs a content specified in a scenario by voice synthesis to realize a voice dialogue with the user has been proposed (for example, Patent Document 3). reference).

JP 2003-271194 A Japanese Patent No. 4495907 Japanese Patent No. 4832097

By the way, it is assumed that a dialogue system that combines the above-described speech synthesis technology and a speech dialogue system to retrieve data and output by speech synthesis in response to a speech by a user's voice is assumed. In this case, a problem has been pointed out that the voice output by the voice synthesis feels unnatural to the user, specifically, the machine sometimes feels roaring.
The present invention has been made in view of such circumstances, and one of its purposes is to give the user a natural feeling in answering the user's remarks and to interact with the user. Is to provide a speech synthesizer and a program that give a kind of joyful feeling.

  When examining the man-machine system for outputting (replying) the response to the user's utterance by speech synthesis, the present inventor first determines the pitch (frequency) ).

  Here, as a dialogue between people, the other person (assuming b) answers (including conflicts) to one person (assuming a) (including questions, monologues, questions, etc.) consider. In this case, when “a” speaks, not only “a” but also “b” trying to reply to the said speech often leaves the pitch in a certain section of the said speech with a strong impression. b, when replying with consent, approval, or affirmation, the pitch of the utterance that remains in the impression, for example, the pitch of the ending or beginning of the sentence Speak to be in a Kyowa pitch. The person a who heard the answer has the above-mentioned relationship between the pitch that remains in the impression about his / her speech and the pitch of the part that characterizes the answer to the person's reply, so that the answer to b is comfortable and reassuring. The present inventor thought that he had a good impression.

  For example, when “a” says “Yes?”, “A” and “b” are in a state of memorizing the pitch of “Sho” at the end of the said utterance that strongly expresses willingness or confirmation . In this state, when b tries to affirmatively answer “Yes, yes” to the statement, the part characterizing the answer, for example, the ending, Answer “Yes, yes” so that the pitch of “Yes” is in the above relationship.

FIG. 2 shows a formant in such an actual dialogue. In this figure, the horizontal axis is time, the vertical axis is frequency, and the spectrum shows a state where the intensity increases as it becomes white.
As shown in the figure, a spectrum obtained by frequency analysis of human speech appears as a plurality of peaks that move in time, that is, formants. Specifically, a formant corresponding to “Yeah?” And a formant corresponding to “Ah, yes” each appear as three peak bands (white band-like portions moving along the time axis).
When attention is paid to the first formant having the lowest frequency among these three peak bands, the frequency of the code A (the central part) corresponding to “Sho” of “Oh, right?” Is approximately 400 Hz. On the other hand, for the code B, the frequency of the code B corresponding to “Yes” of “A, Yes” is approximately 260 Hz. For this reason, it can be seen that the frequency of the code A is approximately 3/2 with respect to the frequency of the code B.

  The relationship that the ratio of the frequencies is 3/2 is the relationship between “seo” in the same octave and “mi” in the octave one lower than “mi”. As will be described later, there is a complete 5 degree relationship. This frequency ratio (predetermined relationship between pitches) is a preferred example, but various examples can be given as will be described later.

  FIG. 3 is a diagram showing a relationship between a pitch name (floor name) and a human voice frequency. In this example, the frequency ratio when the fourth octave “do” is used as a reference is also shown, and “so” is 3/2 as described above when “do” is used as a reference. Further, the frequency ratio when the third octave “La” is used as a reference is also illustrated in parallel.

  In this way, in the dialogue between people, it can be considered that the pitch of the utterance and the pitch of the answer to be answered are not irrelevant but have the above relationship. Then, the present inventor analyzed many dialogue examples and statistically aggregated evaluations by many people to prove that this idea is roughly correct.

On the other hand, when considering a dialogue system that outputs (replies) speech replies to a user's utterances, it is assumed that the user is used by people of various attributes, for example, regardless of age or gender. The In addition, there is a model to be collected for data such as speech segments used for speech synthesis. In other words, if a plurality of models are prepared, answers can be synthesized with various voice qualities. For this reason, when an answer is output by speech synthesis, the answer can be output with various attributes (agent attributes).
For this reason, in the interactive system, it must be considered that there are various combinations of the user's speaker attribute and the agent attribute.
Specifically, for example, when the speaker is a female and the respondent is a male, the pitch of the ending of a reply to the male is predetermined for the pitch of the female. Even if you try to answer in a relationship, the pitch of the ending of the answer is too high for men, and on the contrary, it becomes unnatural. Conversely, when the speaker is a male and the respondent is a female, the pitch of the ending of the answer to the utterance of the female has a predetermined relationship with the pitch of the ending of the utterance by the male. Even when trying to answer like this, the pitch of the ending of the answer is too low for women.
Therefore, in order to achieve the above-mentioned purpose when speech synthesis is performed on an answer to a user's utterance, the following configuration is adopted.

That is, in order to achieve the above object, a speech synthesizer according to an aspect of the present invention analyzes a speech input unit that inputs a speech by a speaker, and a pitch of a specific first section of the speech. A pitch analysis unit, an acquisition unit that acquires an answer to the utterance, a voice synthesis unit that synthesizes the acquired answer with a predetermined agent attribute, and a specific second section in the answer to the voice synthesis unit The voice synthesis is controlled according to a rule for changing the pitch of the speaker to be a pitch having a predetermined relationship with the pitch of the first section, and the speaker attribute of the speaker or the agent attribute A voice control unit that modifies the rule according to at least one of the rules.
According to this aspect, the voice is changed according to a rule in which the pitch of the specific second section in the answer is changed so that the pitch has a predetermined relationship with the pitch of the specific first section of the utterance. Synthesis is controlled. Furthermore, the rule is modified according to at least one of the speaker attribute and the agent attribute of the speaker. For this reason, it becomes possible to give a kind of joy to interacting with the user while giving the user a natural feeling in the reply to the user's remarks.
The speaker attribute of the speaker is, for example, the gender of the speaker. Gender includes male, female, and neutral. In addition to sex, speaker attributes may include age, age, age of children / adults / old people, and so on. This speaker attribute may be preset for the speech synthesizer or may be obtained on the speech synthesizer side.
The agent attribute is an attribute of a model used for speech synthesis, and is gender and age (age) as in the case of the speaker attribute. This agent attribute is preset in the speech synthesizer, for example.

In this aspect, it is preferable that the first interval is, for example, the ending of a statement, and the second interval is the beginning or ending of an answer. As described above, the section characterizing the impression impression is the ending of the comment, and the section characterizing the answer impression is often the beginning or ending of the answer.
Moreover, it is preferable that the predetermined relationship is a relationship of Kyowa intervals excluding perfect 1 degree. Here, “Kyowa” means a relationship in which when a plurality of musical sounds are generated at the same time, they are fused and well harmonized, and these pitch relationships are called Kyowa pitches. The degree of cooperation is higher as the frequency ratio (frequency ratio) between two sounds is simpler. The simplest frequency ratios of 1/1 (perfect 1 degree) and 2/1 (perfect 8 degree) are called absolute consonance pitches, and 3/2 (perfect 5 degree) and 4/3 (perfect) 4 degrees) and is called the perfect harmony pitch. 5/4 (3 degrees long), 6/5 (3 degrees short), 5/3 (6 degrees long) and 8/5 (6 degrees short) are called incomplete harmony intervals, and all other frequency ratios This relationship (long and short 2 degrees and 7 degrees, various increase / decrease intervals, etc.) is called dissonance interval.

If the pitch of the beginning or ending of the answer is the same as the pitch of the ending of the utterance, it is considered that there is an unnatural feeling as a dialogue. Is excluded.
In the above aspect, the most desirable example of the predetermined relationship is considered to be a relationship in which the pitch of the second section is 5 degrees below the pitch of the first section as described above. However, the predetermined relationship is not limited to a consonant pitch except for a perfect degree, but may be a dissonant pitch relationship, or may be a pitch relationship within the range of one octave above and below, excluding the same.
In addition, the answer is not limited to a specific answer to the question, but includes an answer (interjection) such as “I see” or “I think so”.

In the above aspect, a language analysis unit that analyzes language information of the utterance is provided, and when the language information is predetermined, the answering unit obtains a conflict as an answer to the utterance, and the voice control unit It is good also as a structure which controls the output of the said cooperation with respect to the part according to the speaker attribute of the said speaker. According to this configuration, from the user's point of view, since a conflict is output according to his / her attributes, the user feels more natural as if he / she is interacting with a person.
In addition to controlling the output timing of the conflict, the control mode of the conflict output includes control for repeatedly outputting the conflict (sequential call) and control for not outputting the silence (silently).

The aspect of the present invention can be conceptualized as a program that causes a computer to function as the speech synthesizer as well as the speech synthesizer.
In the present invention, the pitch (frequency) of the speech is the analysis target, and the pitch of the answer is the control target. As is clear from the above-described formant example, human speech has a certain frequency range. Therefore, it is inevitable that the analysis and control have a certain frequency range. In addition, as a matter of course, errors occur in analysis and control. For this reason, in this case, the pitch analysis and control are allowed not only to have the same numerical value of the pitch (frequency) but also to have a certain range.

It is a block diagram which shows the structure of the speech synthesizer which concerns on embodiment. It is a figure which shows the example of the sound formant in a dialog. It is a figure which shows the relationship between a pitch name, a frequency, etc. It is a flowchart which shows the speech synthesis process in a speech synthesizer. It is a flowchart which shows the speech synthesis process in a speech synthesizer. It is a figure which shows the specific specific example of an ending. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence.

Embodiments of the present invention will be described below with reference to the drawings.
<Speech synthesizer>

FIG. 1 is a diagram showing a configuration of a speech synthesizer 10 according to an embodiment of the present invention.
In this figure, the speech synthesizer 10 is a terminal device such as a mobile phone having a CPU (Central Processing Unit), a speech input unit 102, and a speaker 142. In the speech synthesizer 10, the CPU executes an application program installed in advance, so that a plurality of functional blocks are constructed as follows.
Specifically, in the speech synthesizer 10, an utterance section detection unit 104, a pitch analysis unit 106, a language analysis unit 108, a speech control unit 109, an answer creation unit (acquisition unit) 110, a speech synthesis unit 112, a language database 122, An answer database 124, an information acquisition unit 126, and an audio library 128 are constructed.
Although not particularly illustrated, the speech synthesizer 10 also includes a display unit, an operation input unit, and the like, so that the user can check the status of the device and input various operations to the device. Can be done. The voice synthesizer 10 is not limited to the terminal device 10 such as a mobile phone, and may be a notebook or tablet personal computer.

Although not described in detail, the audio input unit 102 is a microphone that converts audio into an electric signal, an LPF (low-pass filter) that cuts a high-frequency component of the converted audio signal, and an audio signal that is cut from a high-frequency component. And an A / D converter that converts the signal into a digital signal.
The utterance section detection unit 104 processes the voice signal converted into the digital signal to detect the utterance (sound) section.

The pitch analysis unit 106 performs volume analysis and frequency analysis on the speech of the speech signal detected as the speech interval, and uses the speech data indicating the pitch in a specific interval (first interval) of the speech as speech. This is supplied to the control unit 109.
Here, the first section is, for example, the ending of a statement. The pitch here is, for example, a first formant that is the lowest frequency component among a plurality of formants obtained by frequency analysis of an audio signal. In FIG. This is the frequency (pitch) indicated by the peak band. For frequency analysis, FFT (Fast Fourier Transform) or other known methods can be used. An example of a specific method for specifying the ending in the utterance will be described later.

  On the other hand, the language analysis unit 108 determines which phoneme is close to the speech signal detected as the speech section by referring to a phoneme model created in advance in the language database 122, and the speech specified by the speech signal. Is analyzed (specified), and the analysis result is supplied to the answer creating unit 110.

The answer creation unit 110 creates an answer corresponding to the utterance analyzed by the language analysis unit 108 with reference to the answer database 124 and the information acquisition unit 126.
In the present embodiment, the answer created by the answer creating unit 110 includes:
(1) An answer indicating the affirmation or denial of the statement,
(2) Reply to specific content for remarks,
(3) Reply as a reconciliation to remarks,
Is assumed. Examples of (1) are “Yes”, “No”, etc., and (2) is “Hare, what is tomorrow ’s weather?” There is an example of answering the content specifically. (3) can be “Yes”, “Et”, etc., and the remarks can be answered with “Yes” or “No” as in (1), and specific as in (2) It is created (acquired) in cases other than remarks that need to be answered.

For the answer of (1), for example, in response to an utterance “It is 3 o'clock?”, If the time information is acquired from the built-in real-time clock (not shown), the answer creation unit 110 For example, it is possible to determine which of “Yes” and “No” should be answered.
On the other hand, for example, in response to the statement “Are you sure? (Tomorrow is sunny)?” If the weather information is not acquired by accessing an external server, the speech synthesizer 10 responds alone. I can't. As described above, when the answer cannot be made only by the speech synthesizer 10, the information acquisition unit 126 accesses an external server via the Internet, acquires information necessary for creating the answer, and supplies the information to the answer creation unit 110. Thereby, the said reply creation part 110 can discriminate | determine whether the said comment is correct and can create a reply.
With respect to the answer (2), for example, in response to the statement “Now what? (Now what time?)”, The answer creating unit 110 obtains the time information and sends information other than the time information to the answer database. By acquiring from 124, it is possible to create an answer “I am now XX hour XX minutes”. On the other hand, in response to the statement “What is Asuno Tenki? (Tomorrow's weather?)”, The information acquisition unit 126 accesses an external server to acquire information necessary for an answer, and an answer creation unit. 110 is configured to create, for example, an answer “swell” for the utterance based on the answer database 124 and the acquired information.

The answer creation unit 110 creates and outputs a speech sequence from the created / acquired answers. This speech sequence is a phoneme string and defines a pitch and a sounding timing corresponding to each phoneme.
For the answers (1) and (3), for example, a voice sequence corresponding to the answer is stored in the answer database 124, while a voice sequence corresponding to the determination result is read from the answer database 124. good. Specifically, in the answer (1), the answer creating unit 110 may read a voice sequence such as “Yes” or “No” according to the determination result, and the answer (3) In this case, it is only necessary to read out a speech sequence such as “I think so” or “Et” according to the analysis result of the utterance and the determination result in the answer creation unit 110.
Note that the speech sequences created and acquired by the answer creation unit 110 are supplied to the speech control unit 109 and the speech synthesis unit 112, respectively.

Since the voice sequence defines the pitch of the utterance and the sounding timing, the voice synthesizer 112 can output the basic voice of the answer by simply synthesizing the voice according to the voice sequence. However, since the basic voice of the answer does not take into account the pitch of the ending in the utterance, it is as described above that the machine may give a feeling of being angry.
Therefore, in the present embodiment, the voice control unit 109 changes the pitch of the entire voice sequence by applying a rule as follows. That is, the voice control unit 109 changes a pitch of a specific section (second section) in the voice sequence from the answer creation unit 110 to a pitch having a predetermined relationship with the pitch data (default). Rule). However, if this rule is followed, the answer to be synthesized may be unnatural, so the default rule is appropriately modified according to the speaker attribute and the agent attribute.

  In the present embodiment, the second section is the ending of the answer, but is not limited to the ending as described later. In the present embodiment, as a default rule, the pitch of the ending of the answer is a pitch having a predetermined relationship with the pitch indicated by the pitch data, specifically, a sound having a relationship of 5 degrees below. Although it is defined as high, as will be described later, the pitch may have a relationship other than 5 degrees below. In the present embodiment, the speaker attribute is data that defines the gender of the user. For example, personal information about the user registered in the terminal device as the speech synthesizer 10 is used. The agent attribute is information indicating a virtual personality of the speech synthesizer 10. That is, the agent attribute is data indicating the attribute of the person in order to define what kind of person is supposed to be used for speech synthesis. Here, for simplification of explanation, the agent attribute is described above. Similarly to the above, the data defines gender. The agent attribute is set by the user via the operation input unit.

The speech synthesis unit 112 uses speech segment data registered in the speech library 128 when synthesizing speech. The speech library 128 previously stores speech unit data that defines the waveforms of various speech units that are the source of speech, such as a single phoneme or a transition from phoneme to phoneme, for each of a plurality of agent attributes. Is. Specifically, the speech synthesizer 112 uses speech unit data defined by the agent attribute to combine speech unit data of one sound per phoneme (phoneme) so that the connected portions are continuous. The pitch of the answer is changed according to the rules defined by the voice control unit 109 (including the revised rules), and a voice signal is generated.
Note that the synthesized voice signal is converted into an analog signal by a D / A converter (not shown), then acoustically converted by the speaker 142 and output.

Next, the operation of the speech synthesizer 10 will be described.
FIG. 4 is a flowchart showing a speech synthesis process in the speech synthesizer 10.
First, when a user performs a predetermined operation, for example, when an operation for selecting an icon or the like corresponding to an interactive process is performed on a main menu screen (not shown), the CPU starts an application program corresponding to the process. By executing this application program, the CPU constructs the functional block shown in FIG.

  First, the user inputs a speech by voice to the voice input unit 102 (step Sa11). For example, the utterance section detection unit 104 detects the utterance section by comparing the amplitude of the speech with a threshold value, and supplies the speech signal of the utterance section to each of the pitch analysis unit 106 and the language analysis unit 108 (step Sa12). ).

The language analyzing unit 108 analyzes the meaning of the utterance in the supplied voice signal and supplies data indicating the meaning content to the answer creating unit 110 (step Sa13).
The answer creating unit 110 creates an answer corresponding to the language analysis result of the utterance by using the answer database 124 or acquiring it from an external server via the information acquiring unit 126 as necessary (step Sa14). Then, as described above, the answer creating unit 110 creates a speech sequence based on the answer and supplies it to the speech synthesizing unit 112 (step Sa15).

For example, if the linguistic analysis result of the user's utterance means “Are you tomorrow (is it fine tomorrow?)?”, The answer creation unit 110 accesses the external server and is necessary for the answer. The weather information is acquired, and if the acquired weather information is clear, a sound sequence “Yes” is output, and if it is not clear, a sound sequence “No” is output.
Also, if the language analysis result of the user's utterance is “Asen Tenki is (Tomorrow's weather)?”, The answer creation unit 110, for example, according to the weather information obtained from the external server, Outputs an audio sequence such as “It is cloudy”.
On the other hand, if the linguistic analysis result of the user's utterance means “Ashha Hareka”, it is a monologue (or tweet). Is read from the answer database 124 and output.
The voice control unit 109 specifies the ending pitch (initial pitch) in the voice sequence from the voice sequence supplied from the answer creating unit 110 (step Sa16).

  On the other hand, the pitch analysis unit 106 analyzes the speech signal of the utterance in the detected utterance section, specifies the pitch of the first section (ending part) in the utterance, and converts the pitch data indicating the pitch into voice. It supplies to the control part 109 (step Sa17). Here, an example of a specific method for specifying the ending of a statement in the pitch analysis unit 106 will be described.

Assuming a dialogue in which a person who speaks wants an answer to the comment, the volume corresponding to the tail of the comment is considered to be temporarily higher than the other parts. Therefore, the pitch of the first section (ending) by the pitch analysis unit 106 can be obtained as follows, for example.
First, the pitch analysis unit 106 divides the speech signal detected as an utterance section into a waveform by dividing it into a volume and a pitch (pitch). FIG. 6A is an example of a volume waveform in which the volume of an audio signal is represented on the vertical axis and the elapsed time is represented on the horizontal axis. FIG. 6B is a first waveform obtained by frequency analysis of the same audio signal. It is a pitch waveform in which the pitch of formant is represented on the vertical axis and the elapsed time is represented on the horizontal axis. The time axis of the volume waveform in (a) and the pitch waveform in (b) are common.
Secondly, the pitch analysis unit 106 identifies the timing of the last local maximum P1 in the volume waveform of (a).
Thirdly, the pitch analysis unit 106 determines that a predetermined time range (for example, 100 μsec to 300 μsec) including the timing of the specified maximum P1 before and after is the ending.
Fourth, the pitch analysis unit 106 outputs the average pitch of the section Q1 corresponding to the recognized ending in the pitch waveform of (b) as pitch data.
Thus, it is considered that the erroneous detection of the ending of the speech as a conversation can be reduced by specifying the final maximum P1 of the volume waveform in the utterance section as the timing corresponding to the ending of the speech.
Here, in the volume waveform of (a), a predetermined time range including the timing of the last local maximum P1 before and after is recognized as the ending, but the predetermined time period having the maximum P1 timing as the start or end is determined. The time range may be recognized as the ending. Moreover, it is good also as a structure which outputs not the average pitch of the area Q1 corresponding to the recognized ending but the pitch of the start of the period Q1, the end, and the timing of local maximum P1 as pitch data.

On the other hand, the voice control unit 109 that has received the pitch data executes the following rule correction process (step Sa18).
FIG. 5 is a flowchart showing details of the rule correction processing. First, the voice control unit 109 acquires data indicating speaker attributes and data indicating agent attributes (step Sb11).

Next, the voice control unit 109 determines whether the speaker attribute, that is, whether the user attribute is female or not is based on the acquired data (step Sb12).
If the speaker attribute is female (if the determination result in step Sb12 is “Yes”), the voice control unit 109 sets the pitch of the ending of the answer to 5 degrees with respect to the pitch indicated by the pitch data. For example, the default rule is corrected so that the pitch is 6 degrees below, which is in the relation of the Kyowa pitch one rank lower than the lower pitch. Thereby, the pitch of the ending of the answer is lowered below the pitch defined by the default rule (step Sb13).
Note that the rank here is not a musical meaning, but merely a convenience, and one rank is defined based on a pitch 5 degrees below the pitch indicated by the pitch data. When it is lowered, it means a pitch that is 6 degrees (long 6 degrees), and when it is lowered further, it means a pitch that is 8 degrees below. Also, with the pitch 5 degrees below as a reference, when the rank is increased by 1, the pitch is 3 degrees (long 3 degrees), and when it is further raised, the pitch is 4 degrees above. .
On the other hand, if the speaker attribute of the user is not female (if the determination result in step Sb12 is “No”), the voice control unit 109 determines whether the speaker attribute is male (step Sb12). Sb14).
If the speaker attribute is male (if the determination result of step Sb14 is “Yes”), the voice control unit 109 sets the pitch of the ending of the answer to 3 based on the pitch indicated by the pitch data. Modify the default rule to have a lower pitch. Thereby, the pitch of the ending of the answer is raised above the pitch defined by the default rule (step Sb15).
When the speaker attribute is neutral or the speaker attribute is not registered (when the determination result of step Sb14 is “No”), the voice control unit 109 performs the process of step Sb13 or Sb15. Skip the default rule and leave it unmodified.

Subsequently, the voice control unit 109 determines whether the agent attribute is female (step Sb16). If the agent attribute is female (if the determination result in step Sb16 is “Yes”), the voice control unit 109 sets the pitch at the end of the answer to 1 in the modified rule (or the unmodified default rule). Correction is made to raise the pitch to a higher rank (step Sb17).
For example, the voice control unit 109 has modified the default rule so that the pitch of the ending of the answer is set to a pitch 6 degrees below the pitch indicated by the pitch data in step Sb13. If there is, in step Sb17, the default rule is restored so that the original pitch is 5 degrees below. In addition, since the voice control unit 109 has modified the default rule so that the pitch of the ending of the answer is set to a pitch that is three degrees below the pitch indicated by the pitch data in step Sb15. If there is, in step Sb17, the rule is re-corrected so that the pitch is 4 degrees higher by one rank.
If the process of step Sb13 or Sb15 is skipped, the voice control unit 109 raises the pitch of the ending of the answer by one rank with respect to the pitch indicated by the pitch data in step Sb17. The default rule is modified so that the pitch is 3 degrees below.

On the other hand, if the agent attribute is not female (if the determination result in step Sb16 is “No”), the voice control unit 109 determines whether the attribute of the agent is male (step Sb18). If the agent attribute is male (if the determination result in step Sb18 is “Yes”), the voice control unit 109 increases the pitch of the ending of the answer to a pitch one rank lower in the modified rule. (Step Sb19).
For example, the voice control unit 109 has modified the default rule so that the pitch of the ending of the answer is set to a pitch 6 degrees below the pitch indicated by the pitch data in step Sb13. If there is, in step Sb19, the rule is re-corrected so that the pitch is 8 degrees lower by one rank. In addition, since the voice control unit 109 has modified the default rule so that the pitch of the ending of the answer is set to a pitch that is three degrees below the pitch indicated by the pitch data in step Sb15. If there is, in step Sb19, the default rule is restored so that the original pitch is 5 degrees below. If the process of step Sb13 or Sb15 is skipped, the voice control unit 109 lowers the pitch of the ending of the answer by one rank with respect to the pitch indicated by the pitch data in step Sb19. The default rule is modified so that the pitch is 6 degrees below.

  If the agent attribute is neutral or if the agent attribute is not set (when the determination result in step Sb18 is “No”), the voice control unit 109 skips the process in step Sb17 or Sb19. . After the end of the process of step Sb17 or Sb19 or after skipping, the process procedure proceeds to step Sa19 in FIG.

Next, the voice control unit 109 determines to change the voice sequence supplied from the answer creating unit 110 by applying the rule (or default rule) corrected in step Sa18 (step Sa19). Specifically, when the revised rule stipulates that the pitch of the ending of the answer is a pitch that is, for example, three degrees below the pitch indicated by the pitch data, The control unit 109 adjusts the ending of the answer specified by the voice sequence supplied from the answer creating unit 110 so that the pitch is 3 degrees below the pitch indicated by the pitch data. Decide to shift the pitch of the entire speech sequence.
The voice control unit 109 controls the voice synthesis by the voice synthesis unit 112 with the determined content (step Sa20). As a result, the speech synthesizer 112 synthesizes the speech of the speech sequence determined to be changed by the speech controller 109 with the determined pitch and outputs the synthesized speech.
When the answer sound is output, although not particularly illustrated, the CPU ends the execution of the application program and returns to the menu screen.

  Next, the pitch of the speech, the basic pitch of the voice sequence, and the pitch of the changed voice sequence will be described with specific examples.

The left column of (b) of FIG. 7 is an example of a statement made by the user. In this figure, the linguistic analysis result of the utterance is “Tomorrow is it? (Tomorrow is sunny?)?” And the pitch is shown in the same column for each note. It is an example in case shown by. Note that the pitch waveform of the speech is actually a waveform as shown in FIG. 6B, but here, the pitch is expressed as a note for convenience of explanation.
In the example in this case, as described above, the answer creating unit 110 outputs, for example, a voice sequence of “Yes” if the weather information acquired according to the utterance is sunny, and “ Outputs the “No” audio sequence.
(A) in FIG. 7 is an example of the voice sequence “Yes”. In this example, notes are assigned to each note, and the pitch and pronunciation timing of each word (phoneme) of the basic voice are defined. ing. In this example, for simplicity of explanation, one note is assigned to one note (phoneme), but a plurality of notes may be assigned to one note such as a slur or a tie.

The voice sequence by the answer creating unit 110 is changed by the voice control unit 109 as follows if the default rule is applied. That is, in the utterance shown in the left column of (b), when the pitch of the ending “ka” indicated by the symbol A is indicated by the pitch data as “mi”, the voice control unit 109 , Of the answer “yes”, the entire speech sequence is set so that the pitch of the section “i” at the end indicated by the symbol B becomes “la”, which is 5 degrees below “mi”. Is changed (see the right column of FIG. 7B).
In this embodiment, as a case where the default rule is applied, in FIG. 5, first, when the determination results of steps Sb12, Sb14, Sb16, and Sb18 are all “No”, secondly, When the determination result of step Sb12 is “Yes” and the determination result of step Sb16 is “Yes”, thirdly, the determination result of step Sb12 is “No”, and the determination result of step Sb14 is “Yes”. In addition, there are three cases where the determination result of step Sb16 is “No” and the determination result of step Sb18 is “Yes”.

When the remark is shown in the left column of FIG. 7B, if a modified rule, for example, 6 degrees below, is applied, the speech sequence by the answer creating unit 110 is Will be changed as follows. That is, in the answer “Yes”, the voice sequence is set so that the pitch of the section “Yes” at the end of the word “B” indicated by the symbol B becomes “So” that is six degrees below “Mi”. The overall pitch is changed (see the right column in FIG. 8).
In the present embodiment, as a case where the 6th lower rule is applied, firstly, the determination result in step Sb12 is “Yes”, and the determination results in steps Sb16 and Sb18 are “No”. Secondly, there are two cases: the determination result of steps Sb12 and Sb14 is “No”, the determination result of step Sb16 is “No”, and the determination result of step Sb18 is “Yes”. .

If the remark is shown in the left column of FIG. 7B and the modified rule, for example, 8 degrees below, is applied, the speech sequence by the answer creating unit 110 is Will be changed as follows. That is, among the answers “yes”, the pitch of “i” at the end indicated by the symbol B is “mi”, which is 8 degrees (one octave) lower than “mi”. The pitch of the entire speech sequence is changed (see the right column in FIG. 9).
In the present embodiment, as a case where the rule of 8 degrees below is applied, the determination result of step Sb12 is “Yes”, the determination result of step Sb16 is “No”, and the determination result of step Sb18 is “Yes”. "Is one way.

When the remark is shown in the left column of FIG. 7B, if a modified rule, for example, 3rd down is applied, the voice sequence by the answer creating unit 110 is Will be changed as follows. That is, in the answer “Yes”, the voice sequence is set so that the pitch of the section “Yes” at the end indicated by the symbol B becomes “Do” that is three times lower than “Mi”. The overall pitch is changed (see the right column in FIG. 10).
In the present embodiment, as a case where the rule three times below is applied, first, the determination result in step Sb12 is “No”, the determination result in step Sb14 is “Yes”, and steps Sb16 and Sb18 are performed. The determination result of “No” is “No”, and secondly, the determination result of Steps Sb12 and Sb14 is “No”, and the determination result of Step Sb16 is “Yes”.

When the remark is shown in the left column of FIG. 7B, if a modified rule, for example, 4 degrees above, is applied, the speech sequence by the answer creating unit 110 is Will be changed as follows. That is, in the answer “Yes”, the voice sequence is set so that the pitch of the section “Yes” at the end indicated by the symbol B becomes “La” that is four times higher than “Mi”. The entire pitch is changed (see the right column in FIG. 11).
In this embodiment, when the rule four times higher is applied, the determination result of step Sb12 is “No”, the determination result of step Sb14 is “Yes”, and the determination result of step Sb16 is “ There is one way of being “Yes”.

  Here, “Yes” has been described as an example, but although not particularly illustrated, the pitch of the entire audio sequence is similarly changed even in the case of “No”. In addition, in response to the statement “What is Asuno Tenki?”, For example, when the content is specifically answered as “Hare”, the pitch of the entire speech sequence is similarly changed.

In this embodiment, the default rule is modified so that the pitch of the ending of the answer is in the relationship of the Kyowa interval within one octave with respect to the pitch of the ending of the utterance, so that the answer to the utterance seems unnatural. Do not give the user a feeling.
According to this embodiment, in the default rule in which the pitch of the ending of the answer is 5 degrees below the pitch of the ending of the utterance, if the speaker attribute is female, the pitch is lowered by one rank, If the person attribute is male, the answer is synthesized by speech so that the pitch is increased by one rank. In the default rule, the answer is synthesized by voice so that if the agent attribute is female, the pitch is increased by one rank, and if the agent attribute is male, the pitch is decreased by one rank. Thus, since the pitch of the answer is changed according to the speaker attribute and the agent attribute, a kind of freshness and joy can be given to the user.

<Application examples and modifications>
The present invention is not limited to the above-described embodiments, and various applications and modifications as described below are possible, for example. In addition, one or more arbitrarily selected aspects of application / deformation described below can be appropriately combined.

<Voice input part>
In the embodiment, the voice input unit 102 is configured to input a user's voice (speech) with a microphone and convert the voice signal into a voice signal. However, the voice input unit described in the claims is limited to this configuration. I can't. That is, the voice input unit described in the claims may be configured to input or input a speech by a voice signal in some form. Specifically, the voice input unit described in the claims is configured to input a voice signal processed by another processing unit, a voice signal supplied (or transferred) from another device, It is a concept that includes an input interface circuit or the like that is built in an LSI and that simply receives an audio signal and transfers it to a subsequent stage.

<Audio waveform data>
In the embodiment, the answer creating unit 110 outputs a voice sequence in which a pitch is assigned to each note as an answer to the utterance. However, the answer is output as, for example, voice waveform data in wav format. It is good also as a structure.
In addition, since the sound waveform data is not assigned a pitch to each sound as in the above-described sound sequence, for example, the sound control unit 109 specifies the ending pitch when simply reproduced, Configuration that outputs (reproduces) audio waveform data after performing pitch conversion (pitch conversion) such as filter processing so that the specified pitch has a predetermined relationship with the pitch indicated by the pitch data What should I do?
Also, pitch conversion may be performed by a so-called key control technique that shifts the pitch (pitch) without changing the speaking speed, which is well known in karaoke equipment.

<End of answer, beginning of answer>
In the embodiment, the pitch of the ending of the answer is controlled in accordance with the pitch of the ending of the utterance, but depending on the language, dialect, wording, etc., the part other than the ending of the answer, for example, the beginning of the answer is characteristic. Sometimes it becomes. In such a case, the person who made the speech unconsciously compares the pitch of the ending of the speech with the pitch of the characteristic beginning of the reply, when there is an answer to the speech. Determine the impression of the answer. Therefore, in this case, the pitch at the beginning of the answer may be controlled in accordance with the pitch at the end of the utterance. According to this configuration, when the head of the answer is characteristic, it is possible to give a psychological impression to the user who receives the answer.

  The same applies to utterances, not limited to endings, but may be determined by the beginning of a sentence. In addition, the remarks and answers are not limited to the beginning and end of the word, but may be determined based on an average pitch, or may be determined based on the pitch of the most pronounced portion. For this reason, it can be said that the 1st section of an utterance and the 2nd section of an answer are not necessarily restricted to an initial or ending.

<Speaker attributes>
In the embodiment, the personal information of the user registered in the terminal device as the speech synthesizer 10 is used as the speaker attribute. However, the configuration may be such that detection is performed on the speech synthesizer 10 side. For example, the user's remarks are compared with patterns corresponding to various combinations of gender and age that have been stored in advance by volume analysis and frequency analysis, etc. Can be detected.
If the speaker attribute cannot be detected, steps Sb12 and Sb14 in FIG. 5 are determined as “No”.

<Agent attribute>
In the embodiment, the agent attribute is gender, but three or more types may be combined with gender and age.

<Continuous call of mutual, output timing of mutual, etc.>
By the way, when the dialogue between people is seen from the viewpoint of the gender of the speaker, the following tendency may be observed. For example, women tend to place importance on atmosphere and harmony in dialogue, and tend to excite the place. Specifically, there is a tendency to use a lot of concessions, to call concessions repeatedly, to shorten the time between remarks and answers, and so on. For this reason, if the user is a woman, such a tendency should be expected also for the speech synthesizer 10 that outputs an answer to the speech by speech synthesis. Therefore, if the speaker attribute is female, the voice control unit 109 notifies the answer creating unit 110 to that effect, and the answer creating unit 110 sets the frequency of creating the answer of (3) as a response to the utterance. It may be higher or the same sequence of audio sequences may be output repeatedly. In addition, the voice control unit 109 may control the voice synthesis unit 112 so that the time from the end of the speech by the user to the start of output of the answer is relatively advanced.
On the other hand, in the case of men, there is a tendency to emphasize content, logic, personality, etc. in the dialogue. Specifically, there is a tendency of not using the conflict more than necessary, and depending on the situation, no answer (silently), or lengthening the time between remarks and answers.
Therefore, if the speaker attribute is male, the voice control unit 109 notifies the answer creating unit 110 to that effect, and the answer creating unit 110 lowers the frequency of creating the conflict for the utterance and has a predetermined probability. And it is good as no answer. Further, the voice control unit 109 may control the voice synthesis unit 112 so that the time from the end of the utterance by the user to the start of output of the answer is relatively delayed.

<Pitch relationship>
In the above-described embodiment, the default rule is that the pitch of the ending of the answer is 5 degrees below the ending of the utterance, but is controlled to a relationship of Kyowa intervals other than 5 degrees below. It may be a configuration. For example, as described above, it may be complete 8 degrees, complete 5 degrees, complete 4 degrees, long / short 3 degrees, and long / short 6 degrees.
In addition, even if it is not the relationship of the Kyowa interval, there may be a relationship of the interval that gives a good (or bad) impression empirically, so the pitch of the answer may be controlled based on the relationship of the interval good. However, even in this case, if the pitch between the two pitches of the ending of the utterance and the pitch of the ending of the answer is too far apart, the answer to the utterance tends to become unnatural. It is desirable that the pitch of the answer is in the range of one octave above and below.

Further, in step Sb13 in FIG. 5, as a condition for lowering the pitch of the ending of the answer below the pitch set in the default rule, the speaker attribute is female, It may be weighted that the pitch of the ending is equal to or higher than the first threshold pitch (frequency) (* in step Sb13). This is to avoid an unnatural increase in the answer by speech synthesis when the pitch of a speech by a woman is high.
Similarly, in step Sb15, as a condition for raising the pitch of the ending of the answer above the pitch defined by the default rule, the speaker attribute is male, and further It may be weighted that the pitch such as is equal to or lower than the second threshold pitch (* in step Sb15). This is to avoid an unnaturally low answer by speech synthesis when the pitch of a man's speech is low.

<Others>
In the embodiment, the language analysis unit 108, the language database 122, and the answer database 124, which are configured to obtain responses to utterances, are provided on the side of the speech synthesizer 10, but the processing load is heavy in a terminal device or the like. In view of the above, the storage capacity is limited, and the like may be provided on the external server side. That is, in the speech synthesizer 10, it is sufficient that the answer creating unit 110 obtains an answer to the utterance in some form and outputs data defining the speech of the answer. It does not matter whether it is created on the side or on the side of another configuration (for example, an external server) other than the speech synthesizer 10.
Note that the information acquisition unit 126 is not necessary if the speech synthesizer 10 can create an answer to a statement without accessing an external server or the like.

DESCRIPTION OF SYMBOLS 102 ... Voice input part, 104 ... Speech section detection part, 106 ... Pitch analysis part, 108 ... Language analysis part, 109 ... Speech control part, 110 ... Answer preparation part, 112 ... Speech synthesis part, 126 ... Information acquisition part.

Claims (2)

A voice input unit for inputting a speech by a speaker;
Among the said remarks, a pitch analysis unit that analyzes the pitch of a specific first section;
An acquisition unit for acquiring an answer to the statement;
A voice synthesizer that synthesizes the obtained answer with a predetermined agent attribute;
While controlling the voice synthesis according to a rule that causes the voice synthesizer to change the pitch of a specific second section in the answer to a pitch that has a predetermined relationship with the pitch of the first section. A voice control unit that modifies the rule according to at least one of the speaker attribute of the speaker or the agent attribute;
A speech synthesizer characterized by comprising: Computer
An acquisition unit for acquiring an answer to a comment made by a speaker;
Among the said remarks, a pitch analysis unit that analyzes the pitch of a specific first section,
A speech synthesizer that synthesizes the obtained answer with a predetermined agent attribute; and
While controlling the voice synthesis according to a rule that causes the voice synthesizer to change the pitch of a specific second section in the answer to a pitch that has a predetermined relationship with the pitch of the first section. A voice control unit that modifies the rule according to at least one of the speaker attribute of the speaker or the agent attribute,
A program characterized by functioning as

Images