JP2015069038A - Voice synthesizer and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform voice synthesis with respect to utterance of a user as if it is a conversation with a person.SOLUTION: A voice synthesizer comprises a voice input section 102 to which utterance by a voice signal is input, a non-language analysis section 106 for analyzing a pitch in a specific first section in the utterance and a change state of the pitch in the utterance, an answer creation section 110 for acquiring an answer with respect to the utterance, a voice synthesis section 112 for performing voice synthesis on the acquired answer, and a voice control section 109 for making the voice synthesis section 112 change a pitch in a specific second section in the answer to become a pitch having a predetermined relation with respect to the pitch in the first section and change a change state of the pitch in the answer according to the change state of the pitch in the utterance.

Description

  The present invention relates to a speech synthesizer and a program.

In recent years, the following have been proposed as speech synthesis techniques. That is, by synthesizing and outputting speech corresponding to the user's speech tone and voice quality, a technique for sounding more humanly (see, for example, Patent Document 1), analyzing the user's speech, A technique for diagnosing a health condition or the like (see, for example, Patent Document 2) has been proposed.
In addition, a voice dialogue system that recognizes a voice input by a user and outputs a content specified in a scenario by voice synthesis to realize a voice dialogue with the user has been proposed (for example, Patent Document 3). reference).

JP 2003-271194 A Japanese Patent No. 4495907 Japanese Patent No. 4832097

By the way, it is assumed that a dialogue system that combines the above-described speech synthesis technology and a speech dialogue system to retrieve data and output by speech synthesis in response to a speech by a user's voice is assumed. In this case, a problem has been pointed out that the voice output by the voice synthesis feels unnatural to the user, specifically, the machine sometimes feels roaring.
The present invention has been made in view of such circumstances, and one of its purposes is to give a natural feeling to the user's remarks. An object of the present invention is to provide a speech synthesizer and a program capable of giving an impression that the user is interacting with a person.

  In examining the man-machine system for outputting (replying) the response to the user's utterance by speech synthesis, the present inventor first determines the pitch (frequency) ).

  Here, as a dialogue between people, the other person (assuming b) answers (including conflicts) to one person (assuming a) (including questions, monologues, questions, etc.) consider. In this case, when “a” speaks, not only “a” but also “b” trying to reply to the said speech often leaves the pitch in a certain section of the said speech with a strong impression. b, when replying with consent, approval, or affirmation, the pitch of the utterance that remains in the impression, for example, the pitch of the ending or beginning of the sentence Speak to be in a Kyowa pitch. The person a who heard the answer has the above-mentioned relationship between the pitch that remains in the impression about his / her speech and the pitch of the part that characterizes the answer to the person's reply, so that the answer to b is comfortable and reassuring. The present inventor thought that he had a good impression.

  For example, when “a” says “Yes?”, “A” and “b” are in a state of memorizing the pitch of “Sho” at the end of the said utterance that strongly expresses willingness or confirmation. . In this state, when b tries to affirmatively answer “Yes, yes” to the statement, the part characterizing the answer, for example, the ending, Answer “Yes, yes” so that the pitch of “Yes” is in the above relationship.

FIG. 2 shows a formant in such an actual dialogue. In this figure, the horizontal axis is time, the vertical axis is frequency, and the spectrum shows a state where the intensity increases as it becomes white.
As shown in the figure, a spectrum obtained by frequency analysis of human speech appears as a plurality of peaks that move in time, that is, formants. Specifically, a formant corresponding to “Yeah?” And a formant corresponding to “Ah, yes” each appear as three peak bands (white band-like portions moving along the time axis).
When attention is paid to the first formant having the lowest frequency among these three peak bands, the frequency of the code A (the central part) corresponding to “Sho” of “Oh, right?” Is approximately 400 Hz. On the other hand, for the code B, the frequency of the code B corresponding to “Yes” of “A, Yes” is approximately 260 Hz. For this reason, it can be seen that the frequency of the code A is approximately 3/2 with respect to the frequency of the code B.

  The relationship that the ratio of the frequencies is 3/2 is the relationship between “seo” in the same octave and “mi” in the octave one lower than “mi”. As will be described later, there is a complete 5 degree relationship. This frequency ratio (predetermined relationship between pitches) is a preferred example, but various examples can be given as will be described later.

  FIG. 3 is a diagram showing a relationship between a pitch name (floor name) and a human voice frequency. In this example, the frequency ratio when the fourth octave “do” is used as a reference is also shown, and “so” is 3/2 as described above when “do” is used as a reference. Further, the frequency ratio when the third octave “La” is used as a reference is also illustrated in parallel.

In this way, in the dialogue between people, it can be considered that the pitch of the utterance and the pitch of the answer to be answered are not irrelevant but have the above relationship. Then, the present inventor analyzed many dialogue examples and statistically aggregated evaluations by many people to prove that this idea is roughly correct.
On the other hand, it is empirically recognized that not only the pitch but also the way of answering is different, for example, depending on how the pitch changes. For example, even if the speech utterance is “tomorrow is fine (tomorrow is sunny)”, if the pitch goes up toward the end of the word, for example, the utterance means “is tomorrow sunny?” It becomes a question (question sentence).
In addition, in the remark “tomorrow”, if the pitch is almost constant, the remark is just a monologue or a tweet. For this reason, the pitch (for example) of the answer (consideration) to the said utterance is almost constant.
Therefore, when considering a dialogue system that outputs (replies) a reply to a user's utterance by speech synthesis, not only the pitch of the utterance but also non-linguistic information such as a change in the pitch synthesizes the reply. Above, it can be an important factor.
Therefore, in order to achieve the above object for the speech synthesis, the following configuration is adopted.

That is, in order to achieve the above object, a speech synthesizer according to an aspect of the present invention includes a speech input unit that inputs speech by a speech signal, a pitch of a specific first section of the speech, The non-linguistic analysis unit that analyzes the change in the pitch of the utterance, the acquisition unit that acquires an answer to the utterance, the speech synthesis unit that synthesizes the acquired answer by speech, and the answer to the speech synthesis unit The pitch of the specific second section in the above is changed so that the pitch has a predetermined relationship with the pitch of the first section, and the change in the pitch of the answer is And a voice control unit that changes the pitch according to the change in pitch.
According to this aspect, the pitch of the specific second section in the answer is changed so as to be a pitch having a predetermined relationship with respect to the pitch of the specific first section of the utterance. Further, the change in the pitch of the answer is controlled according to the change in the pitch of the speech. For this reason, it is possible to give the user an impression as if they were interacting with a person.
As an example of control to change the pitch of the answer according to the change in the pitch of the speech, if there is almost no change in the pitch of the speech (if it is flat), the sound of the answer as a match Examples include making the pitch flat, and in the case of a question sentence in which the pitch of the utterance increases toward the end of the sentence, examples such as lowering the pitch of the answer toward the end of the sentence .

In this aspect, it is preferable that the first interval is, for example, the ending of a statement, and the second interval is the beginning or ending of an answer. As described above, the section characterizing the impression impression is the ending of the comment, and the section characterizing the answer impression is often the beginning or ending of the answer.
Moreover, it is preferable that the predetermined relationship is a relationship of Kyowa intervals excluding perfect 1 degree. Here, “Kyowa” means a relationship in which when a plurality of musical sounds are generated at the same time, they are fused and well harmonized, and these pitch relationships are called Kyowa pitches. The degree of cooperation is higher as the frequency ratio (frequency ratio) between two sounds is simpler. The simplest frequency ratios of 1/1 (perfect 1 degree) and 2/1 (perfect 8 degree) are called absolute consonance pitches, and 3/2 (perfect 5 degree) and 4/3 (perfect) 4 degrees) and is called the perfect harmony pitch. 5/4 (3 degrees long), 6/5 (3 degrees short), 5/3 (6 degrees long) and 8/5 (6 degrees short) are called incomplete harmony intervals, and all other frequency ratios This relationship (long and short 2 degrees and 7 degrees, various increase / decrease intervals, etc.) is called dissonance interval.

If the pitch of the beginning or ending of the answer is the same as the pitch of the ending of the utterance, it is considered that there is an unnatural feeling as a dialogue. Is excluded.
In the above aspect, the most desirable example of the predetermined relationship is considered to be a relationship in which the pitch of the second section is 5 degrees below the pitch of the first section as described above. However, the predetermined relationship is not limited to a consonant pitch except for a perfect degree, but may be a dissonant pitch relationship, or may be a pitch relationship within the range of one octave above and below, excluding the same.
In addition, the answer is not limited to a specific answer to the question, but includes an answer (interjection) such as “I see” or “I think so”.

In the above aspect, the non-linguistic analysis unit also analyzes changes in non-linguistic information other than the pitch of the utterance, and the speech control unit performs the speech synthesis on changes in non-linguistic information other than the pitch. It is good also as a structure controlled according to.
According to this configuration, the speech synthesis of the answer is controlled according to the change in the non-linguistic information other than the pitch, so that it is possible to give the user the impression that they are interacting with a person more. Become. An example of non-linguistic information other than the pitch is typically the volume, and the average volume of the answer is controlled according to the volume of the utterance or the answer volume is changed according to the change in the volume of the utterance. It is also possible to control how the volume changes. In addition, the speed of speech (speaking speed) can be cited, and the speaking speed of answers may be controlled in accordance with the speaking speed of speech.

The aspect of the present invention can be conceptualized as a program that causes a computer to function as the speech synthesizer as well as the speech synthesizer.
In the present invention, the pitch (frequency) of the speech is the analysis target, and the pitch of the answer is the control target. As is clear from the above-described formant example, human speech has a certain frequency range. Therefore, it is inevitable that the analysis and control have a certain frequency range. In addition, as a matter of course, errors occur in analysis and control. For this reason, in this case, the pitch analysis and control are allowed not only to have the same numerical value of the pitch (frequency) but also to have a certain range.

It is a block diagram which shows the structure of the speech synthesizer which concerns on embodiment. It is a figure which shows the example of the sound formant in a dialog. It is a figure which shows the relationship between a pitch name, a frequency, etc. It is a flowchart which shows the speech synthesis process in a speech synthesizer. It is a figure which shows the specific example of a ending and a pitch change. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a block diagram which shows the structure of the speech synthesizer which concerns on a modification. It is a figure which shows the principal part of the process in an application example (the 1). It is a figure which shows the principal part of the process in an application example (the 2). It is a figure which shows the principal part of the process in an application example (the 3).

Embodiments of the present invention will be described below with reference to the drawings.
<Speech synthesizer>

FIG. 1 is a diagram showing a configuration of a speech synthesizer 10 according to an embodiment of the present invention.
In this figure, the speech synthesizer 10 is a terminal device such as a mobile phone having a CPU (Central Processing Unit), a speech input unit 102, and a speaker 142. In the speech synthesizer 10, the CPU executes an application program installed in advance, so that a plurality of functional blocks are constructed as follows.
Specifically, in the speech synthesizer 10, the speech segment detection unit 104, the non-language analysis unit 106, the language analysis unit 108, the speech control unit 109, the answer creation unit (acquisition unit) 110, the speech synthesis unit 112, the language database 122, An answer database 124, an information acquisition unit 126, and an audio library 128 are constructed.
Although not particularly illustrated, the speech synthesizer 10 also includes a display unit, an operation input unit, and the like, so that the user can check the status of the device and input various operations to the device. Can be done. The voice synthesizer 10 is not limited to the terminal device 10 such as a mobile phone, and may be a notebook or tablet personal computer.

Although not described in detail, the audio input unit 102 is a microphone that converts audio into an electric signal, an LPF (low-pass filter) that cuts a high-frequency component of the converted audio signal, and an audio signal that is cut from a high-frequency component. And an A / D converter that converts the signal into a digital signal.
The utterance section detection unit 104 processes the voice signal converted into the digital signal to detect the utterance (sound) section.

The non-linguistic analysis unit 106 performs volume analysis and frequency analysis on the speech of the speech signal detected as the speech segment, and uses the specific segment (first segment) of the speech as non-language information other than the language information in the speech. ) And the data indicating the change in the pitch in the utterance.
The data indicating the pitch is supplied to the voice control unit 109, and the data indicating the change in the pitch is supplied to the voice control unit 109 and the answer creating unit 110, respectively.
Here, the first section is, for example, the ending of a statement. The pitch here is, for example, a first formant that is the lowest frequency component among a plurality of formants obtained by frequency analysis of an audio signal. In FIG. This is the frequency (pitch) indicated by the peak band. For frequency analysis, FFT (Fast Fourier Transform) or other known methods can be used. An example of a specific method for specifying the ending in the utterance and a specific method for specifying the degree of change in pitch will be described later.

  On the other hand, the language analysis unit 108 determines which phoneme is close to the speech signal detected as the speech section by referring to a phoneme model created in advance in the language database 122, and the speech specified by the speech signal. Is analyzed (specified), and the analysis result is supplied to the answer creating unit 110.

The answer creation unit 110 uses the answer database 124 and the information acquisition unit 126 while using the data indicating the degree of change in the pitch analyzed by the non-language analysis unit 106 for the answer corresponding to the utterance analyzed by the language analysis unit 108. Create by referring to.
In the present embodiment, the answer created by the answer creating unit 110 includes:
(1) An answer indicating the affirmation or denial of the statement,
(2) Reply to specific content for remarks,
(3) Reply as a reconciliation to remarks,
Is assumed. Examples of (1) are “Yes”, “No”, etc., and (2) is “Hare, what is tomorrow ’s weather?” There is an example of answering the content specifically. (3) can be “Yes”, “Et”, etc., and the remarks can be answered with “Yes” or “No” as in (1), and specific as in (2) It is created (acquired) in cases other than remarks that need to be answered.

For the answer of (1), for example, in response to an utterance “It is 3 o'clock?”, If the time information is acquired from a built-in real-time clock (not shown), the answer creation unit 110 responds to the utterance. For example, it is possible to determine which of “Yes” and “No” should be answered.
On the other hand, for example, in response to the statement “Are you sure? (Tomorrow is sunny)?” If the weather information is not acquired by accessing an external server, the speech synthesizer 10 responds alone. I can't. As described above, when the answer cannot be made only by the speech synthesizer 10, the information acquisition unit 126 accesses an external server via the Internet, acquires information necessary for creating the answer, and supplies the information to the answer creation unit 110. Thereby, the said reply creation part 110 can discriminate | determine whether the said comment is correct and can create a reply.
With respect to the answer (2), for example, in response to the statement “Now what? (Now what time?)”, The answer creating unit 110 obtains the time information and sends information other than the time information to the answer database. By acquiring from 124, it is possible to create an answer “I am now XX hour XX minutes”. On the other hand, in response to the statement “What is Asuno Tenki? (Tomorrow's weather?)”, The information acquisition unit 126 accesses an external server to acquire information necessary for an answer, and an answer creation unit. 110 is configured to create, for example, an answer “swell” for the utterance with reference to the answer database 124.

The answer creation unit 110 creates and outputs a speech sequence from the created / acquired answers. This speech sequence is a phoneme string and defines a pitch and a sounding timing corresponding to each phoneme.
For the answers (1) and (3), for example, a voice sequence corresponding to the answer is stored in the answer database 124, while a voice sequence corresponding to the determination result is read from the answer database 124. good. Specifically, in the answer (1), the answer creating unit 110 may read a voice sequence such as “Yes” or “No” according to the determination result, and the answer (3) In this case, it is only necessary to read out a speech sequence such as “I think so” or “Et” according to the analysis result of the utterance and the determination result in the answer creation unit 110.
Note that the speech sequences created and acquired by the answer creation unit 110 are supplied to the speech control unit 109 and the speech synthesis unit 112, respectively.

  The voice control unit 109 determines the control content for the voice sequence according to the pitch data supplied from the non-linguistic analysis unit 106 and the data indicating how the pitch changes in the speech.

Since the voice sequence defines the pitch of the utterance and the sounding timing, the voice synthesizer 112 can output the basic voice of the answer even if the voice synthesizer 112 simply synthesizes the voice according to the voice sequence. However, since the basic voice of the answer does not take into account the pitch of the ending in the utterance, it is as described above that the machine may give a feeling of being angry.
Therefore, in this embodiment, the speech synthesizer 112 synthesizes speech by changing the basic speech defined by the speech sequence as follows according to the control content of the speech controller 109. That is, the voice control unit 109 changes the pitch of a specific section (second section) in the voice sequence so that the pitch has a predetermined relationship with the pitch data, and reaches the ending. In some cases, the change in pitch up to is changed according to data indicating the change in pitch in the speech.
In the present embodiment, the second section is the ending of the answer, but is not limited to the ending as described later. In this embodiment, the pitch having a predetermined relationship with the pitch data is set to a pitch having a relationship of 5 degrees below, but as will be described later, a sound having a relationship other than 5 degrees is used. It can be high.

The speech synthesizer 112 uses speech unit data registered in the speech library 128 when synthesizing speech. The speech library 128 is a database of speech unit data defining waveforms of various speech units that are speech materials, such as a single phoneme or a transition part from a phoneme to a phoneme. Specifically, the speech synthesizer 112 combines the speech segment data of one sound per phoneme (phoneme) and corrects the connected portion to be continuous, and corrects the ending sound of the answer as described above. Change the height to generate an audio signal.
Note that the synthesized voice signal is converted into an analog signal by a D / A converter (not shown), then acoustically converted by the speaker 142 and output.

Next, the operation of the speech synthesizer 10 will be described.
FIG. 4 is a flowchart showing a speech synthesis process in the speech synthesizer 10.
First, when a user performs a predetermined operation, for example, when an operation for selecting an icon or the like corresponding to an interactive process is performed on a main menu screen (not shown), the CPU starts an application program corresponding to the process. By executing this application program, the CPU constructs the functional block shown in FIG.

First, the user inputs a speech by voice to the voice input unit 102 (step Sa11). The utterance section detection unit 104 detects the utterance section by comparing the amplitude of the speech with a threshold value, and supplies the speech signal of the utterance section to each of the non-linguistic analysis unit 106 and the language analysis unit 108 (step Sa12). .
The language analysis unit 108 performs language analysis on the utterances in the supplied voice signal, and supplies data indicating the meaning to the answer creation unit 110 (step Sa13).

  On the other hand, the non-linguistic analysis unit 106 analyzes the voice signal in the detected utterance section, and divides it into a sound volume and a pitch (pitch) to form a waveform (step Sa14). FIG. 5A is an example of a volume waveform in which the volume of an audio signal is represented on the vertical axis and the elapsed time is represented on the horizontal axis. FIG. 5B is a first waveform obtained by frequency analysis of the same audio signal. It is a pitch waveform in which the pitch of formant is represented on the vertical axis and the elapsed time is represented on the horizontal axis. The time axis of the volume waveform in (a) and the pitch waveform in (b) are common.

Assuming a dialogue in which the user wants an answer to the utterance, it is considered that the volume of the portion corresponding to the ending of the utterance temporarily increases compared to the other portions. Therefore, the pitch of the first section (end of word) by the non-language analyzing unit 106 is obtained as follows, for example, and pitch data indicating the pitch is output (step Sa15).
That is, first, the non-language analyzing unit 106 specifies the timing of the last maximum P1 in terms of time in the volume waveform of FIG.
Second, the non-linguistic analysis unit 106 determines that a predetermined time range (for example, 100 μsec to 300 μsec) including the timing of the specified maximum P1 before and after is the ending Q1.
Thirdly, the non-linguistic analysis unit 106 obtains an average pitch N1 of a section corresponding to the recognized ending Q1 in the pitch waveform of (b), and outputs pitch data indicating the pitch N1. .
As described above, it is considered that the erroneous detection of the utterance ending Q1 as a conversation can be reduced by specifying the final maximum P1 of the volume waveform in the utterance section as the timing corresponding to the ending of the utterance.
Here, in the volume waveform of (a), the time range including the timing of the last maximum P1 in time is recognized as the ending Q1, but the time range having the timing of the maximum P1 as the start or end is determined. It may be recognized as ending Q1. Moreover, it is good also as a structure which outputs not the average pitch of the area corresponding to the recognized ending, but the beginning and end of ending Q1, and the timing of the maximum P1 timing as pitch data.

In parallel with the specification of the ending pitch of the utterance, the non-linguistic analysis unit 106 specifies the change in the pitch in the utterance as follows, for example (step Sa16).
That is, first, the non-linguistic analysis unit 106 obtains a pitch N0 at a timing P0 that is back by the time Ts (eg, 0.3 seconds) from the timing of the maximum P1 of the volume waveform in FIG.
Second, the non-linguistic analysis unit 106 obtains a pitch change (N1-N0) from the pitch N0 to the pitch N1, and supplies it to the voice control unit 109 and the answer creation unit 110.
Note that the timing P0 may be regarded as a change in pitch from the beginning to the end of the statement as a timing corresponding to the beginning of the statement, or a change pattern in the pitch of each word of the statement, It may be understood as a change in pitch in a statement.

  On the other hand, the answer creating unit 110 uses the answer database 124 to refer to the answer corresponding to the linguistic analysis result of the utterance with reference to the change in pitch of the utterance, or from an external server via the information acquisition unit 126 as necessary. Or obtain it (step Sa17). Then, the answer creating unit 110 creates a speech sequence based on the answer and supplies it to the speech synthesizing unit 112 (step Sa18).

For example, even if the language analysis result of a user's utterance is “Ace Hare”, if the pitch of the utterance rises toward the end of the word, the utterance will mean “Sunny tomorrow?” It becomes a question (question sentence). For this reason, the answer creation unit 110 accesses an external server to acquire the weather information necessary for the answer. If the acquired weather information is sunny, the voice sequence “Yes” is displayed. Are output respectively.
On the other hand, even if the linguistic analysis result of the user's utterance is “Tomorrow”, if the pitch of the utterance is flat toward the end of the utterance or decreases, the utterance will be “Tomorrow ( It ’s a monologue (or tweet) that means “ For this reason, the answer creating unit 110 reads out and outputs a speech sequence such as “That's right” from the answer database 124.
Note that the answer creation unit 110 determines that the pitch of the speech is rising toward the end of the speech if, for example, the change in pitch of the speech exceeds the threshold, and if the pitch of the speech is below the threshold, the pitch of the speech is Judged as flat (or down) toward the end of the word.
Also, if the language analysis result of the user's utterance is “What is Asuno Tenki?”, The answer creation unit 110 may, for example, “are it” or “is cloudy” according to the weather information acquired from the external server. An audio sequence like this is output.

The voice control unit 109 specifies the ending pitch (initial pitch) in the voice sequence from the voice sequence supplied from the answer creating unit 110 (step Sa19).
Next, the voice control unit 109 changes the pitch of the voice sequence based on the pitch data supplied from the non-linguistic analysis unit 106 and the data indicating the change in pitch as follows. Determine (step Sa20). Specifically, the voice control unit 109 determines that the initial pitch of the ending specified in the voice sequence is the pitch indicated by the pitch data if the pitch of the utterance by the user increases toward the ending. Then, it is determined that the pitch of the entire speech sequence is to be changed so that the relationship is 5 degrees below. On the other hand, if the pitch of the speech by the user is flat (or lowered) toward the end of the speech, the voice control unit 109 converts all the pitches of the speech sequence to 5 degrees lower than the above. Decide to change to pitch.
The voice control unit 109 controls voice synthesis by the voice synthesis unit 112 with the determined content (step Sa21). As a result, the speech synthesizer 112 synthesizes the speech of the speech sequence determined to be changed by the speech controller 109 with the determined pitch and outputs the synthesized speech.
When the answer sound is output, although not particularly illustrated, the CPU ends the execution of the application program and returns to the menu screen.

  Next, the pitch of the utterance, the change in pitch, and the change of the voice sequence will be described with specific examples.

The left column in (b) of FIG. 6 is an example of a statement made by the user. This figure shows an example in which the language analysis result of the utterance is “Tomorrow” and the pitch of each utterance of the utterance is indicated by a note shown in the same column. Note that the pitch waveform of the utterance is actually a waveform as shown in FIG. 5B, but here the pitch is expressed as a note for convenience of explanation. In the example of this figure, since the pitch of the utterance rises toward the end of the word, the answer creating unit 110 determines that the utterance is a question (question sentence). For this reason, as described above, the answer creating unit 110 outputs, for example, an audio sequence of “Yes” if the weather information acquired in response to the statement is sunny, and “No” if the weather information is not sunny. Output an audio sequence.
FIG. 6 (a) is an example of a “Yes” speech sequence. In this example, notes are assigned to each note and the pitch and pronunciation timing of each word (phoneme) of the basic speech are defined. ing. In this example, for simplicity of explanation, one note is assigned to one note (phoneme), but a plurality of notes may be assigned to one note such as a slur or a tie.
This voice sequence is changed by the voice control unit 109 as follows.
That is, in the utterance shown in the left column of (b), when the pitch of the ending “re” section indicated by the symbol A is indicated as “So” by the pitch data, the voice control unit 109 , Of the answer “yes”, the entire speech sequence is set so that the pitch of the section “i” at the end indicated by the symbol B becomes “do” which is 5 degrees below “so”. (See the right column of FIG. 6B).
Here, “Yes” has been described as an example, but although not particularly illustrated, the pitch of the entire audio sequence is similarly changed even in the case of “No”. In addition, in response to the statement “What is Asuno Tenki?”, For example, when the content is specifically answered as “Hare”, the pitch of the entire speech sequence is similarly changed.

  In this way, the answer is synthesized so that the pitch at the end of the answer is 5 degrees lower than the pitch at the end of the comment, so that it does not give the user an unnatural feeling. You can give a good impression as if you were interacting.

On the other hand, even if the language analysis result is “tomorrow”, as shown in the left column of FIG. 7B, if the pitch of the utterance is flat toward the ending, 110 determines that the utterance is a monologue or the like. Therefore, as described above, the answer creating unit 110 outputs, for example, a sound sequence of “Yes”.
FIG. 7A shows an example of a sound sequence of “Yes”.
This voice sequence is changed by the voice control unit 109 as follows.
That is, among the utterances shown in the left column of (b) in the figure, when the pitch of the ending “re” section indicated by the symbol A is indicated as “So” by the pitch data, the voice control The part 109 makes all the pitches of the answer “Yes” (including the ending “ne” indicated by the symbol B) to “do”, which is a pitch 5 degrees below “so”. As described above, the pitch of the voice sequence is changed (see the right column of FIG. 7B).

Also in this case, the answer is synthesized with speech so that the pitch of the ending of the answer as a companion is 5 degrees lower than the pitch at the end of the utterance, which makes the user feel unnatural. Without giving, you can give a good impression as if you were talking.
Moreover, in this embodiment, even if the linguistic language analysis results are the same, an answer is created according to the pitch change toward the ending in the utterance. Further, if the pitch of the speech is flat, the pitch of the companion to the speech is also flattened, that is, the change in the pitch defined by the original speech sequence is changed. For this reason, it is possible to give the user an impression as if they were interacting with a person, not with a machine.

<Application examples and modifications>
The present invention is not limited to the above-described embodiments, and various applications and modifications as described below are possible, for example. In addition, one or more arbitrarily selected aspects of application / deformation described below can be appropriately combined.

<Voice input part>
In the embodiment, the voice input unit 102 is configured to input a user's voice (speech) with a microphone and convert the voice signal into a voice signal. However, the voice input unit described in the claims is limited to this configuration. I can't. That is, the voice input unit described in the claims may be configured to input or input a speech by a voice signal in some form. Specifically, the voice input unit described in the claims is configured to input a voice signal processed by another processing unit, a voice signal supplied (or transferred) from another device, It is a concept that includes an input interface circuit or the like that is built in an LSI and that simply receives an audio signal and transfers it to a subsequent stage.

<Audio waveform data>
In the embodiment, the answer creating unit 110 outputs a voice sequence in which a pitch is assigned to each note as an answer to the utterance. However, the answer is output as, for example, voice waveform data in wav format. It is good also as a structure.
In addition, since the sound waveform data is not assigned a pitch to each sound as in the above-described sound sequence, for example, the sound control unit 109 specifies the ending pitch when simply reproduced, Configuration that outputs (reproduces) audio waveform data after performing pitch conversion (pitch conversion) such as filter processing so that the specified pitch has a predetermined relationship with the pitch indicated by the pitch data What should I do?
Also, pitch conversion may be performed by a so-called key control technique that shifts the pitch (pitch) without changing the speaking speed, which is well known in karaoke equipment.

<End of answer, beginning of answer>
In the embodiment, the pitch of the ending of the answer is controlled in accordance with the pitch of the ending of the utterance, but depending on the language, dialect, wording, etc., the part other than the ending of the answer, for example, the beginning of the answer is characteristic. Sometimes it becomes. In such a case, the person who made the statement, when there is an answer to the statement, unconsciously compares the pitch of the statement and the pitch of the characteristic beginning of the answer. Judge the impression. Therefore, in this case, the pitch at the beginning of the answer may be controlled in accordance with the pitch at the end of the utterance. According to this configuration, when the head of the answer is characteristic, it is possible to give a psychological impression to the user who receives the answer.

  The same applies to utterances, not limited to endings, but may be determined by the beginning of a sentence. In addition, the remarks and answers are not limited to the beginning and end of the word, but may be determined based on an average pitch, or may be determined based on the pitch of the most pronounced portion. For this reason, it can be said that the 1st section of an utterance and the 2nd section of an answer are not necessarily restricted to an initial or ending.

<Changes in the pitch of responses, average pitch of responses, etc.>
In the embodiment, by shifting the pitch of the entire speech sequence such that the pitch of the ending of the answer is 5 degrees below the pitch of the ending of the utterance, or By flattening the pitch of the voice sequence, the pitch of the answer defined in the original voice sequence and the average pitch of the answer are changed as a result.
For example, if the pitch of the utterance is rising toward the ending, the pitch of the answer is lowered toward the ending, and the pitch of the utterance is decreasing toward the ending. If so, the pitch of the original voice sequence may be changed so that the pitch of the answer rises toward the end of the answer.
In addition, the entire average pitch of the answer is changed in accordance with the pitch of the ending of the utterance or the change in the pitch of the utterance. Also good.

<Speak volume, volume change, answer volume, volume change>
In the embodiment, the ending of the utterance is specified by using the volume change of the utterance. However, various uses of the utterance volume can be considered as non-language information other than the pitch. For example, it may be configured to control the volume of the answer that is synthesized by speech according to the average volume of the speech. Moreover, it is good also as a structure which controls the volume change of an answer according to the volume change (amplitude envelope) of an utterance.

<Content of conversation>
In the embodiment, the operation is terminated when the speech synthesizer 10 outputs an answer by speech synthesis in response to the user's utterance. However, in the dialogue between people, the utterance and the answer to the utterance are terminated. Rather, the remarks and answers are often repeated, and the number of repetitions increases or decreases depending on the semantic content of the remarks and answers.
Therefore, as shown in FIG. 8, the language analysis unit 108 performs linguistic analysis not only on the user's remarks but also on the response created by the response creation unit 110, and supplies the language analysis result to the voice control unit 109. The voice control unit 109 may control the pitch of the answer ending, the change in the answer pitch, the average answer pitch, and the like.

<Pitch relationship>
In the embodiment described above, the voice synthesis is controlled so that the pitch of the ending of the answer is 5 degrees lower than the ending of the utterance, but is controlled to have a relationship of Kyowa intervals other than 5 degrees below. It may be a configuration. For example, as described above, it may be complete 8 degrees, complete 5 degrees, complete 4 degrees, long / short 3 degrees, and long / short 6 degrees.
In addition, there is a case where a relationship of a pitch that gives a good (or bad) impression is empirically recognized even if it is not a relationship of the Kyowa pitch, so that the pitch of the answer is controlled according to the relationship of the pitch. good. However, even in this case, if the pitch between the two pitches of the ending of the utterance and the pitch of the ending of the answer is too far apart, the answer to the utterance tends to become unnatural. It is desirable that the pitch of the answer is in the range of one octave above and below.

<Pitch shift of answer>
By the way, in the configuration in which the pitch of the ending of the answer specified by the speech sequence or the like is controlled so as to have a predetermined relationship with the pitch of the ending of the utterance, for example, as in the embodiment, for example, In the configuration in which the pitch is changed to be 5 degrees below, if the pitch to be changed is too low, the answer may be synthesized with an unnatural low tone. Next, application examples (No. 1 and No. 2) for avoiding such a case will be described.

FIG. 9 is a diagram showing a main part of processing in the application example (part 1). Note that the main part of the process mentioned here refers to a process executed in “determination of answer pitch” in step Sa20 in FIG. That is, in the application example (No. 1), the processing shown in FIG. 9 is executed in step Sa20 shown in FIG. 4, and the details are as follows.
First, the voice control unit 109 temporarily determines a pitch such as a ending of the answer to a pitch that is 5 degrees below the pitch indicated by the pitch data from the non-linguistic analysis unit 106 ( Step Sb171).
Next, the voice control unit 109 determines whether or not the temporarily determined pitch is lower than a predetermined threshold pitch (step Sb172). Note that the threshold pitch is set to a pitch corresponding to the lower limit frequency when speech synthesis is performed, or to a pitch that gives an unnatural feeling if it is lower than this.

If the tentatively determined pitch, that is, the pitch 5 degrees below the ending pitch in the utterance is lower than the threshold pitch (if the determination result in step Sb172 is “Yes”), the voice control unit 109 The temporarily determined pitch is shifted to a pitch one octave higher (step Sb173).
On the other hand, if the temporarily determined pitch is equal to or higher than the threshold pitch (if the determination result in step Sb172 is “No”), the process in step Sb173 is skipped.
Then, the voice control unit 109 determines the target ending pitch at the time of shifting the answer to the following pitch (step Sb174). That is, if the tentatively determined pitch is lower than the threshold pitch, the voice control unit 109 sets the tentatively determined pitch to one pitch above, and the tentatively determined pitch is equal to or higher than the threshold pitch. If so, the target pitches are finally determined without changing the temporarily determined pitches.
After step Sb174, the processing procedure proceeds to step Sa21 in FIG. 4, and the voice control unit 109 shifts the pitch of the ending of the answer to the determined pitch as the control content. The control content is determined, whereby the speech synthesizer 112 synthesizes and outputs speech sequence speech with the determined control content.

According to this application example (No. 1), if the pitch to be changed is lower than the threshold pitch, the pitch is shifted so as to be one octave higher than the pitch. It is possible to avoid the point that the answer is synthesized by voice.
In this example, the pitch of the ending of the answer is shifted to a pitch one octave higher, but may be shifted to a pitch one octave lower. Specifically, since the pitch of the ending of the utterance of the user's utterance is high, if the pitch 5 degrees below the pitch is too high, the answer is synthesized with an unnatural high tone. . In order to avoid this, if the pitch (temporarily determined pitch) that is 5 degrees below the pitch indicated by the pitch data is higher than the threshold pitch, the pitch of the ending of the answer is set. The pitch may be shifted to a pitch one octave lower than the temporarily determined pitch.

  In addition, when voice synthesis is performed, it may be possible to output a voice of a virtual character in which sex or age (child / adult) is determined. When a female or child character is specified as in this case, if the pitch is lowered to 5 degrees below the ending of the speech, the answer is synthesized with a low sound that is not suitable for the character. Therefore, similarly, it may be configured to shift so that the pitch becomes one octave higher.

  FIG. 10 is a diagram showing a main part of processing in such an application example (part 2), and shows processing executed in “answer pitch determination” in step Sa20 in FIG. The explanation will focus on the points different from FIG. 9. In step Sb171, the voice control unit 109 temporarily calculates a pitch that is 5 degrees below the pitch indicated by the pitch data from the non-linguistic analysis unit 106. After the determination, it is determined whether or not a woman or a child is designated as an attribute that defines the character (step Sc172).

If female or child is specified as the attribute (if the determination result in step Sc172 is “Yes”), the voice control unit 109 shifts the temporarily determined pitch to a pitch one octave higher (step S109). Sb173) On the other hand, if female or child is not specified as the attribute, for example, if male or adult is specified (if the determination result of step Sc172 is “No”), the process of step Sb173 is skipped. Is done. The subsequent steps are the same as in the application example (No. 1).
According to this application example (No. 2), if it is set to answer with a voice of a woman or a child, the pitch is shifted to an octave higher than the temporarily determined pitch. Thus, it is possible to avoid the problem that the answer is synthesized with an unnatural low tone while maintaining the pitch relationship.
In this example, if female or child is specified as an attribute, the pitch is shifted to one octave above. However, for example, if an adult male is specified as an attribute, the character corresponding to the attribute is not recognized. In order to avoid that the answer is synthesized with high-pitched sounds, the pitch may be shifted to a pitch one octave below.

<Dissonance>
In the embodiment described above, the voice synthesis is controlled so that the pitch of the ending of the answer is related to the harmonious pitch with respect to the ending of the utterance, but the voice synthesis is made so as to be related to the dissonant pitch. May be controlled. It should be noted that if the answers are synthesized with pitches that are in a dissonant pitch relationship, the user who gave the speech will be given an unnatural feeling, a bad impression, a rough feeling, etc., and a smooth dialogue will not be established. There are also concerns, but there is a view that this feeling is good for stress relief.
Therefore, as an operation mode, a mode (first mode) for which a response such as a good impression is desired and a mode (second mode) for which a response such as a bad impression is desired are prepared, and speech synthesis is controlled according to any mode. It is good also as composition to do.

  FIG. 11 is a diagram showing a main part of processing in such an application example (part 3), and shows processing executed in “answer pitch determination” in step Sa20 in FIG. The description will focus on the points different from FIG. 9. The sound control unit 109 determines whether or not the first mode is set as the operation mode (step Sd172).

If the first mode is set as the operation mode (if the determination result in step Sd172 is “Yes”), the voice control unit 109 changes the pitch of the answer, for example, to the pitch of the utterance, for example. On the other hand, the pitch is determined so as to be a pitch having a relation of Kyowa pitch (step Sd173A). On the other hand, if the second mode is set as the operation mode (if the determination result in step Sd172 is “No”), the voice control unit 109 changes the pitch of the answer ending to the pitch of the utterance ending. On the other hand, the pitch is determined to be a pitch having a dissonant pitch relationship (step Sd173B). The subsequent processes are the same as those of the application example (part 1) and the application example (part 2).
Therefore, according to this application example (No. 3), if the first mode is set, speech synthesis is performed with a pitch in which the ending of the answer is related to the pitch of the Kyowa pitch with respect to the pitch of the ending of the utterance. If the second mode is set, voice synthesis is performed with a pitch in which the ending of the answer has a dissonant pitch with respect to the pitch of the ending of the utterance, so that the user can use the operation mode appropriately. It will be possible.

  The application example (part 1), the application example (part 2), and the application example (part 3) have been described as examples using voice sequences. However, it goes without saying that voice waveform data may be used. is there.

<Others>
In the embodiment, the language analysis unit 108, the language database 122, and the answer database 124, which are configured to obtain responses to utterances, are provided on the side of the speech synthesizer 10, but the processing load is heavy in a terminal device or the like. In view of the above, the storage capacity is limited, and the like may be provided on the external server side. That is, in the speech synthesizer 10, it is sufficient that the answer creating unit 110 obtains an answer to the utterance in some form and outputs data defining the speech of the answer. It does not matter whether it is created on the side or on the side of another configuration (for example, an external server) other than the speech synthesizer 10.
Note that the information acquisition unit 126 is not necessary if the speech synthesizer 10 can create an answer to a statement without accessing an external server or the like.

DESCRIPTION OF SYMBOLS 102 ... Voice input part, 104 ... Speech area detection part, 106 ... Non-language analysis part, 108 ... Language analysis part, 109 ... Speech control part, 110 ... Answer preparation part, 112 ... Speech synthesis part, 126 ... Information acquisition part.

Claims (3)

A voice input unit for inputting a speech by a voice signal;
Among the utterances, a non-linguistic analysis unit that analyzes the pitch of a specific first section and how the pitch of the utterance changes,
An acquisition unit for acquiring an answer to the statement;
A speech synthesizer that synthesizes the obtained answers,
The voice synthesizer is caused to change the pitch of the specific second section in the answer so that the pitch has a predetermined relationship with the pitch of the first section, and the sound of the answer A voice control unit that changes the degree of change in height according to the degree of change in the pitch of the speech;
A speech synthesizer characterized by comprising: The non-linguistic analysis unit also analyzes changes in non-linguistic information other than the pitch of the utterance,
The voice control unit
Controlling the speech synthesis in accordance with a change in non-linguistic information other than the pitch,
The speech synthesizer according to claim 1. Computer
A non-linguistic analysis unit that analyzes a pitch of a specific first section and a change in a pitch of the utterance among utterances by an input voice signal;
An acquisition unit for acquiring an answer to the statement;
A speech synthesizer that synthesizes the obtained answers, and
The voice synthesizer is caused to change the pitch of the specific second section in the answer so that the pitch has a predetermined relationship with the pitch of the first section, and the sound of the answer A voice control unit for changing a change in height according to a change in the pitch of the speech;
A program characterized by functioning as

Images