JP6375605B2 - Voice control device, voice control method and program - Google Patents

Description

The present invention relates to a voice control device , a voice control method, and a program.

In recent years, the following have been proposed as speech synthesis techniques. That is, by synthesizing and outputting speech corresponding to the user's speech tone and voice quality, a technique for sounding more humanly (see, for example, Patent Document 1), analyzing the user's speech, A technique for diagnosing a health condition or the like (see, for example, Patent Document 2) has been proposed.
In addition, a voice dialogue system that recognizes a voice input by a user and outputs a content specified in a scenario by voice synthesis to realize a voice dialogue with the user has been proposed (for example, Patent Document 3). reference).

JP 2003-271194 A Japanese Patent No. 4495907 Japanese Patent No. 4832097

By the way, it is assumed that a dialogue system that combines the above-described speech synthesis technology and a speech dialogue system and outputs some answer by speech synthesis in response to a speech by a user's voice is assumed. In this case, it has been pointed out that the voice output by the speech synthesis may feel unnatural to the user, specifically giving the machine a feeling of being uttered.
The present invention has been made in view of such circumstances, and one of its purposes is to respond to a user's voice-like speech as if it were interacting with a person. An object of the present invention is to provide a technique capable of outputting by voice synthesis.

  In examining the man-machine system for outputting (replying) the response to the user's utterance by speech synthesis, the inventor first impresses the conversation as to what kind of dialogue is made between people. We focused on the pitch (frequency).

  Here, as a dialogue between people, a case where the other person (referred to as “b”) answers (including conflicts) to a statement (including a question and a question) by one person (referred to as “a”) will be considered. In this case, when “a” speaks, not only “a” but also “b” trying to reply to the said speech often leaves the pitch in a certain section of the said speech with a strong impression. b, when replying with consent, approval, or affirmation, the pitch of the utterance that remains in the impression, for example, the pitch of the ending or beginning of the sentence Speak to be in a Kyowa pitch. The person a who heard the answer has the above-mentioned relationship between the pitch that remains in the impression about his / her speech and the pitch of the part that characterizes the answer to the person's reply, so that the answer to b is comfortable and reassuring. The present inventor thought that he had a good impression.

  For example, when “a” says “Yes?”, “A” and “b” are in a state of memorizing the pitch of “Sho” at the end of the said utterance that strongly expresses willingness or confirmation. . In this state, when b tries to affirmatively answer “Yes, yes” to the statement, the part characterizing the answer, for example, the ending, Answer “Yes, yes” so that the pitch of “Yes” is in the above relationship.

FIG. 3 shows a formant in such an actual dialogue. In this figure, the horizontal axis is time, the vertical axis is frequency, and the spectrum shows a state where the intensity increases as it becomes white.
As shown in the figure, a spectrum obtained by frequency analysis of human speech appears as a plurality of peaks that move in time, that is, formants. Specifically, a formant corresponding to “Yeah?” And a formant corresponding to “Ah, yes” each appear as three peak bands (white band-like portions moving along the time axis).
When attention is paid to the first filmant having the lowest frequency among these three peak bands, the frequency of the symbol A (the central part) corresponding to “Sho” of “Yeah?” Is approximately 400 Hz. . On the other hand, for the code B, the frequency of the code B corresponding to “Yes” of “A, Yes” is approximately 260 Hz. For this reason, it can be seen that the frequency of the code A is approximately 3/2 with respect to the frequency of the code B.

  The relationship that the frequency ratio is 3/2 is the same octave “de” for “so” or “la” one octave lower for “mi”. As will be described later, there is a complete 5 degree relationship. This frequency ratio (predetermined relationship between pitches) is a preferred example, but various examples can be given as will be described later.

  FIG. 4 is a diagram illustrating a relationship between a pitch name (floor name) and a human voice frequency. In this example, the frequency ratio when the fourth octave “do” is used as a reference is also shown, and “so” is 3/2 as described above when “do” is used as a reference. Further, the frequency ratio when the third octave “La” is used as a reference is also illustrated in parallel.

  In this way, in the dialogue between people, it can be considered that the pitch of the utterance and the pitch of the answer to be answered are not irrelevant but have the above relationship. Then, the present inventor analyzed many dialogue examples and statistically aggregated evaluations by many people to prove that this idea is roughly correct.

Now, a dialogue between people is typically an utterance by a and an utterance of b for the utterance, but the utterance of b is not necessarily an answer composed of a specific sentence with respect to the utterance of a. Not only that, but also the “A, Yes”, “Yeah”, “That's right”, “So,” etc. Furthermore, in the dialogue between people, it is well known from a rule of thumb that such a response is returned with good response to the utterance a.
Therefore, when considering a dialogue system that outputs (replies) an answer by speech synthesis in response to a user's utterance, it is important to be able to output the answer as an answer with good response.
Therefore, in order to achieve the above object for the speech synthesis, the following configuration is adopted.

In other words, in order to achieve the above object, an encoding / decoding device according to an aspect of the present invention includes an A / D converter that converts a speech by a speech signal into a digital signal, and a specification among the speech by the digital signal. A pitch analysis unit that analyzes the pitch of the first section, a conflict acquisition unit that acquires the conflict data according to the meaning content of the statement when the conflict is returned to the statement, and the acquired conflict data A pitch control unit that changes the pitch of a specific second section to a pitch that has a predetermined relationship with the pitch of the first section, and analog signal that has the tone data changed. And a D / A converter for converting the data into an output.
According to this aspect, in the case where there is a conflict with a word based on the input voice signal, the conflict data corresponding to the meaning content of the word is acquired, so that the conflict can be output with good response. In addition, it is possible to prevent an unnatural feeling from being accompanied by speech synthesis.

In addition, it is preferable that a 1st area is the ending of a speech, for example, and a 2nd area is the beginning or the ending of a conflict. This is because, as described above, the section characterizing the impression of the comment is the ending of the comment, and the section characterizing the impression of the answer as an answer to the comment is often the beginning or ending of the comment.
Moreover, it is preferable that the predetermined relationship is a relationship of Kyowa intervals excluding perfect 1 degree. Here, “Kyowa” means a relationship in which when a plurality of musical sounds are generated at the same time, they are fused and well harmonized, and these pitch relationships are called Kyowa pitches. The degree of cooperation is higher as the frequency ratio (frequency ratio) between two sounds is simpler. The simplest frequency ratios of 1/1 (perfect 1 degree) and 2/1 (perfect 8 degree) are called absolute consonance pitches, and 3/2 (perfect 5 degree) and 4/3 (perfect) 4 degrees) and is called the perfect harmony pitch. 5/4 (3 degrees long), 6/5 (3 degrees short), 5/3 (6 degrees long) and 8/5 (6 degrees short) are called incomplete harmony intervals, and all other frequency ratios This relationship (long and short 2 degrees and 7 degrees, various increase / decrease intervals, etc.) is called dissonance interval.
If the pitch of the second section is the same as the pitch of the first section, it is considered that the dialogue is accompanied by an unnatural feeling, so in the relationship between the pitch of the utterance and the pitch of the answer Exactly 1 degree is excluded.
In the above aspect, the most desirable example of the predetermined relationship is considered to be a relationship in which the pitch of the second section is 5 degrees below the pitch of the first section as described above. However, the predetermined relationship is not limited to a consonant pitch except for a perfect degree, but may be a dissonant pitch relationship, or may be a pitch relationship within the range of one octave above and below, excluding the same.

In the above aspect, when the A / D converter supplies an audio signal converted into a digital signal to a host computer and returns something other than the conflict based on the conflict data for the utterance, the pitch control unit Of the answer data to the utterance supplied from the host computer, the pitch of the second section is changed to the pitch having the predetermined relationship with the pitch of the first section, and the D / A conversion The instrument may be configured to convert the answer data whose pitch is changed into an analog signal.
According to this configuration, the response can be created (acquired) with good response by the single unit of the encoding / decoding device, and the answer other than the conflict based on the conflict data can be created (acquired) with high accuracy by the host computer.

The aspect of the present invention can be conceptualized not only as a coding / decoding device, but also as a speech synthesizer including the coding / decoding device and a host computer, or a program that functions as the coding / decoding device. is there.
In the present invention, the pitch (frequency) of the speech is the analysis target, and the pitch of the answer is the control target. As is clear from the above-described formant example, human speech has a certain frequency range. Therefore, it is inevitable that the analysis and control have a certain frequency range. In addition, as a matter of course, errors occur in analysis and control. For this reason, in this case, the pitch analysis and control are allowed not only to have the same numerical value of the pitch (frequency) but also to have a certain range.

It is a block diagram which shows the structure of the speech synthesizer which concerns on 1st Embodiment. It is a figure which shows the structure of the functional block of a speech synthesizer. It is a figure which shows the example of the sound formant in a dialog. It is a figure which shows the relationship between a pitch name, a frequency, etc. It is a flowchart which shows the operation | movement of the speech process in a speech synthesizer. It is a flowchart which shows the operation | movement of the conflict process in an audio | voice process. It is a flowchart which shows the operation | movement of the reply process in an audio | voice process. It is a figure which shows the specific specific example of an ending. It is a figure which shows the example of the pitch conversion with respect to audio | voice waveform data. It is a figure which shows the psychological influence which a synthetic speech gives with respect to the speech by a user. It is a figure which shows the principal part of the process in an application example (the 1). It is a figure which shows the principal part of the process in an application example (the 2). It is a figure which shows the principal part of the process in an application example (the 3).

  Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment>
First, the encoding / decoding device according to the first embodiment of the present invention will be described.
FIG. 1 is a diagram illustrating a hardware configuration of a speech synthesizer 10 to which the encoding / decoding device according to the first embodiment is applied.
In this figure, a speech synthesizer 10 is a terminal device such as a mobile phone, for example, a speech input unit 102, a speaker 104, a coding / decoding device (hereinafter referred to as “CODEC”) 120, The configuration includes a host computer 160 that is a host computer as viewed from the CODEC 120.
Although not described in detail, the voice input unit 102 includes a microphone that converts a user's voice into an electric signal and an LPF (low-pass filter) that cuts a high-frequency component of the converted voice signal. The speaker 104 acoustically converts and outputs the audio signal converted into an analog signal by the CODEC 120.

The CODEC 120 is a module configured, for example, as a single chip or a multichip, and includes a microprocessor 122, a memory 124, an A / D converter 132, and a D / A converter 134. In the CODEC 120, the microprocessor 122 processes the audio signal by executing the program P1 (firmware) stored in the memory 124.
The host computer (host) 160 includes a CPU (Central Processing Unit) 162 and a memory 164. The CPU 162 is connected to the CODEC 120 and the memory 164 via the bus B.
In addition to the operating system, the memory 164 stores a voice processing program P2. In this voice processing, when the user speaks, in addition to the dialogue processing that outputs the answer and the answer to the speech by speech synthesis, the voice recognition processing that converts the user's speech into a character string, the host 160 side A reproduction process for reproducing and outputting the processed music, voice, and the like is included.

  Although not particularly illustrated, the speech synthesizer 10 also includes a display unit, an operation input unit, and the like, so that the user can check the status of the device and input various operations to the device. Can be done. The speech synthesizer 10 is not limited to a terminal device such as a mobile phone, and may be a notebook or tablet personal computer.

FIG. 2 is a diagram showing functional blocks showing the configuration of the speech synthesizer 10. This functional block is constructed by the microprocessor 122 executing the program P1 in the CODEC 120 and the CPU 162 executing the program P2 in the host 160, respectively.
As shown in this figure, in the CODEC 120, a pitch analysis unit 136, a language analysis unit 138, a linkage unit 140, a reading unit 142, a companion data storage unit 143, and a pitch control unit 144 are constructed. In the host 160, A language analysis unit 178, a linkage unit 180, an answer creation unit 182, a language database 184, an answer database 186, and an audio library 188 are constructed.

As will be described in detail later, the pitch analysis unit 136 on the CODEC 120 side performs volume analysis and frequency analysis on the speech indicated by the audio signal of the digital signal, and a specific section (first section) of the speech. ) Is obtained, and pitch data indicating the pitch is output. The pitch analysis unit 136 supplies the volume waveform used for volume analysis and the pitch waveform used for frequency analysis to the language analysis unit 138, respectively.
Here, the first section is, for example, the ending of a statement. The pitch here is, for example, a first formant that is the lowest frequency component among a plurality of formants obtained by frequency analysis of an audio signal. In FIG. This is the frequency (pitch) indicated by the peak band. For frequency analysis, FFT (Fast Fourier Transform) or other known methods can be used. An example of a specific method for specifying the ending in the utterance will be described later.

The language analysis unit 138 analyzes the speech indicated by the audio signal of the digital signal and performs the following determination. Specifically, the language analysis unit 138 determines whether the content of the statement is to return an answer or an answer other than an answer. As a specific method of discrimination in the language analysis unit 138, for example, a volume / pitch change pattern toward a ending of a typical question (a statement that should return an answer other than a conflict) is stored in advance, and a pitch analysis is performed. If the volume change indicated by the volume waveform from the section 136 and the pitch change indicated by the pitch waveform match the stored volume / pitch change pattern (or if the degree of similarity is high), the message Is determined to be a question, and if it does not match the stored change pattern (or if the degree of similarity is low), it is determined that the utterance should return an answer.
If the language analysis unit 138 cannot determine or the reliability of the determination result by the language analysis unit 138 is low, the determination is intentionally left to the host 160 (language analysis unit 178). good. In addition, because the sumo is what improves the so-called twist of the conversation, and its meaning can be neglected, even if the remark is a question, even if it is a question, it is a big problem in practice. Must not.

When it is determined that the content of the utterance should return an answer, for example, the linkage unit 140 controls the pitch control unit 144 to set the processing target as the answer data, while on the host 160 side. , It notifies that the process about the said statement is unnecessary.
Further, the cooperation unit 140 determines that the content of the utterance should return an answer other than the conflict (if the content of the utterance cannot be determined under the above rules, the reliability of the determination result is The pitch control unit 144 is controlled so that the processing target is the answer data created by the answer creating unit 182 described later, while the host 160 sends an answer to the comment. Notify that it should be created (acquired).

The consideration data storage unit 143 stores a plurality of consideration data. Here, the sum data is the speech waveform data including the answers such as “E-to”, “I see”, “That's right”, “Ah, yes”, “N-”. Examples of the format of the audio waveform data include a wav format.
In the case where the language analysis unit 138 determines that the statement should return an answer, the reading unit 142 selects one of the plurality of pieces of consideration data stored in the opinion data storage unit 143 in a predetermined order (or (Randomly) selected and read out and supplied to the pitch controller 144.
In addition, in the configuration in which the conflict data is read out regardless of the content of the utterance as described above, an irrelevant conflict may be output for the utterance, but the meaning of the conflict can be neglected as described above. It doesn't matter.

Note that the answer data read by the reading unit 142 and the answer data supplied by the answer creating unit 182 are both voice waveform data. I can. For example, if the conflict is “Ah, yes”, as shown in FIG.
However, simply replaying such speech waveform data ignores the pitch of the speech that triggered this interaction. For this reason, it cannot be denied that even if the compatibility data is simply reproduced in response to the user's remarks, it becomes a mechanical feeling.

For this reason, the pitch control unit 144 performs speech synthesis based on the conflict data read from the conflict data storage unit 143 or the answer data supplied from the answer creation unit 182 to be described later. Control according to high data.
Specifically, the pitch control unit 144 sets the pitch of a specific section (second section) of the conflict data or answer data so as to have a predetermined relationship with the pitch indicated by the pitch data. Control speech synthesis.
The audio signal synthesized by the pitch control unit 144 is converted into an analog signal by the D / A conversion unit 134, and then is acoustically converted by the speaker 104 and output.

  In the present embodiment, the second section is the ending of the answer specified by the answer data or the answer specified by the answer data, but is not limited to the ending as described above. In this embodiment, the pitch having a predetermined relationship with the pitch data is set to a pitch having a relationship of 5 degrees below, but as will be described later, a sound having a relationship other than 5 degrees is used. It can be high.

On the other hand, on the host 160 side, the linkage unit 180 supplies information such as various parameters and various states to the linkage unit 140 of the CODEC 120. On the other hand, when a response creation request is notified from the linkage unit 140, the language The analysis unit 178 and the answer creation unit 182 are controlled.
Specifically, the language analysis unit 178 analyzes the semantic content of the speech of the audio signal converted into a digital signal by the A / D converter 132. Specifically, the language analysis unit 178 determines which phoneme the speech signal is close to by referring to a phoneme model created in advance in the language database 184, and the semantic content of the speech defined by the speech signal Is analyzed. As such a phoneme model, for example, a hidden Markov model can be used.
The answer database 186 is a database that is referred to when creating an answer (character string) to the meaning content of the utterance. The voice library 188 is a database required when creating answer data in a format in which a character string of the answer is read out. Specifically, in the speech library 188, speech unit data defining waveforms of various speech units that are speech materials such as a single phoneme or a transition portion from a phoneme to a phoneme is stored in a database in advance. Yes.

The answer creating unit 182 first creates an answer (character string) corresponding to the semantic content of the utterance analyzed by the language analyzing unit 178 with reference to the answer database 186, and second, depending on the answer The answer data is created with reference to the audio library 188.
For example, if the comment by the audio signal is “now what? (Now what time?)”, The answer creating unit 182 acquires time information from a built-in real-time clock (not shown), and other than time information. Is obtained from the answer database 186 to create a reply “I am now at XX hour XX minutes”, and then create answer data in the form of speech waveform data that is read out by voice.
It should be noted that the speech synthesizer 10 alone cannot create a reply to the statement “What is tomorrow's weather? (Tomorrow's weather?)”. As described above, when the answer cannot be created only by the speech synthesizer 10, the answer creating unit 182 obtains information necessary for creating the answer by accessing an external server via the Internet (not shown). Yes.
Also, under the application of the above rule, if the content of the statement cannot be determined by the language analysis unit 138 or if the determination result has low reliability, the answer creating unit 182 Sometimes the answer to the answer is made.
By the way, the answer creating unit 182 does not need to create answer data by itself from the answer. For example, an answer may be supplied to another external server, requesting the creation of answer data, and the answer data created on the other external server may be acquired.
As described above, the answer creating unit 182 may create the answer to the utterance and the answer data corresponding to the answer by itself, or may use one created by another server or the like. In other words, the answer creating unit 182 only needs to acquire the answer data for the utterance in some form.
The answer data created / acquired by the answer creating unit 182 is supplied to the pitch control unit 144.

Next, the operation of the speech synthesizer 10 will be described. FIG. 5 is a flowchart showing the operation of speech processing in the speech synthesizer 10.
First, when the user performs a predetermined operation, for example, when an icon corresponding to audio processing is selected on the main menu screen (not shown), the CPU 162 starts the program P2. As a result, the functional blocks shown in FIG. 2 are constructed in the CODEC 120 and the host 160.

First, when a user speaks and inputs a voice to the voice input unit 102, the voice is converted into a voice signal by the voice input unit 102 and then converted into a digital signal by the A / D converter 132. (Step S11).
In the setting of the host 160, the cooperation unit 180 determines whether or not the execution of the dialogue process is designated among the activated voice processes (step S12).
If the execution of the interactive process is not specified (if the determination result in step S12 is “No”), other processes are executed (step S16). As other processing, for example, an audio signal converted into a digital signal is supplied to another functional block (not shown) in the host 160 as indicated by * 1 in FIG. , Voice recognition processing for converting a user's utterance into a character string, and, as indicated by * 2, the data processed by another functional block is converted into an analog signal by the D / A converter 134. For example, reproduction processing to be output by the speaker 104 can be given.

On the other hand, if execution of dialogue processing is specified (if the determination result in step S12 is “Yes”), the linkage unit 180 notifies the linkage unit 140 in the CODEC 120 to that effect, The language analysis unit 138 is made to determine whether or not the content of the input utterance should return an answer, that is, whether or not the processing is sufficient with only the CODEC 120 (step S13).
If the content of the input utterance is to return an affirmation (if the determination result of step S13 is “Yes”), the affirmative processing described next is executed (step S14), while the content of the utterance If the answer is not to be returned (if the determination result in step S13 is “No”), an answer process described later is executed (step S15).
Note that the audio processing ends after steps S14, S15, and S16.

FIG. 6 is a flowchart showing details of the conflicting process in step S14.
First, the language analysis unit 138 that has determined that the content of the input utterance should return a conflict notifies the cooperation unit 140 to that effect, and the cooperation unit 140 notifies the cooperation unit 180 in the host 160. Then, it is notified that it is not necessary to create an answer to the statement (step Sa11).
Upon receiving this notification, the cooperation unit 180 instructs the language analysis unit 178 to ignore the digital signal corresponding to the message. Thereby, the audio processing for the utterance is not executed on the host 160 side (step Sb11).

On the other hand, the pitch analysis unit 136 analyzes the voice signal of the input utterance as follows, specifies the ending pitch in the utterance, and sets the pitch data indicating the pitch as the pitch control unit. It supplies to 144 (step Sa12).
Specifically, first, the pitch analysis unit 136 divides an audio signal corresponding to a speech into a waveform by dividing the audio signal into a volume and a pitch (pitch). FIG. 8A is an example of a volume waveform in which the volume of an audio signal is represented on the vertical axis and the elapsed time is represented on the horizontal axis. FIG. 8B is a graph obtained by frequency analysis of the same audio signal. This is a pitch waveform in which the pitch of one formant is represented on the vertical axis and the elapsed time is represented on the horizontal axis. The time axis of the volume waveform in (a) and the pitch waveform in (b) are common.
Second, the pitch analysis unit 136 specifies the timing of the last local maximum P1 in the volume waveform of (a).
Third, the pitch analysis unit 136 determines that a predetermined time range (for example, 100 μsec to 300 μsec) including the timing of the specified maximum P1 before and after is the end of the pitch.
Fourth, the pitch analysis unit 136 outputs the average pitch of the section Q1 corresponding to the recognized ending in the pitch waveform of (b) as pitch data.
As described above, it is considered that erroneous detection of the ending of the utterance as a dialogue can be reduced by specifying the final maximum P1 of the volume waveform in the utterance as the timing corresponding to the ending of the utterance.
Here, in the volume waveform of (a), a predetermined time range including the timing of the last local maximum P1 before and after is recognized as the ending, but the predetermined time period having the maximum P1 timing as the start or end is determined. The time range may be recognized as the ending. Instead of the average pitch of the section Q1 corresponding to the authorized ending, the pitch at the beginning and end of the section Q1 and the timing of the maximum P1 may be output as pitch data. Further, the pitch of the ending may be specified by other analysis.

  On the other hand, in parallel with the pitch analysis, the reading unit 142 selects and reads one of the plurality of pieces of conflict data stored in the conflict data storage unit 143, and supplies the selected one to the pitch control unit 144 (step). Sa13).

  FIG. 9A is an example of the compatibility data “A, Yes”, for example, and indicates the pitch of each sound when reproduced without changing the pitch, as a black circle. In this state, even if the conflict data is reproduced, there is often a mechanical feeling because of the inflection.

  Therefore, the pitch controller 144 analyzes the pitch of the portion corresponding to the ending when the conflict data is simply reproduced, and the analyzed pitch is indicated by the pitch data from the pitch analyzer 136. For example, after shifting the pitch (pitch) of the entire consideration data so as to have a relationship that is 5 degrees below the pitch to be generated, the shifted consideration data is output (step Sa14).

(B) in FIG. 9 shows that when the user says “Yes?”, The pitch of the section (ending) of the ending “Sho” indicated by the symbol A in the utterance is based on the pitch data. It is a figure which shows the example of the pitch shift in the consideration data of (a) when it is shown that it is "seo". As shown in this figure, the pitch control unit 144 determines that the pitch of the section (end) of the ending “I” indicated by the symbol B is “So” among the answers “A, Yes”. The pitch of the entire conflict data is shifted so that the pitch is 5 ° lower. Then, the pitch controller 144 reproduces the compatibility data with the pitch shifted, and supplies it to the D / A converter 134.
As a result, the speaker 104 outputs a tone with the pitch shifted in response to the statement. After the output of the relevant data, the speech processing (see FIG. 5) is completed together with the relevant processing.

FIG. 10 is a diagram for explaining an impression given to the user by the speech synthesizer 10 according to the present embodiment. As shown in (a) of the figure, the user W inputs a message “That's right?” To the speech synthesizer 10 which is a terminal device. If the pitch of “Sho” corresponding to the ending of the utterance at this time is “So”, in the embodiment, as shown in FIG. The pitch is shifted and output so that the corresponding pitch of “I” becomes “do”. For this reason, it is possible to give the user W a good impression as if the speech synthesizer 10 which is a terminal device agrees and agrees with his / her speech.
On the other hand, when the pitch of the companion data “A, Yes” is not shifted, the pitch of “I” corresponding to the ending is reflected by the state stored in the companion data storage unit 143, and the remark by the user W Is not considered. For this reason, as shown in (c) of the figure, if the pitch of “I” corresponding to the ending of the sumo is “Fa”, it corresponds to the ending of the statement “That's right?” It becomes a dissonant pitch relationship with “So”, the pitch of “Sho”. That is, referring to FIG. 4, the frequency of “So” (396.0 Hz) is 9/8 relative to the frequency of “Fa” (352.0 Hz). For this reason, when assuming the worst case, the user W is given a bad impression such as disgust.
As will be described later, the speech synthesizer 10 may be configured to positively give such a bad impression to the user.

Next, the answer process in step S15 of the voice process will be described.
FIG. 7 is a flowchart showing details of the answer processing.
The language analysis unit 138, which has determined that the content of the input utterance should not return an agreement, notifies the cooperation unit 140 to that effect, and the cooperation unit 140 notifies the cooperation unit 180 in the host 160 of the relevant statement. It is notified that an answer to the statement should be created (answer request) (step Sa31).
Upon receiving this notification, the cooperation unit 180 instructs the language analysis unit 178 to analyze the semantic content of the statement. In accordance with this instruction, the language analysis unit 178 analyzes the semantic content of the statement as described above (step Sb31). Then, the answer creating unit 182 creates (acquires) answer data for the meaning content and supplies the answer data to the pitch control unit 144 (step Sb32).

On the other hand, in the answer process, the pitch analysis unit 136 specifies the pitch of the ending in the input utterance, as in step Sa12 in the conflict process, and sets the pitch data indicating the pitch as the pitch control unit 144. (Step Sa32).
Then, the pitch control unit 144 analyzes the pitch of the portion corresponding to the ending when the answer data is simply reproduced, and the analyzed pitch is indicated by the pitch data from the pitch analysis unit 136. The pitch of the answer data is shifted so that the relation is, for example, 5 degrees lower than the pitch to be played, and then the answer data after the shift is output (step Sa34).

Here, the result of shifting the pitch of the answer data is almost the same as (b) in FIG. 9 because the answer data is merely replaced with the answer data, and the impression given to the user W is also shown in FIG. It is almost the same as the contents shown.
Note that after the answer data is output, the voice process ends together with the answer process.

According to the speech synthesizer 10 according to this embodiment, in response to a user's remarks, even if a reconciliation is output based on the reconciliation data or a response other than the reconciliation is output based on the replay data, it is as if people are interacting. It is possible to give the user a natural feeling as if it were.
In the present embodiment, when an answer is given to an utterance, the process is completed on the CODEC 120 without being processed on the host 160 side, and therefore the answer can be output with a good response to the utterance. Further, when an answer other than a conflict is output in response to the utterance, since the answer is created and acquired on the host 160 side, the accuracy of the answer can be improved. For this reason, in the present embodiment, the answers can be output with good response, while answers other than the answers can be output with high accuracy.

<Applications / Modifications>
The present invention is not limited to the above-described embodiments, and various applications and modifications as described below are possible, for example. In addition, one or more arbitrarily selected aspects of application / deformation described below can be appropriately combined.

<Voice input part>
In the embodiment, the voice input unit 102 is configured to input a user's voice (speech) using a microphone and convert the voice signal into a voice signal. However, the voice input unit 102 is not limited to this configuration, and the voice signal processed by another processing unit or A configuration may be adopted in which an audio signal supplied (or transferred) from another device is input. In other words, the voice input unit 102 only needs to be configured to input a speech by a voice signal in some form.

<Remarks and reconciliation>
In the embodiment, when it is determined that a statement should return an answer, one of a plurality of pieces of consideration data stored in the opinion data storage unit 143 is read. As such, there is a case where an unfavorable conflict is output for the utterance (although it is rarely a problem). Therefore, the correspondence data is associated with a typical change pattern of the volume and pitch of the speech that is the basis of the conflict, and the volume change and pitch change of the speech supplied from the pitch analysis unit 136 are: When the language analysis unit 138 determines that the change pattern of the volume / pitch of a certain comment is matched, the reading unit 142 may be designated to read out the conflict data associated with the comment. According to this configuration, for example, if the corresponding data “Yes” is stored in the related data storage unit 143 in association with the sound volume / pitch change pattern of the message “Samu Naa”, the user actually If you say `` Samunaa '', if the volume change and pitch change of the utterance match the memorized `` Samuinaa '' utterance volume / pitch change pattern, then you have the right answer Is output. Therefore, in this case, it is possible to avoid an out-of-order conflict such as “Um”, “I see”, “So”, etc., in response to the user's statement “Samu Naa”.

<Language Analysis Department>
In the embodiment, the CODEC 120 has the language analysis unit 138 and the host 160 has the language analysis unit 178. However, for example, the language analysis unit 138 can be omitted.
When the language analysis unit 138 is omitted, the language analysis unit 178 assumes the function of the language analysis unit 138. In other words, if the language analysis unit 178 determines whether or not the utterance indicated by the audio signal of the digital signal should return an answer, and if it determines that the utterance should return an answer, information to that effect May be supplied directly to the reading unit 142 of the CODEC 120 or indirectly via the cooperation units 180 and 140 to instruct reading of the conflict data.

<End of answer, beginning of answer>
In the embodiment, it is configured to control the pitch of the ending of the answer including the answer corresponding to the pitch of the ending of the utterance.However, depending on the language, dialect, wording, etc. It may be characteristic. In such a case, the person who made the statement, when there is an answer to the statement, unconsciously compares the pitch of the statement and the pitch of the characteristic beginning of the answer. Judge the impression. Therefore, in this case, the pitch at the beginning of the answer may be controlled in accordance with the pitch at the end of the utterance. According to this configuration, when the head of the answer is characteristic, it is possible to give a psychological impression to the user who receives the answer.

  The same applies to utterances, not limited to endings, but may be determined by the beginning of a sentence. In addition, an answer including a conflict is not limited to the beginning or end of the word, and may be determined based on an average pitch, or may be determined based on the pitch of the most pronounced portion. For this reason, it can be said that the 1st section of an utterance and the 2nd section of an answer are not necessarily restricted to an initial or ending.

<Pitch relationship>
In the embodiment described above, the voice synthesis is controlled such that the pitch of the ending of the answer is 5 degrees lower than the ending of the question. However, the control is performed in a relationship of Kyowa intervals other than 5 degrees below. It may be a configuration. For example, as described above, it may be complete 8 degrees, complete 5 degrees, complete 4 degrees, long / short 3 degrees, and long / short 6 degrees.
In addition, there is a case where a relationship of a pitch that gives a good (or bad) impression is empirically recognized even if it is not a relationship of the Kyowa pitch, so that the pitch of the answer is controlled according to the relationship of the pitch. good. However, even in this case, if the pitch between the pitch of the question ending and the pitch of the answer ending is too far apart, the answer to the question tends to be unnatural. It is desirable that the pitch of the answer is in the range of one octave above and below.

<Pitch shift of answer>
By the way, in a configuration in which control is made so that the pitch of the answer including the conflict has a predetermined relationship with respect to the pitch at the end of the utterance or the like, the details are, for example, 5 degrees lower as in the embodiment. In such a shifting configuration, if the pitch to be lowered by 5 degrees is too low, the answer may be synthesized with an unnatural low tone. Next, application examples (No. 1 and No. 2) for avoiding such a case will be described.

  FIG. 11 is a diagram illustrating a main part of processing in the application example (part 1). Here, the main part of the processing refers to processing executed in “Saving pitch control” in step Sa14 in FIG. 6 or “Reply pitch control” in step Sa34 in FIG. That is, in the application example (No. 1), the processing shown in FIG. 11 is executed in step Sa14 or Sa34, and details are as follows.

First, the pitch controller 144 obtains and temporarily determines a pitch that is, for example, 5 degrees below the pitch indicated by the pitch data from the pitch analyzer 136 (step Sc11).
Next, the pitch controller 144 determines whether or not the temporarily determined pitch is lower than a predetermined threshold pitch (step Sc12). Note that the threshold pitch is set to a pitch corresponding to the lower limit frequency for speech synthesis, or a pitch that gives an unnatural feeling if it is lower than this.

If the tentatively determined pitch, that is, the pitch 5 degrees below the ending pitch in the utterance is lower than the threshold pitch (if the determination result in step Sc12 is “Yes”), the pitch control unit 144 The temporarily determined pitch is changed to a pitch one octave higher (step Sc13). On the other hand, if the temporarily determined eye pitch is equal to or higher than the threshold pitch (if the determination result in step Sc12 is “No”), the process in step Sc13 is skipped.
Subsequently, the pitch controller 144 determines the target ending pitch at the time of shifting the pitch of the conflict data (answer data) as the following pitch (step Sc14). That is, the pitch controller 144 determines that the temporarily determined pitch is changed to one octave higher if the temporarily determined pitch is lower than the threshold pitch, and that the temporarily determined pitch is the threshold pitch. If it is above, the pitch which becomes the target each is determined this time as it is.

  Then, the pitch control unit 144 analyzes the pitch of the portion corresponding to the ending when the compatibility data supplied from the reading unit 142 (or the answer data supplied from the answer creating unit 182) is simply reproduced. At the same time, after shifting the pitch of the conflict data (answer data) so that the analyzed pitch becomes the determined pitch, the shifted conflict data (response data) is output (step Sc15). ).

According to this application example (part 1), if the pitch when shifted is lower than the threshold pitch, it is shifted to a pitch one octave higher than the pitch, so that it is a conflict with an unnatural bass. -It is possible to avoid the point that answers are synthesized.
In this example, the pitch of the ending of the answer is shifted to a pitch one octave higher, but may be shifted to a pitch one octave lower. Specifically, since the pitch of the ending of the question issued by the user is high, if the pitch 5 degrees below the pitch is too high, the answer is synthesized with an unnatural high tone. . In order to avoid this, if the pitch (temporarily determined pitch) that is 5 degrees below the pitch indicated by the pitch data is higher than the threshold pitch, the pitch of the ending of the answer is set. The pitch may be shifted to a pitch one octave lower than the temporarily determined pitch.

  In addition, when speech synthesis is performed, there are cases where it is possible to divide and output by sex or age (child / adult). For example, the conflict data can be associated with the attribute data by previously dividing the conflict data by sex or age in the conflict data storage unit 143 and storing the conflict data in association with the attribute data defining the attribute such as the sex. Affinity data can be read. On the other hand, in the speech library 188, if the speech segment data is similarly divided in advance by gender, age, etc. and associated with the attribute data to create a database, answer data associated with the attribute data can be created.

  In this case, when the attributes of women and children are specified, if the pitch is lowered to 5 degrees below the ending of the statement, there will be a low tone that is not suitable for that attribute. Since voice synthesis is performed, similarly, a configuration may be adopted in which the pitch is shifted to an octave higher.

FIG. 12 is a diagram showing a main part of processing in such an application example (part 2), and shows processing executed in step Sa14 in FIG. 6 or Sa34 in FIG.
Explaining mainly the points different from FIG. 11, in step Sd11, the pitch control unit 144 selects a pitch that is 5 degrees below the pitch indicated by the pitch data from the pitch analysis unit 136. After obtaining and tentatively determining, it is determined whether or not the attributes of women and children are specified (step Sd12).

If a predetermined attribute, that is, an attribute such as a woman or a child is designated (if the determination result in step Sd12 is “Yes”), the pitch control unit 144 sets the tentatively determined pitch to a sound one octave higher. If the attribute is not specified as female or child, for example, if male or adult is specified (if the determination result in step Sd12 is “No”), the shift to high (step Sd13). The process of step Sb13 is skipped. The subsequent steps are the same as in the application example (No. 1).
According to this application example (part 2), when outputting the answer / answer in the voice of a woman or a child, the pitch is shifted so that the pitch is one octave higher than the temporarily determined pitch. It is possible to avoid the point that the answer / answer is synthesized with a low tone.
In this example, if female or child is specified as an attribute, the pitch shifts to a pitch one octave above. However, for example, if an adult male is specified as an attribute, the character corresponding to the attribute is not suitable. In order to avoid that the answer is synthesized with high-pitched sounds, the pitch may be shifted to a pitch one octave below.

<Dissonance>
In the above-described embodiment, the voice synthesis is controlled so that the pitch of the ending and the like of the reconciliation / reply becomes the relationship of the consonant pitch with respect to the ending of the remark. Speech synthesis may be controlled. In addition, when synthesizing and answering with a pitch that is in a dissonant pitch relationship, the spoken user is given an unnatural feeling, a bad impression, a harsh feeling, etc., and a smooth dialogue is not established. There are also concerns, but there is a view that this feeling is good for stress relief.
Therefore, as an operation mode, a mode (first mode) in which a favorable impression is desired and a response is desired (first mode) and a mode in which a negative impression is desired (second mode) are prepared. A configuration may be adopted in which speech synthesis is controlled accordingly.

FIG. 13 is a diagram showing a main part of processing in such an application example (part 3), and shows processing executed in step Sa14 of FIG. 6 or Sa34 of FIG.
Description will be made centering on differences from FIG. 10. The pitch controller 144 determines whether or not the first mode is set as the operation mode (step Se <b> 11).

  If the first mode is set as the operation mode (if the determination result in Step Se11 is “Yes”), the pitch control unit 144 displays the pitch of the ending and the answer of the answer / answer, for example, the ending of the utterance. The pitch is determined so as to be a pitch that has a relationship of a concerted pitch with respect to the pitch (step Se12). On the other hand, if the second mode is set as the operation mode (if the determination result in step Sd11 is “No”), the pitch controller 144 determines the pitch of the ending of the answer / answer as the ending of the utterance. Determination is made so that the pitch is in a dissonant pitch relationship with the pitch (step Se13). The subsequent processes are the same as those of the application example (part 1) and the application example (part 2).

  Therefore, according to this application example (No. 3), if the first mode is set, the answer / answer is synthesized with the pitch that is in the relationship of the Kyowa pitch with respect to the pitch of the speech. If the two modes are set, the user can properly select the operation mode as appropriate because the answer / answer is synthesized with a pitch that is in a dissonant pitch with respect to the pitch of the speech.

<Others>
In the embodiment, an answer other than a conflict is created by the host 160 in response to a statement. However, in a terminal device or the like, the processing load is heavy or the storage capacity is limited. Considering this, it may be configured to be provided on the external server side. That is, in the speech synthesizer 10, it is sufficient that the answer creating unit 182 is configured to supply the answer data indicating the answer to the utterance to the pitch control unit 144, and the answer and the answer data are received on the speech synthesizer 10 side. It does not matter whether it is created or created on the side of a configuration other than the speech synthesizer 10 (for example, an external server).

DESCRIPTION OF SYMBOLS 10 ... Speech synthesizer, 120 ... Encoding / decoding device, 132 ... A / D converter, 134 ... D / A converter, 160 ... Host computer, 136 ... Pitch analysis unit, 138 ... Language analysis unit, 143 ... Affinity data storage unit, 144... Pitch control unit.

Claims (8)

A pitch analysis unit that analyzes the pitch of a utterance by a digital signal;
A language analysis unit that determines that if the pitch change toward the ending in the pitch of the analyzed utterance does not match the pitch change pattern stored in advance, it is a case of returning a conflict to the utterance;
If it is determined that it is a case of returning an answer to the statement, an interest acquisition unit that acquires the affinity data;
A voice control device comprising: A pitch control unit,
The pitch analysis unit is
Among the utterances by the digital signal, analyze the pitch of a specific first section,
The pitch control unit
Of the acquired conflict data, the pitch of a specific second section is controlled to a pitch having a predetermined relationship with the pitch of the first section.
The voice control apparatus according to claim 1. Supplying a speech by the digital signal to a host computer;
The language analysis unit
If the pitch change toward the ending in the pitch of the analyzed utterance matches the pitch change pattern, it is determined that it is a case of returning something other than a conflict to the utterance,
If it is determined that it is a case where an answer other than the answer based on the answer data is returned in response to the comment, the pitch control unit is configured to output the sound of the second section in the answer data to the comment supplied from the host computer. Controlling the pitch to a pitch that is in the predetermined relationship with respect to the pitch of the first section;
The voice control apparatus according to claim 1. The pitch analysis unit is
Analyzing the volume of speech by the digital signal,
The language analysis unit
The pitch change toward the ending at the pitch of the analyzed utterance does not match the pitch change pattern stored in advance, and the volume change toward the ending at the volume of the analyzed utterance is
2. The voice control device according to claim 1, wherein if it does not match a volume change pattern stored in advance, it is determined that an answer is returned for the utterance. An A / D converter that converts a voice signal to a digital signal;
A D / A converter that converts the analog data whose pitch is controlled to an analog signal and outputs the analog signal;
The voice control device according to claim 1, further comprising: An encoding / decoding device and a host computer;
The encoding / decoding device comprises:
A pitch analysis unit that analyzes the pitch of a utterance by a digital signal;
If the pitch change toward the ending in the pitch of the analyzed utterance does not match the pre-stored pitch change pattern, it is determined that there is a case of returning a conflict with the utterance. For example, a language analysis unit for determining that it is a case of returning something other than a conflict for the utterance;
When returning an answer to the statement, an agreement acquisition unit that acquires the agreement data according to the meaning content of the statement;
Have
The host computer
An answer acquisition unit for acquiring answer data for the comment when returning other than the answer based on the opinion data for the comment;
A voice control device characterized by that. Analyzing the pitch of speech by digital signals,
If the pitch change toward the ending in the pitch of the analyzed utterance does not match the pre-stored pitch change pattern, it is determined that it is a case where a reconciliation is returned to the utterance,
If it is determined that it is a case of returning an answer to the statement, obtain the answer data.
A voice control method characterized by the above. A program of a voice control device having a microprocessor,
Said microprocessor;
A pitch analysis unit that analyzes the pitch of speech from digital signals,
A language analysis unit for determining that if the pitch change toward the ending in the pitch of the analyzed utterance does not match the pitch change pattern stored in advance, it is a case of returning a conflict to the utterance; and ,
If it is determined that it is a case of returning an answer to the remark, an interest acquisition unit that acquires the affinity data;
A program characterized by functioning as

Images