JP2018151661A - Speech control apparatus, speech control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To synthesize speech in response to a user's utterance as if the user is interacting with a person.SOLUTION: A speech recognition apparatus includes a speech input unit 102 to which speech by a speech signal is input, a reply generation unit 110 for obtaining a reply to the speech, and a voice control unit 109 that changes an interval between input of a speech signal for a speech and output of a speech signal for an answer to an interval in a relationship defined by one output rule of a plurality of preset output rules, and sets one output rule in which the proportion of speech being made with respect to the answer among the plurality of output rules satisfies a predetermined condition within a predetermined period.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a voice control device, a voice control method, and a program.

In recent years, the following have been proposed as speech synthesis techniques. That is, by synthesizing and outputting speech corresponding to the user's speech tone and voice quality, a technique for sounding more humanly (see, for example, Patent Document 1), analyzing the user's speech, A technique for diagnosing a health condition or the like (see, for example, Patent Document 2) has been proposed.
In addition, a voice dialogue system that recognizes a voice input by a user and outputs a content specified in a scenario by voice synthesis to realize a voice dialogue with the user has been proposed (for example, Patent Document 3). reference).

JP 2003-271194 A Japanese Patent No. 4495907 Japanese Patent No. 4832097

By the way, it is assumed that a dialogue system that combines the above-described speech synthesis technology and a speech dialogue system to retrieve data and output by speech synthesis in response to a speech by a user's voice is assumed. In this case, a problem has been pointed out that the voice output by the voice synthesis feels unnatural to the user, specifically, the machine sometimes feels roaring.
The present invention has been made in view of such circumstances, and one of its purposes is to provide a voice control device, a voice control method, and a program that make a reply to a user's utterance feel natural. There is.

  When examining the man-machine system for outputting (replying) the response to the user's utterance by speech synthesis, the present inventor first determines the pitch (frequency) ).

  Here, as a dialogue between people, the other person (assuming b) answers (including conflicts) to one person (assuming a) (including questions, monologues, questions, etc.) consider. In this case, when “a” speaks, not only “a” but also “b” to reply to the said speech often leaves a strong impression of the pitch in a certain section of the said speech. b, when replying with consent, approval, or affirmation, the pitch of the utterance that remains in the impression, for example, the pitch of the ending or beginning of the sentence Speak to be in a Kyowa pitch. The person a who heard the answer has the above-mentioned relationship between the pitch that remains in the impression about his / her speech and the pitch of the part that characterizes the answer to the person's reply, so that the answer to b is comfortable and reassuring. The present inventor thought that he had a good impression.

  For example, when “a” says “Yes?”, “A” and “b” are in a state of memorizing the pitch of “Sho” at the end of the said utterance that strongly expresses willingness or confirmation. . In this state, when b tries to affirmatively answer “Yes, yes” to the statement, the part characterizing the answer, for example, the ending, Answer “Yes, yes” so that the pitch of “Yes” is in the above relationship.

FIG. 2 shows a formant in such an actual dialogue. In this figure, the horizontal axis is time, the vertical axis is frequency, and the spectrum shows a state where the intensity increases as it becomes white.
As shown in the figure, a spectrum obtained by frequency analysis of human speech appears as a plurality of peaks that move in time, that is, formants. Specifically, a formant corresponding to “Yeah?” And a formant corresponding to “Ah, yes” each appear as three peak bands (white band-like portions moving along the time axis).
When attention is paid to the first formant having the lowest frequency among these three peak bands, the frequency of the code A (the central part) corresponding to “Sho” of “Oh, right?” Is approximately 400 Hz. On the other hand, for the code B, the frequency of the code B corresponding to “Yes” of “A, Yes” is approximately 260 Hz. For this reason, it can be seen that the frequency of the code A is approximately 3/2 with respect to the frequency of the code B.

  The relationship that the ratio of the frequencies is 3/2 is the relationship between “seo” in the same octave and “mi” in the octave one lower than “mi”. As will be described later, there is a complete 5 degree relationship. This frequency ratio (predetermined relationship between pitches) is a preferred example, but various examples can be given as will be described later.

  FIG. 3 is a diagram showing a relationship between a pitch name (floor name) and a human voice frequency. In this example, the frequency ratio when the fourth octave “do” is used as a reference is also shown, and “so” is 3/2 as described above when “do” is used as a reference. Further, the frequency ratio when the third octave “La” is used as a reference is also illustrated in parallel.

In this way, in the dialogue between people, it can be considered that the pitch of the utterance and the pitch of the answer to be answered are not irrelevant but have the above relationship. Then, the present inventor analyzed many dialogue examples and statistically aggregated evaluations by many people to prove that this idea is roughly correct.
However, although it may be statistically correct, the relationship of the pitches that feels comfortable is each person. Also, when considering a dialogue system that outputs (replies) a reply to a user's utterance by speech synthesis, the user is encouraged to increase the number and frequency of utterances. That is important.
Therefore, in order to achieve the above-mentioned purpose when speech synthesis is performed on an answer to a user's utterance, the following configuration is adopted.

In other words, in order to achieve the above object, a speech synthesizer according to an aspect of the present invention analyzes a speech input unit that inputs a speech by a speech signal, and a pitch of a specific first section of the speech. A pitch analysis unit, an acquisition unit that acquires an answer to the utterance, a voice synthesis unit that synthesizes the acquired answer by voice, and a pitch of a specific second section in the answer to the voice synthesis unit The pitch of the first section is changed to a pitch having a relationship defined by a preset pitch rule, and the pitch rule is set in response to a comment made on the answer. And a voice control unit.
According to this aspect, the pitch of the specific second section in the answer is changed to a pitch having a relationship defined by the pitch rule with respect to the pitch of the specific first section of the utterance. Synthesis is controlled. The pitch rule is set according to the utterance made to the answer, so that it is possible to guide the direction in which the conversation with the machine is played.

In this aspect, it is preferable that the first interval is, for example, the ending of a statement, and the second interval is the beginning or ending of an answer. As described above, the section characterizing the impression impression is the ending of the comment, and the section characterizing the answer impression is often the beginning or ending of the answer.
Moreover, it is preferable that the predetermined relationship is a relationship of Kyowa intervals excluding perfect 1 degree. Here, “Kyowa” means a relationship in which when a plurality of musical sounds are generated at the same time, they are fused and well harmonized, and these pitch relationships are called Kyowa pitches. The degree of cooperation is higher as the frequency ratio (frequency ratio) between two sounds is simpler. The simplest frequency ratios of 1/1 (perfect 1 degree) and 2/1 (perfect 8 degree) are called absolute consonance pitches, and 3/2 (perfect 5 degree) and 4/3 (perfect) 4 degrees) and is called the perfect harmony pitch. 5/4 (3 degrees long), 6/5 (3 degrees short), 5/3 (6 degrees long) and 8/5 (6 degrees short) are called incomplete harmony intervals, and all other frequency ratios This relationship (long and short 2 degrees and 7 degrees, various increase / decrease intervals, etc.) is called dissonance interval.

If the pitch at the beginning or end of the answer is the same as the pitch at the end of the utterance, it is considered that the dialogue is unnatural. Is excluded.
In addition, the answer is not limited to a specific answer to the question, but includes an answer (interjection) such as “I see” or “I think so”.

  In a dialogue between people, a period from the remark to the answer, that is, a so-called interval, is one element that determines the momentum of the dialogue. Therefore, in the above aspect, the voice control unit controls a period from a speech to the output of the answer using a preset output rule, and the output is performed in response to the speech made to the answer. It is good also as a structure which sets a rule. According to this configuration, since the period from the remark to the output of the answer is set according to the remark being made with respect to the reply, it is possible to guide in a direction that encourages dialogue with the machine.

In the aspect described above, the pitch rule may be set according to any one of a plurality of scenes prepared in advance. The scenes here are the combination of the gender and age of the speaker, the gender and age of the voice to be synthesized, the combination of the speed of speech (fast and slow) and the speed of the answer to be synthesized, dialogue Purpose (voice guidance).
Similarly, the output rule may be set according to any of a plurality of scenes prepared in advance.

The aspect of the present invention can be conceptualized as a program that causes a computer to function as the speech synthesizer as well as the speech synthesizer.
In the present invention, the pitch (frequency) of the speech is the analysis target, and the pitch of the answer is the control target. As is clear from the above-described formant example, human speech has a certain frequency range. Therefore, it is inevitable that the analysis and control have a certain frequency range. In addition, as a matter of course, errors occur in analysis and control. For this reason, in this case, the pitch analysis and control are allowed not only to have the same numerical value of the pitch (frequency) but also to have a certain range.

It is a block diagram which shows the structure of the speech synthesizer which concerns on 1st Embodiment. It is a figure which shows the example of the sound formant in a dialog. It is a figure which shows the relationship between a pitch name, a frequency, etc. It is a figure which shows an example of the parameter | index table in a speech synthesizer. It is a figure which shows the example of a switching of the operation period in a speech synthesizer. It is a flowchart which shows the speech synthesis process in a speech synthesizer. It is a flowchart which shows the speech synthesis process in a speech synthesizer. It is a figure which shows the specific specific example of an ending. It is a figure which shows the example of the pitch shift with respect to an audio | voice sequence. It is a figure which shows an example of the parameter | index table in 2nd Embodiment. It is a figure which shows an example between a statement and a reply. It is a figure which shows an example of the parameter | index table in 3rd Embodiment.

  Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment>
FIG. 1 is a diagram showing a configuration of a speech synthesizer 10 according to an embodiment of the present invention.
In this figure, the speech synthesizer 10 is a terminal device such as a mobile phone having a CPU (Central Unit), a speech input unit 102, and a speaker 142. In the speech synthesizer 10, the CPU executes an application program installed in advance, so that a plurality of functional blocks are constructed as follows.
Specifically, in the speech synthesizer 10, an utterance section detection unit 104, a pitch analysis unit 106, a language analysis unit 108, a speech control unit 109, an answer creation unit (acquisition unit) 110, a speech synthesis unit 112, a language database 122, An answer database 124, an information acquisition unit 126, a management database 127, and an audio library 128 are constructed.
Although not particularly illustrated, the speech synthesizer 10 also includes a display unit, an operation input unit, and the like, so that the user can check the status of the device and input various operations to the device. Can be done. Similarly, although not particularly illustrated, the speech synthesizer 10 has a configuration in which a real-time clock is incorporated to acquire time information such as the current time. The voice synthesizer 10 is not limited to the terminal device 10 such as a mobile phone, and may be a notebook or tablet personal computer.

Although not described in detail, the audio input unit 102 is a microphone that converts a user's voice (speech) into an electric signal, an LPF (low-pass filter) that cuts a high frequency component of the converted audio signal, and a high frequency It is composed of an A / D converter that converts an audio signal with components cut into a digital signal.
The utterance section detection unit 104 processes the voice signal converted into the digital signal to detect the utterance (sound) section.

The pitch analysis unit 106 performs volume analysis and frequency analysis on the speech of the speech signal detected as the speech interval, and uses the speech data indicating the pitch in a specific interval (first interval) of the speech as speech. This is supplied to the control unit 109.
Here, the first section is, for example, the ending of a statement. The pitch here is, for example, a first formant that is the lowest frequency component among a plurality of formants obtained by frequency analysis of an audio signal. In FIG. This is the frequency (pitch) indicated by the peak band. For frequency analysis, FFT (Fast Transform) or other known methods can be used. An example of a specific method for specifying the ending in the utterance will be described later.

  On the other hand, the language analysis unit 108 determines which phoneme is close to the speech signal detected as the speech section by referring to a phoneme model created in advance in the language database 122, and the speech specified by the speech signal. Is analyzed (specified), and the analysis result is supplied to the answer creating unit 110.

The answer creation unit 110 creates an answer corresponding to the utterance analyzed by the language analysis unit 108 with reference to the answer database 124 and the information acquisition unit 126.
In the present embodiment, the answer created by the answer creating unit 110 includes:
(1) An answer indicating the affirmation or denial of the statement,
(2) Reply to specific content for remarks,
(3) Reply as a reconciliation to remarks,
Is assumed. Examples of (1) are “Yes”, “No”, etc., and (2) is “Hare, what is tomorrow ’s weather?” There is an example of answering the content specifically. (3) can be “Yes”, “Et”, etc., and the remarks can be answered with “Yes” or “No” as in (1), and specific as in (2) It is created (acquired) in cases other than remarks that need to be answered.

For the answer of (1), for example, in response to an utterance “It is 3 o'clock?”, If the time information is acquired from a built-in real-time clock (not shown), the answer creation unit 110 responds to the utterance. For example, it is possible to determine which of “Yes” and “No” should be answered.
On the other hand, for example, in response to the statement “Are you sure? (Tomorrow is sunny)?” If the weather information is not acquired by accessing an external server, the speech synthesizer 10 responds alone. I can't. As described above, when the answer cannot be made only by the speech synthesizer 10, the information acquisition unit 126 accesses an external server via the Internet, acquires information necessary for creating the answer, and supplies the information to the answer creation unit 110. Thereby, the said reply preparation part 110 can discriminate | determine whether it should replies, for example with "Yes" or "No" with respect to the said statement.
With respect to the answer (2), for example, in response to the statement “Now what? (Now what time?)”, The answer creating unit 110 obtains the time information and sends information other than the time information to the answer database. By acquiring from 124, it is possible to create an answer “I am now XX hour XX minutes”. On the other hand, in response to the statement “What is Asuno Tenki? (Tomorrow's weather?)”, The information acquisition unit 126 accesses an external server to acquire information necessary for an answer, and an answer creation unit. 110 is configured to create, for example, an answer “excludes” from the answer database 124 and an external server in response to the utterance.

The answer creation unit 110 creates and outputs a speech sequence from the created / acquired answers. This speech sequence is a phoneme string and defines a pitch and a sounding timing corresponding to each phoneme.
For the answers (1) and (3), for example, a voice sequence corresponding to the answer is stored in the answer database 124, while a voice sequence corresponding to the determination result is read from the answer database 124. good. Specifically, in the answer (1), the answer creating unit 110 may read a voice sequence such as “Yes” or “No” according to the determination result, and the answer (3) In this case, it is only necessary to read out a speech sequence such as “I think so” or “Et” according to the analysis result of the utterance and the determination result in the answer creation unit 110.
Note that the speech sequences created and acquired by the answer creation unit 110 are supplied to the speech control unit 109 and the speech synthesis unit 112, respectively.

The voice control unit 109 controls voice synthesis in the voice synthesis unit 112.
Since the voice sequence defines the pitch of the utterance and the sounding timing, the voice synthesizer 112 can output the basic voice of the answer by simply synthesizing the voice according to the voice sequence.
However, since the basic voice of the answer does not take into account the pitch of the ending in the utterance, it is as described above that the machine may give a feeling of being angry. Therefore, in the present embodiment, firstly, the voice control unit 109 determines a pitch of a specific section (second section) in the voice sequence supplied from the answer creation unit 110 to a pitch data. The pitch of the entire speech sequence is changed so that the pitch of the above relationship is obtained. In the present embodiment, the second section is the ending of the answer, but is not limited to the ending.

On the other hand, the relationship between the pitch of the second section of the answer and the pitch of the ending of the utterance (pitch rule) feels comfortable, etc. It depends on the user. Therefore, secondly, in the present embodiment, an evaluation period is provided as an operation period, and in the evaluation period, an answer is voice-synthesized with a plurality of pitch rules for the utterance, and at the end of the evaluation period, It was set as the pitch rule that the conversation bounced and reflected in the subsequent speech synthesis.
The management database 127 is managed by the voice control unit 109 and stores a table (index table) in which a pitch rule is associated with an index indicating the degree of conversational interaction.

FIG. 4 is a diagram illustrating an example of the contents stored in the index table. As shown in this figure, in the index table, the number of times of speech and the number of times of application are associated with each pitch rule.
Here, the pitch rule defines whether the pitch of the ending of the answer is related to the pitch of the ending of the utterance. For example, as shown in FIG. It is defined as up, down 3 degrees, down 5 degrees, down 6 degrees, down 8 degrees.
The number of utterances is the number of times when the user further utters a response within a predetermined time when the speech synthesizer 10 synthesizes an answer to the utterance by the user during the evaluation period. It is the value. In other words, during the evaluation period, even if the answer is voice-synthesized in response to a user's comment, there is no user's comment after that answer, or after a predetermined time has passed even if there is a comment If so, it is not counted as the number of utterances.
The number of times of application indicates the number of times that the corresponding pitch rule is applied in the evaluation period.
Therefore, by comparing the values obtained by dividing the number of utterances by the number of application times, the case where the number of times the user utters the answer to the answer is the maximum, that is, the case where the conversation is the most played is which pitch rule. The user can know whether it was a case of applying.
In addition, even if a certain pitch rule is applied and a response is synthesized by speech, the user may not speak within a predetermined time for the response. Is increasing.

The speech synthesizer 112 synthesizes speech from the speech sequence according to control by the speech controller 109. Specifically, the speech synthesizer 112 uses speech segment data registered in the speech library 128 for speech synthesis. The speech library 128 is a database of speech unit data that defines waveforms of various speech units that are speech materials, such as a single phoneme or a transition portion from a phoneme to a phoneme. The speech synthesizer 112 combines the speech unit data of each sound (phoneme) of the speech sequence and corrects the connected parts to be continuous, while replying according to the pitch rule determined by the speech controller 109. The sound signal is generated by changing the pitch of.
Note that the synthesized voice signal is converted into an analog signal by a D / A converter (not shown), then acoustically converted by the speaker 142 and output.

Next, the operation of the speech synthesizer 10 will be described.
First, when a user performs a predetermined operation, for example, when an operation for selecting an icon or the like corresponding to an interactive process is performed on a main menu screen (not shown), the CPU starts an application program corresponding to the process. By executing this application program, the CPU constructs the functional block shown in FIG.

  FIG. 5 is a diagram showing an operation period due to execution of the application program. As shown in the figure, in the present embodiment, the rule fixing period and the evaluation period are alternately repeated in the operation period. Among these, the rule fixed period is a period in which the answer is synthesized with the pitch rule set at the end of the evaluation period. Here, it is assumed that the set pitch rule is 5 degrees below as indicated by white triangles in FIG.

On the other hand, the evaluation period is a period for synthesizing an answer with a plurality of pitch rules in response to a utterance by a user and setting a pitch rule that is played most by the dialogue.
In this embodiment, as shown in FIG. 5, the rule fixing period and the evaluation period are alternately repeated every predetermined time. However, only when a predetermined condition is satisfied, for example, there is a user instruction. It may be configured to shift to the evaluation period only when

  FIG. 6 is a flowchart showing the speech synthesis process. This speech synthesis process is executed regardless of the rule fixed period and the evaluation period.

  First, the user inputs a speech by voice to the voice input unit 102 (step Sa11). For example, the utterance section detection unit 104 detects the utterance section by comparing the amplitude of the speech with a threshold value, and supplies the speech signal of the utterance section to each of the pitch analysis unit 106 and the language analysis unit 108 (step Sa12). ).

The language analyzing unit 108 analyzes the meaning of the utterance in the supplied voice signal and supplies data indicating the meaning content to the answer creating unit 110 (step Sa13).
The answer creating unit 110 creates an answer corresponding to the language analysis result of the utterance by using the answer database 124 or acquiring it from an external server via the information acquiring unit 126 as necessary (step Sa14). Then, the answer creating unit 110 creates a speech sequence based on the answer and supplies it to the speech synthesizer 112 (step Sa15).

For example, if the linguistic analysis result of the user's utterance means “Are you tomorrow (is it fine tomorrow?)?”, The answer creation unit 110 accesses the external server and is necessary for the answer. The weather information is acquired, and if the acquired weather information is clear, a sound sequence “Yes” is output, and if it is not clear, a sound sequence “No” is output.
Also, if the language analysis result of the user's utterance is “Asen Tenki is (Tomorrow's weather)?”, The answer creation unit 110, for example, according to the weather information obtained from the external server, Outputs an audio sequence such as “It is cloudy”.
On the other hand, if the linguistic analysis result of the speech by the user means “Ashha Haraka”, it is a self-speaking (or tweet), so the answer creation unit 110, for example, has a compatible speech sequence such as “Yes”. Is read from the answer database 124 and output.

  The voice control unit 109 specifies the ending pitch (initial pitch) in the voice sequence from the voice sequence supplied from the answer creating unit 110 (step Sa16).

Next, the voice control unit 109 determines whether or not the current time is the rule fixed period (step Sa17). If the current time is the rule fixed period (if the determination result in step Sa17 is “Yes”), the voice control unit 109 applies the pitch rule set in the evaluation period before the rule fixed period (step Sa18). ).
On the other hand, if the current time is not the rule fixed period but the evaluation period (if the determination result in step Sa17 is “No”), the voice control unit 109 is set, for example, in the evaluation period immediately before the evaluation period. The selected pitch rule is applied by selecting any one of the pitch rules and the pitch rules that sandwich the pitch rule up and down in the index table (step Sa19). Specifically, if the set pitch rule is 5 degrees below indicated by a white triangle mark in FIG. 4, the voice control unit 109 is 5 degrees below and 5 degrees in the indicator table. One of the three pitch rules of 3 degrees below and 6 degrees below that sandwich the bottom up and down is selected at random or in a predetermined order.

  On the other hand, the pitch analysis unit 106 analyzes the speech signal of the utterance in the detected utterance section, specifies the pitch of the first section (ending part) in the utterance, and converts the pitch data indicating the pitch into voice. It supplies to the control part 109 (step Sa20). Here, an example of a specific method for specifying the ending of a statement in the pitch analysis unit 106 will be described.

Assuming a dialogue in which a person who makes a statement wants an answer to the statement, the volume corresponding to the end of the statement is considered to be temporarily higher than the other parts. Therefore, the pitch of the first section (ending) by the pitch analysis unit 106 can be obtained as follows, for example.
First, the pitch analysis unit 106 divides the speech signal detected as an utterance section into a waveform by dividing it into a volume and a pitch (pitch). FIG. 8A shows an example of a volume waveform in which the volume of an audio signal is represented on the vertical axis and the elapsed time is represented on the horizontal axis. FIG. 8B is a first waveform obtained by frequency analysis of the same audio signal. It is a pitch waveform in which the pitch of formant is represented on the vertical axis and the elapsed time is represented on the horizontal axis. The time axis of the volume waveform in (a) and the pitch waveform in (b) are common.
Secondly, the pitch analysis unit 106 identifies the timing of the last local maximum P1 in the volume waveform of (a).
Thirdly, the pitch analysis unit 106 determines that a predetermined time range (for example, 100 μsec to 300 μsec) including the timing of the specified maximum P1 before and after is the ending.
Fourth, the pitch analysis unit 106 supplies the average pitch of the section Q1 corresponding to the recognized ending in the pitch waveform of (b) to the voice control unit 109 as pitch data.
Thus, it is considered that the erroneous detection of the ending of the speech as a conversation can be reduced by specifying the final maximum P1 of the volume waveform in the utterance section as the timing corresponding to the ending of the speech.
Here, in the volume waveform of (a), a predetermined time range including the timing of the last local maximum P1 before and after is recognized as the ending, but the predetermined time period having the maximum P1 timing as the start or end is determined. The time range may be recognized as the ending. Moreover, it is good also as a structure which outputs not the average pitch of the area Q1 corresponding to the recognized ending but the pitch of the start of the period Q1, the end, and the timing of local maximum P1 as pitch data.

Upon receiving the pitch data, the voice control unit 109 performs voice synthesis so that the pitch at the end of the answer has a relationship determined by the pitch rule to be applied to the pitch indicated by the pitch data. The unit 112 is instructed (step Sa21). In response to this instruction, the speech synthesizer 112 changes and outputs the pitch of the entire speech sequence so that the pitch of the ending of the answer becomes the pitch determined by the pitch rule.
In the present embodiment, even if an answer is output by speech synthesis, the user may speak after the answer, so the processing procedure returns to step Sa11. Note that the speech synthesis process is terminated by an explicit operation (for example, a software button operation) by the user.

FIG. 7 is a flowchart showing the operation of the table update process.
This table update process is executed independently of the speech synthesis process in FIG. 6, and mainly updates the index table (see FIG. 4) in the evaluation period and sets the pitch rules to be applied in the rule fixed period. Process.

First, the voice control unit 109 determines whether or not the current time (current time) is an evaluation period (step Sb11). If the current time is not the evaluation period (if the determination result in step Sb11 is “No”), the voice control unit 109 returns the processing procedure to step Sb11 again.
If the current time is the evaluation period (if the determination result in step Sb11 is “Yes”), the speech control unit 109 determines whether or not there is an output of the speech synthesized by the speech synthesis unit 112 (step Sb12). ).
If no answer is output (if the determination result in step Sb12 is “No”), the voice control unit 109 returns the processing procedure to step Sb11. For this reason, unless the present time is an evaluation period and an answer is output, the subsequent processing is not executed.
On the other hand, if there is an output of the answer (if the determination result in step Sb12 is “Yes”), the voice control unit 109 receives the user's comment within a predetermined time (for example, 5 seconds) after the output of the answer. It is determined whether or not there has been (step Sb13). This can be determined by, for example, whether or not the pitch data is supplied from the pitch analysis unit 106 within a predetermined time after the answer is output by the voice control unit 109.

If the user's utterance is within the predetermined time after the answer is output (when the determination result of step Sb13 is “Yes”), the voice control unit 109 updates the index table to update the index table. A pitch rule applied in speech synthesis is specified (step Sb14). As for the pitch rule, for example, when the pitch rule is selected in step Sa19, the selected pitch rule and the selected time information are stored in the management database 127 in association with each other. It can be specified by searching the pitch rule with the newest time information.
The voice control unit 109 increments the pitch rule item (the number of utterances and the number of times of application) applied to the speech synthesis of the answer by “1” in the index table (step Sb15).

  On the other hand, if the user does not speak after the answer is output, or if a predetermined time elapses even if there is a speech (when the determination result in step Sb13 is “No”), the voice control unit 109 As in step Sb14, the pitch rule applied in the speech synthesis of the answer is specified (step Sb16). However, in this case, since it is assumed that the user has not made a speech by the answer, the voice control unit 109 sets “1” only to the number of times the pitch rule applied in the voice synthesis of the answer in the index table. Increment by (step Sb17).

Next, the voice control unit 109 determines whether or not the current time is the end timing of the evaluation period (step Sb18).
If it is not the end timing of the evaluation period (if the determination result in step Sb18 is “No”), the voice control unit 109 returns the processing procedure to step Sb11 in order to be prepared when there is a statement after answering.
On the other hand, if it is the end timing of the evaluation period (if the determination result in step Sb18 is “Yes”), the values obtained by dividing the number of utterances by the number of applied times are compared for the three pitch rules in the evaluation period. Then, the pitch rule applied to the case where the dialogue is played most during the evaluation period is set as the pitch rule to be applied to the rule fixed period after the evaluation period (step Sb19). For example, in the process of step Sb18, the three pitch rules in the evaluation period are 3 degrees, 5 degrees, and 6 degrees below, and the number of times of speech and the number of applications in each pitch rule are shown in FIG. In the case of such a value, the pitch rule applied in the rule fixed period is changed from 5 degrees below which has been set so far to 3 degrees below indicated by a solid black triangle.
Thereafter, the voice control unit 109 clears the number of times of speech and the number of times of application in the three pitch rules evaluated in the evaluation period (step Sb20), and performs the same process in the next evaluation period. The procedure returns to step Sb11.

  As described above, in the present embodiment, different pitch rules are applied during the evaluation period to synthesize answers, and if the user's utterance is within a predetermined time for the answers, The number of times of utterance and the number of times of application are updated, and if the user does not utter a response within the predetermined time, only the number of times of application of the applied pitch rule is updated. Then, at the end timing of the evaluation period, the pitch rule played by the most dialogue is set and applied to the next fixed rule period.

  Next, the pitch of the speech, the basic pitch of the voice sequence, and the pitch of the changed voice sequence will be described with specific examples.

FIG. 9A shows an example of a statement made by the user. In this figure, the linguistic analysis result of the utterance is “Tomorrow is it? (Tomorrow is sunny)?”, And the pitch of each utterance is indicated by a note as shown in the figure. This is an example. Note that the pitch waveform of the speech is actually a waveform as shown in FIG. 8B, but here, the pitch is expressed as a note for convenience of explanation.
In the example in this case, as described above, the answer creating unit 110 outputs, for example, a voice sequence of “Yes” if the weather information acquired according to the utterance is sunny, and “ Outputs the “No” audio sequence.
FIG. 9B is an example of a “Yes” speech sequence. In this example, notes are assigned to each note, and the pitch and pronunciation timing of each word (phoneme) of the basic speech are defined. ing. In this example, for simplicity of explanation, one note is assigned to one note (phoneme), but a plurality of notes may be assigned to one note such as a slur or a tie.

  If 3rd lower is applied as the pitch rule, the voice sequence by the answer creating unit 110 is changed by the voice control unit 109 as follows. That is, in the utterance shown in (a), when the pitch of the ending “ka” indicated by the symbol A is indicated as “mi” by the pitch data, the voice control unit 109 sets “Yes”. The pitch of the entire speech sequence is set so that the pitch of the section “i” at the end of the ending indicated by the symbol B becomes “do”, which is three times lower than “mi”. Is changed (see FIG. 9C).

If 5 degrees below is applied as the pitch rule, the voice sequence by the answer creating unit 110 is changed by the voice control unit 109 as follows. That is, the voice control unit 109 determines that the pitch of the section “Yes” at the end of the word “B” indicated by the symbol B is five degrees lower than the symbol “M” in the answer “Yes”. The pitch of the entire audio sequence is changed so that it becomes “L” (see FIG. 9D).
If 6 degrees below is applied as the pitch rule, the voice control unit 109 determines that the pitch of the section “I” at the end indicated by the symbol B is 6 degrees below the “M” of the code A. The pitch of the entire speech sequence is changed so that the pitch is “So” (see (e) of FIG. 9).

  Although not shown in particular, if the fourth pitch is applied as the pitch rule, the voice control unit 109 determines that the pitch of the section “I” at the end indicated by the symbol B is “M” of the symbol A. If the pitch of the entire speech sequence is changed so that the pitch is “La”, which is 4 degrees above, and 8 degrees below is applied as the pitch rule, the speech control unit 109 The pitch of the entire speech sequence is adjusted so that the pitch of the section “I” at the end of “i” is “mi”, which is 8 degrees (one octave) lower than “mi” of code A. change.

  In addition, although “Yes” has been described as an example here, although not particularly illustrated, the pitch of the entire speech sequence is similarly changed in the case of “No”. In addition, in response to the statement “What is Asuno Tenki?”, For example, when the content is specifically answered as “Hare”, the pitch of the entire speech sequence is similarly changed.

In this embodiment, since the answer is synthesized so that the pitch of the ending of the answer is related to the pitch of the ending, the answer to the utterance feels unnatural. Is not given to users.
In addition, the pitch rule applied in the rule fixed period is the pitch rule that the dialogue played most in the evaluation period before the rule fixed period. For this reason, even during the fixed rule period, the conversation is easy to play, and in short, it is easy for the user to speak. Since this pitch rule is set every evaluation period, it converges to a condition that is comfortable and reassuring for the user, and that encourages dialogue.

Second Embodiment
In the first embodiment described above, a plurality of pitch rules are applied during the evaluation period, and among them, the pitch rule that is most played by the dialog is set and used during the rule fixed period. In addition to the pitch, the factor is “between”, that is, the period from speech to answer.
Therefore, as the second embodiment, in addition to the pitch control of the answer by setting the pitch rule of the first embodiment, the answer is output between a plurality of times during the evaluation period, and the most interactive of them is An example of setting and controlling between answers by applying in the rule fixed period will be described.

The functional blocks constructed by executing the application program in the second embodiment are almost the same as those in the first embodiment (FIG. 1).
However, in the second embodiment, as the index table, in addition to the table for evaluating the pitch rule as shown in FIG. 4, for example, the output rule for the answer as shown in FIG. 10 is evaluated. A table is used.

As shown in FIG. 10, in the index table for evaluating the answer output rule, the number of utterances and the number of times of application are associated with each output rule. The output rule here defines, for example, the period from the end of a statement (end of word) to the start of the answer (start of word) when synthesizing an answer. As shown in FIG. .5 seconds, 1.0 seconds, 1.5 seconds, 2.0 seconds, and 2.5 seconds.
Note that the number of utterances and the number of application times associated with each output rule are the same as those in the first embodiment.

The operation of the second embodiment is approximately the same as the “pitch rule and output rule” in FIG. 6 and FIG. 7 in the first embodiment.
Specifically, in step Sa18 of FIG. 6, if the current time is a rule fixed period, the voice control unit 109 applies voice pitch rules and output rules set in the evaluation period before the rule fixed period to perform voice synthesis. Decide what to do. On the other hand, in step Sa19, if the current time is the evaluation period, the voice control unit 109 selects one of the three pitch rules, and the output rule set in the evaluation period immediately before the evaluation period. In the index table (see FIG. 10), any one of a total of three output rules sandwiching the output rule above and below is selected, and the selected pitch rule and output rule are applied. In step Sa21, the voice control unit 109 that has received the pitch data has a relationship in which the pitch at the end of the answer is determined by the pitch rules to be applied to the pitch indicated by the pitch data. In addition, the voice synthesizer 112 is instructed so that the period from the end of the utterance to the start of output of the answer is the period determined by the applied output rule.

In addition, the voice control unit 109 specifies the pitch rule and the output rule applied to the answer in order to update the two index tables in steps Sb14 and Sb16 in FIG. Both items of the applied pitch rule are incremented by “1”, and both items of the output rule applied to the answer are incremented by “1”. In step Sb17, only the number of application of the pitch rule applied to the answer is incremented by “1”, and only the number of application of the output rule applied to the answer is incremented by “1”.
If it is the end timing of the evaluation period, in step Sb19, the voice control unit 109 sets a pitch rule and an output rule that are applied to the case in which the most dialogue was played during the evaluation period, and thereafter, in step Sb20. Clear the pitch rule and output rule items evaluated during the evaluation period.

According to the second embodiment, since the pitch rule and the output rule that the dialogue has played most during the evaluation period are applied to the rule fixed period after the evaluation period, a favorable impression that is comfortable for the user is Will be returned in between.
For example, as shown in FIG. 11, when the user W utters “What is Asuno Tenki?”, When the speech synthesizer 10 outputs, for example, an answer “I am a crawl”, the ending of the utterance A period Ta from “ha” to “ha”, which is the beginning of the answer, is set to a period in which the user W can easily enjoy the conversation. In this case, although not shown in particular, the pitch of “su”, which is the ending of the answer, is set to a pitch rule that facilitates conversation with the pitch of “ha”, which is the ending of the statement. Is done.
Therefore, in the second embodiment, as in the first embodiment, the answer is synthesized with speech so that the pitch of the ending of the answer is related to the pitch of the harmonious pitch with respect to the pitch of the ending of the utterance. Compared with the first embodiment, since speech synthesis is performed while the answer is easy to speak, it is possible to further facilitate interaction with the user.

  In addition, in 2nd Embodiment, it was set as the structure which controls from the said speech to an answer in addition to the pitch control of the reply in 1st Embodiment, However, it separates from the said pitch control and only controls the interval. It is good also as a structure. As a configuration for controlling the interval, the “pitch rule” in FIG. 6 and FIG. 7 in the first embodiment is replaced with “output rule”. A sufficient analogy can be made from the description of the second embodiment.

<Third Embodiment>
Next, a third embodiment will be described.
The premise of the third embodiment will be briefly described. As described above, the relationship between the pitch of the ending of the utterance and the pitch that the ending of the answer feels comfortable is each person. In particular, the pitch of speech differs greatly between women and men (because women are high and men are low).
Further, in recent years, there are cases where a voice of a virtual character with a defined gender, age, etc. can be output during speech synthesis. When the voice of the character to be answered is changed, especially when the gender is changed, the user seems to have a different impression of the answer that has been received.
Therefore, in the third embodiment, a combination of the user's gender (female, male) and the voice gender to be synthesized is assumed as a scene, and an index table is prepared for each of these scenes. We decided to use an index table of scenes that corresponded to the occasion.

FIG. 12 is a diagram illustrating an example of an index table according to the third embodiment. The index table is prepared in accordance with the combination of the gender of the user and the gender of the voice to be synthesized. Specifically, as shown in the figure, the management table 127 includes four types of index tables: two types of female / male users and two types of female / male voices (devices) to answer. To be prepared.
The voice control unit 109 selects one of the four ways as follows.

Specifically, the voice control unit 109 identifies the gender of the user from, for example, personal information of the user who has logged into the terminal device as the voice synthesizer 10. Alternatively, the voice control unit 109 compares the user's remarks with a male / female pattern stored in advance by volume analysis or frequency analysis, and uses the gender of the pattern with the higher similarity. May be specified as the gender of the person. Also, the voice control unit 109 identifies the gender of the answer voice from the set information (gender information of the dialogue agent). In this way, when the voice control unit 109 specifies the gender of the user and the gender of the answer voice, the index table corresponding to the specified gender combination is selected.
After the index table is selected, the rule fixing period and the evaluation period are repeated as in the first embodiment.

According to the third embodiment, the index table of the scene corresponding to the user's utterance is used, and the pitch of the ending of the answer is set in the index table with respect to the pitch of the ending of the utterance in the rule fixed period. The pitch at the end of the answer is controlled so that the pitch rules are related, and the pitch rule played by the dialogue is set in the index table during the evaluation period.
For this reason, in 3rd Embodiment, it can be comfortable for a user corresponding to various scenes, and can make a conversation easy to play.

  In the first embodiment as well, even if the scene changes due to the repetition of the rule fixing period and the evaluation period, it will converge to a condition that is comfortable for the user and makes the conversation easy to play. The number of repetitions of the fixed period and the evaluation period) is expected to take a long time. On the other hand, in the third embodiment, if an appropriate pitch rule is set as the initial state for each scene, it is possible to shorten the time until the conversation is converged to a condition where it is easy to play.

In the third embodiment, the pitch table of the first embodiment is used as the index table. However, the output rule of the second embodiment may be used together and switched according to the scene. .
Moreover, about a scene, you may combine not only sex but age (age). The scene is not limited to the gender and age of the user and the character of the answer, the speed of speaking, the speed of answer, the use of the speech synthesizer 10, for example, voice guidance in facilities (museums, museums, zoos, etc.), automatic sales It may be prepared assuming use such as voice dialogue in the machine.

<Application examples and modifications>
The present invention is not limited to the above-described embodiments, and various applications and modifications as described below are possible, for example. In addition, one or more arbitrarily selected aspects of application / deformation described below can be appropriately combined.

<Voice input part>
In the embodiment, the voice input unit 102 is configured to input a user's voice (speech) with a microphone and convert the voice signal into a voice signal. However, the voice input unit described in the claims is limited to this configuration. I can't. That is, the voice input unit described in the claims may be configured to input or input a speech by a voice signal in some form. Specifically, the voice input unit described in the claims is configured to input a voice signal processed by another processing unit, a voice signal supplied (or transferred) from another device, It is a concept that includes an input interface circuit or the like that is built in an LSI and that simply receives an audio signal and transfers it to a subsequent stage.

<Audio waveform data>
In each embodiment, the answer creating unit 110 outputs a voice sequence in which a pitch is assigned to each note as an answer to the utterance. However, the answer is output as, for example, voice waveform data in wav format. It is good also as composition to do.
In addition, since the sound waveform data is not assigned a pitch to each sound as in the above-described sound sequence, for example, the sound control unit 109 specifies the ending pitch when simply reproduced, Configuration that outputs (reproduces) audio waveform data after performing pitch conversion (pitch conversion) such as filter processing so that the specified pitch has a predetermined relationship with the pitch indicated by the pitch data What should I do?
Also, pitch conversion may be performed by a so-called key control technique that shifts the pitch (pitch) without changing the speaking speed, which is well known in karaoke equipment.

<End of answer, beginning of answer>
In each embodiment, the pitch of the ending of the answer is controlled corresponding to the pitch of the ending of the utterance. However, depending on the language, dialect, wording, etc. It may become. In such a case, the person who made the speech unconsciously compares the pitch of the ending of the speech with the pitch of the characteristic beginning of the reply, when there is an answer to the speech. Determine the impression of the answer. Therefore, in this case, the pitch at the beginning of the answer may be controlled in accordance with the pitch at the end of the utterance. According to this configuration, when the head of the answer is characteristic, it is possible to give a psychological impression to the user who receives the answer.

  The same applies to utterances, not limited to endings, but may be determined by the beginning of a sentence. In addition, the remarks and answers are not limited to the beginning and end of the word, but may be determined based on an average pitch, or may be determined based on the pitch of the most pronounced portion. For this reason, it can be said that the 1st section of an utterance and the 2nd section of an answer are not necessarily restricted to an initial or ending.

<Pitch relationship>
In each embodiment described above, the pitch rules are exemplified as 4th, 3rd, 5th, 6th, and 8th, but other rules may be used. In addition, there is a case where a relationship of a pitch that gives a good (or bad) impression is empirically recognized even if it is not a relationship of the Kyowa pitch, so that the pitch of the answer is controlled according to the relationship of the pitch. good. However, even in this case, if the pitch between the two pitches of the ending of the utterance and the pitch of the ending of the answer is too far apart, the answer to the utterance tends to become unnatural. It is desirable that the pitch of the answer is in the range of one octave above and below.

<Others>
In the embodiment, the language analysis unit 108, the language database 122, and the answer database 124, which are configured to obtain responses to utterances, are provided on the side of the speech synthesizer 10, but the processing load is heavy in a terminal device or the like. In view of the above, the storage capacity is limited, and the like may be provided on the external server side. That is, in the speech synthesizer 10, it is sufficient that the answer creating unit 110 obtains an answer to the utterance in some form and outputs data defining the speech of the answer. It does not matter whether it is created on the side or on the side of another configuration (for example, an external server) other than the speech synthesizer 10.
Note that the information acquisition unit 126 is not necessary if the speech synthesizer 10 can create an answer to a statement without accessing an external server or the like.

DESCRIPTION OF SYMBOLS 102 ... Voice input part, 104 ... Speech section detection part, 106 ... Pitch analysis part, 108 ... Language analysis part, 109 ... Speech control part, 110 ... Answer preparation part, 112 ... Speech synthesis part, 126 ... Information acquisition part.

Claims (4)

A voice input unit for inputting a speech by a voice signal;
An acquisition unit for acquiring an answer to the statement;
The period from the input of the speech signal of the speech to the output of the speech signal of the answer is changed while being in a relationship defined by one output rule among a plurality of preset output rules. ,
Of the plurality of output rules, a voice control unit that sets one output rule that satisfies a predetermined condition within a predetermined period in which a ratio is given to the answer;
A voice control device comprising: The voice control apparatus according to claim 1, wherein the output rule is set according to any one of a plurality of scenes prepared in advance. Computer
Get an answer to the utterance from the input audio signal,
The period from the input of the speech signal of the speech to the output of the speech signal of the answer is changed while the relationship is determined by one output rule among a plurality of preset output rules. ,
A voice control method characterized in that, among the plurality of output rules, one output rule that satisfies a predetermined condition within a predetermined period is set for a rate at which a reply is made to the answer. Computer
A voice input unit for inputting a speech by a voice signal;
An acquisition unit for acquiring an answer to the remark; and
The period from the input of the speech signal of the speech to the output of the speech signal of the answer is changed while being in a relationship defined by one output rule among a plurality of preset output rules. ,
Of the plurality of output rules, a voice control unit that sets one output rule that satisfies a predetermined condition within a predetermined period in which a ratio is given to the answer;
A program characterized by functioning as

Images