JP2007187799A - Voice interaction apparatus and voice interactive method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice interaction apparatus capable of alleviating influence of estrangement of voice quality in the voice interaction apparatus which generates a response voice signal by recognizing and understanding an input voice signal. <P>SOLUTION: The voice interaction apparatus adjusts an output voice signal in a response voice output section 140 so that, regarding possibility of response voice signal omission, difference of acoustic characteristics of the voice signal (one among the fundamental frequency, energy value, or amplitude quotient (AQ) value) or difference of speech speed etc. may become lower than a predetermined value. This adjustment is performed by inserting a pause into a connection section of the voice signal, or by reducing change in the speech speed, or by reducing change of the fundamental frequency. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a voice interactive apparatus having a voice recognition function and a voice output function. In particular, the present invention relates to an improvement in the performance of a voice interaction apparatus that outputs a voice by combining a system voice signal by a voice output function and a user utterance voice signal.

  2. Description of the Related Art In recent years, there has been provided a voice dialogue apparatus that has voice recognition and voice output functions and can perform voice dialogue between a system, that is, a voice dialogue apparatus and a user. In the basic flow of voice interaction, a user inputs a voice toward the system in order to achieve a certain purpose, and the system outputs the corresponding response voice as a result of recognizing the voice. When sufficient information for accomplishing the task is obtained with a single user utterance, it will exit with audio output indicating that, but if the user's utterance content cannot be understood and input is required again, Even if the content of the utterance is successful, information sufficient to accomplish the task is not included in the content of the utterance and it is necessary to input more information. Perform voice interaction.

  In a normal voice dialogue system, voice output is performed using a recorded voice or voice synthesis in which a narrator's utterance is recorded in advance. Some of these voice output methods have a function of combining and outputting a recorded voice and a synthesized voice. For example, in the case of a navigation device, the common phrases such as “call me” and “set as destination” are the voices recorded by the narrator, and numbers such as “1 (1)” and “(2)” A facility name such as “Tokyo Disneyland” is used as a synthesized voice, and a voice such as “I will call 12-2456” or “I will set“ Tokyo Disneyland ”as the destination” is generated and output.

In addition, when constructing an agent system with a large-scale voice interaction in the future, or considering cooperating devices with voice recognition / output functions such as navigation systems and mobile phones, multiple voice output functions May be used in combination.
As another example, there is a system that generates and presents a combined voice in which a part of a user voice spoken immediately before is cut out and inserted into a response voice. This is an application method in a speech dialogue apparatus having a function of giving an index indicating the probability of a recognized word called “reliability” to speech recognition. The recognition reliability is described in detail in “Non-Patent Document 1” below.

  Here, an example of a voice conversation using a voice recognition device having a function of giving a confidence word (Word confidence) to a recognized word string and outputting the same will be described below. The following is an example of setting a destination in a navigation apparatus using voice conversation. Here, when the user utters “To Yokohama Station in Kanagawa Prefecture”, the speech recognition device recognizes “Kanagawa Prefecture” with high reliability and “Yokohama Station” with low reliability. Suppose that At this time, a response voice is output saying, “I could not understand the“ YOKOHAMAEKI ”part of Kanagawa Prefecture”. Here, “Kanagawa” and “I didn't know the part” used Narrator's voice or voice synthesis, that is, system voice, and “YOKOHAMAEKI” was the last user's utterance, ie “Kanagawa This is an audio clip of “Yokohama Station”, which is the latter half of “Yokohama Station”. The user understands from the system response voice that the latter half cannot be recognized, and speaks the “Yokohama Station” portion again. At this time, the system recognizes “Yokohama Station” with high reliability, and responds with “It is Yokohama Station in Kanagawa Prefecture. Set as the destination”.

  In order to cut out the “Yokohama Station” portion in the user's utterance, it is necessary to specify the position where the word “Yokohama Station” is uttered. For example, “prefecture name” + “garbage” in the recognition dictionary + A dictionary called “station name” is prepared, and extraction is possible by a method in which a section having the maximum likelihood with the word of the “station name” portion is set as a target section. Here, the garbage is a matching target object provided to absorb unknown words other than words registered in the dictionary and interjections (such as “no” and “ga”).

As described above, by outputting a response in which the user voice is inserted into the system voice, the user can know which word is recognized and which word is not recognized. If an unexpected noise or the like is generated when the user speaks, the output response voice also includes the noise, so that the cause of the recognition failure can be intuitively known. A smooth dialogue can be provided to the user through such appropriate audio feedback.
Akinobu Lee, Kiyohiso Shikano, and Tatsuya Kawahara, "Real-time word confidence scoring using local posterior probabilities on tree trellis search," In Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP2004), Vol. I, pp. 793-796, May 2004.

  In a spoken dialogue apparatus that uses a plurality of voice output functions as described above, a plurality of types of “voices” are combined and output. Depending on the speaking speed, the characteristics of the voice felt by the person, the so-called “voice quality”, may vary greatly. For example, the voice quality of the synthesized voice and the voice of the narrator is large, and in the technology for connecting the user voice and the system voice described above, the former may be a male voice and the latter may be a female voice. The voice quality divergence becomes remarkable.

  It is known that when voices containing such voice quality divergence are presented, there is a high possibility that the voice immediately after the voice quality changes will be missed because of the user's auditory characteristics (for example, Robert F. Potter, “The Effects of Voice Changes on Orienting and Immediate Cognitive Overload in Radio Listeners ”Media Psycology, 2000, Vol.2, pp.147-177). This auditory characteristic is a function equivalent to a filter that normally works when we “listen to sound”. In other words, we have a structure that filters (stimulates) the necessary stimulus signals from a lot of input sound information, and the necessary sounds (such as navigation system sounds) even in noisy environments such as in the passenger compartment. Can be heard. However, if the characteristics of the sound of interest change drastically, this filter must be recreated. However, there is a time lag until the filter adapts, so the sound during that time cannot be filtered normally and is overlooked. Is considered to occur.

For example, when considering a response voice in which a narrator voice prepared in advance and a user's utterance voice are combined, there is a high possibility that the voice immediately after the combined portion of these two voices will be missed. In particular, when a user's utterance is recognized with low reliability, it is important to provide a user-intuitive interface by presenting the low-reliability utterance voice as it is as the user's recorded voice. If such an oversight occurs in a system using the present invention, there is a problem that the effectiveness of this method is not fully exhibited.
In view of these problems, an object of the present invention is to provide a voice dialogue apparatus and a voice dialogue method that can reduce oversight due to voice quality divergence in response voice.

  In order to achieve the above-mentioned problem, the present invention has a function of detecting a voice quality divergence, detects the possibility of being missed based on this divergence, and detects the possibility of hearing loss in advance. The basic means is to adjust the output sound so that it is below a predetermined value. That is, the speech signal input from the speech input unit is recognized and understood by the speech understanding unit, and as a result, a word string is obtained. Based on the understanding result by the word string, the voice signals generated in the plurality of voice generation units are selected and connected. Next, the divergence of the voice qualities of the respective audio signals used for the connection of the audio signals is checked with respect to a predetermined parameter, and the possibility of overhearing is detected around the connected portion by the overhearing possibility detecting unit. Based on the detection result, the voice adjustment of the voice signal is performed by the voice adjustment unit so as to reduce the possibility of being overheard.

  With the above configuration, in the present invention, when voices with different voice qualities are combined based on the result of recognizing and understanding the input voice signal to generate response voices, the voice quality divergence is detected, and the user's The possibility of overhearing was quantified, and a function to adjust each combined speech so that the overhearing possibility was below a predetermined value was provided. As a result, when voices having a plurality of voice qualities are combined and output, it is possible to present a voice output with easy-to-hear sound quality that suppresses the possibility of being overlooked by the user.

(Embodiment 1)
In the first embodiment, the basic configuration of the present invention will be described. FIG. 1 shows this basic configuration. In FIG. 1, an arrow (a) indicates an input signal, and an arrow (b) indicates an output signal. As shown in FIG. 1, the present invention includes a voice input unit 110, a voice understanding unit 120, a response voice management unit 130, and a response voice output unit 140.

Hereafter, each part which comprises this Embodiment 1 is demonstrated with reference to FIG.
In FIG. 1, a voice input unit 110 inputs (a) a user's uttered voice and converts it into a voice signal that is an electrical signal. For example, in FIG. 2, a microphone 201 and an AD conversion unit 202 are combined. Realized. The speech understanding unit 120 in FIG. 1 has a speech recognition function that performs speech recognition on the speech signal input from the speech input unit 110 and acquires word string information obtained thereby as an understanding result. The response voice management unit 130 includes a plurality of voice generation units 130a to 130n, and based on the understanding result of the voice understanding unit 120, the voice signal to be output (hereinafter, output target voice signal) is output to each voice generation unit. It has a function of selecting and generating from 130a to 130n, connecting these generated output target audio signals, and outputting a combined audio signal. For the voice generation units 130a to 130n, a general voice synthesis method, a recorded voice reproduction method for selecting a necessary voice from a voice database recorded in advance by a narrator, or the like can be used.

  The response voice output unit 140 detects the voice quality divergence of each output target voice signal used for the connection of the combined voice signals, that is, the voice quality difference, and the user's oversight mainly on the connection portion based on the voice quality divergence. The above-described generation possibility detection unit 141 that calculates the possibility of overhearing that indicates the degree of possibility and the above-described generation so that the overhearing possibility is less than or equal to a predetermined value based on the detection result of the overhearing possibility. And a voice adjustment unit 142 that adjusts the voice quality of the generated combined voice signal so that the difference in voice quality between the output target voice signals is reduced. As described above, each function from the voice understanding unit 120 to the response voice output unit 140 can be realized by combining the arithmetic device 203 and the storage device 204 in FIG.

A specific operation of the apparatus using the above configuration will be described by taking a destination setting task in a navigation apparatus having an interactive function as an example.
In this case, the voice understanding unit 120 in FIG. 1 has a voice recognition function having a grammatical dictionary as shown in FIG. According to the grammar dictionary of FIG. 3A, a prefecture name word is stored in the prefecture name node (401a) as indicated by 402a. Further, the station name node (404a) is connected to the prefecture name 401a, and the station name corresponding to each prefecture such as 405a is stored (for convenience, 405a shows only the station name connected to Kanagawa prefecture, Actually, the station name is stored for each prefecture and connected to the corresponding prefecture name)
. With this dictionary structure, an utterance of “prefecture name + station name” can be recognized, and for example, inputs such as “Muroran City, Hokkaido” and “Atsugi City, Kanagawa Prefecture” can also be recognized. In addition, there is a node labeled “Garbage” before and after the station name node (404a). This is a node that absorbs interjections, words before and after the station name or prefecture name node, that is, an unknown word, and so on. For example, if you say "To Yokohama Station in Kanagawa" Garbage absorbs the parts of “to” and “to”, and as a result, the words “Kanagawa Prefecture” and “Yokohama Station” can be obtained correctly.

  However, having a dictionary with the above configuration waits for prefecture names and stations throughout the country as target words, so a large amount of memory is required to expand the words. Therefore, a method for reducing the required memory is shown in FIGS. This method is a method for recognizing speech speech in a plurality of stages. First, in the first stage recognition, only the dictionary of FIG. 3 (b) is expanded to recognize the prefecture name part, and based on this recognition result, The station name is recognized by developing the dictionary in FIG. 3C in which the station name for the corresponding prefecture is stored. A dictionary developed at once by this method is a prefecture name dictionary or a station name dictionary of a single prefecture, so that the memory consumed can be drastically reduced.

  Next, in the destination setting task, the response voice management unit 130 generates a response voice for the user based on the understanding result of the voice understanding unit 120. At this time, the response voice management unit 130 has a plurality of voice generation units 130a to 130n as shown in FIG. 1, and selects a voice signal generated by the appropriate voice generation units 130a to 130n from these. It also has a function of acquiring one or more voice generation results from a plurality of voice input means and connecting them. Specifically, an audio generation unit 130a that holds an output audio signal related to a navigation function such as “() is set as the destination” and “recalculates route due to traffic jam” as recording data, “(ni) The voice generation unit 130b that holds the output voice signal related to the external communication function such as “I will make a phone call”, “I am a (from) phone”, “I will download ()” as the recording data, and the voice understanding unit 120 is the object of understanding. Prefectural and facility names such as “Kanagawa” and “Yokohama Station”, or names included in user registration data and mobile phone address book data such as “Taro Nihon” and “Hanako Yokohama” Has a voice generation unit 130c, etc., which generates voice as a synthesized voice, and combines these voices into a combined voice “Kanagawa Prefecture, Yokohama Station + ...... is set as the destination”, “Nippon Taro I'll call you. "

  However, since the combined voice here is premised on being adjusted by a response voice output unit 140 described later, it is not always necessary to generate the combined voice data as actual combined voice data at this time. It suffices if the index to the obtained audio data and the connection order thereof can be referred to in a format that can be referenced.

  Here, the reason for having a plurality of sound generation functions will be described. In order to improve the quality of voice output, all voice response sentences should be unified as recorded voices by Narrator. However, when the number of patterns is enormous and new dialogs increase due to data update It is often difficult in practice, such as having to re-record. For this reason, it is common to use a narrator's voice and voice synthesis in combination. As a result, at least two kinds of voice qualities, narrator and synthesized voice, exist on the system. In addition, for example, when considering a system that uses a navigation device and a plurality of devices such as a mobile phone connected, there is a possibility that each device may have a voice recognition / output function individually. However, there are still multiple voice qualities. Apart from these system construction problems, it is also possible to have multiple audio outputs from the viewpoint of usability. For example, navigation-related functions are configured to output female narrator voices, and network-related functions such as telephone / download connections are configured to output male narrator voices. Usability is improved because it can be instantly determined whether it is a function or not.

In the response voice output unit 140, when the combined voice signal is generated in the response voice management unit 130 by the built-in overhearing possibility detection unit 141, “voice quality difference (voice quality ) ”And determine the possibility of oversight from the difference. Here, the difference in voice quality to be calculated includes at least one of parameters indicating the divergence of the acoustic characteristics of the following audio signal, and the fundamental frequency audio energy value
AQ (Amplitude Quotient) value speech rate and the like can be used, and the method of calculating these parameters indicating the difference in voice quality is as follows.
1. The fundamental frequency is also called F0 (F zero). This is a frequency generated by vocal fold vibration, and is a numerical value of “voice pitch”. It is said that it is around 150 Hz for general adult men and about 250 to 300 Hz for women. Note that the word “pitch” may be used as an index of voice pitch, but this is the time length of one cycle of the speech waveform in voiced speech, and the reciprocal of the pitch corresponds to F0. F0 obtains the residual waveform through the LPC inverse filter for the speech waveform, passes through this through the low-pass filter, and then the autocorrelation function

を求め、これを下記（数２）式

Is calculated by the following formula (2)

によって正規化した複数のピーク値の平均値(平均Ｆ０値)、或いは最大Ｆ０値と平均Ｆ０値との差分、あるいは複数のピーク値からパワーが５０ｍｓの区間で６ｄＢ以上落ちないピーク値等として検出する。
２． 音声エネルギー値は、音声の大きさ、すなわち声量に関する指標として用いる。例えば（数３）式のように、音圧の自乗の区間平均値として算出する。

Detected as an average value (average F0 value) of multiple peak values normalized by, a difference between the maximum F0 value and the average F0 value, or a peak value that does not drop more than 6 dB in a section where the power is 50 ms from the multiple peak values To do.
2. The voice energy value is used as an index related to the volume of the voice, that is, the voice volume. For example, it is calculated as a section average value of the square of the sound pressure as shown in the equation (3).

尚、該音声エネルギー値の平方根をとったものをｐとし、（数４）式

Note that p is a value obtained by taking the square root of the voice energy value, and Equation 4

として求められる「平均音圧」として扱ってもよい。

May be treated as “average sound pressure”.

The response voice output unit has a function of detecting the voice quality divergence of the plurality of output voice signals included in the combined voice signal as the divergence of the acoustic features, and if the divergence is large, the response voice output unit It has a function of processing at least one of the fundamental frequency and the sound energy in the plurality of output target audio signals so that the possibility of dropping is determined to be large based on this determination, and the possibility of being dropped is smaller than a predetermined value. ing.
3. The AQ (Amplitude Quotient) value is a value defined as the ratio of the peak-to-peak value of the vocal cord sound source waveform excluding the influence of the formant and the maximum negative peak of the differential waveform, and is said to be an index related to the softness of the voice. And literature: P.I. Alku, T .; Baeckstrom, and E.M. Vilkman, “Normalized amplified quota for parametricization of the global flow”, J. Am. Acoustic. Soc. Am. , Vol. 112, no. 2, pp. 701-710, 2002.
4). The speech rate is detected as, for example, the number of output phonemes per unit time or the number of mora per unit time. A mora is a segmental unit of sound having a certain length of time, and is also called a “beat”. There are basically combinations of vowels (such as “A”), consonants + vowels (such as “ka”), semi-vowels + vowels (such as “ya”), consonants + semi-vowels + vowels (such as “sha”). In order to calculate the number of mora, the voice generation unit needs to hold text data corresponding to each voice. In the case of speech synthesis, since the function itself includes input of a word string (phoneme string), the number of mora can be calculated from the input character string.

  Based on the result of comparing at least one of the four indices and comparing the difference, the degree of missed hearing is calculated. Here, the possibility of being overlooked may be the difference in voice quality itself, that is, a direct proportional relationship as shown in FIG. 4 (a), or whether one or more predetermined thresholds are set and whether or not the threshold is exceeded. You may judge drop possibility. FIG. 4B shows an example in which three types of threshold values a, b, and c (horizontal axis) are provided, and the possibility of overhearing is determined as A, B, and C (vertical axis) based on these threshold values. is there.

The voice adjustment unit 142 of the response voice output unit 140 performs voice quality adjustment on any item indicating the above divergence of the voice signal in order to suppress the voice drop-off possibility when the above-mentioned drop-off possibility is larger than a predetermined value. .
As a voice quality adjustment method of the audio signal,
A. B. Insert a pause (silent signal of a predetermined time length) at the boundary of the coupling parts of the output voice signals 130a to 130n. Methods such as processing the fundamental frequency, voice energy value, and speech rate of the speech to be connected can be used.
An example of the pose length to be inserted is shown in FIG. 5 for the former method (A) of inserting a pose. FIG. 5 (a) shows the correspondence of the insertion pose length when the possibility of overhearing is determined by the method of FIG. 4 (a) described above. In this method, the signal length (vertical axis) of the pause to be inserted is increased with an increase in the possibility of overhearing (horizontal axis). However, the maximum value of the insertion signal length is limited to the value “A” in FIG. The value of “A” is, for example, 1.5 seconds. Actually, it is preferably determined by an experiment or the like from the relationship between the pause time and the oversight. On the other hand, FIG. 5B shows the correspondence of the inserted pose length when the possibility of oversight is determined by the method of FIG. 4B described above. The insertion pause lengths are set as α, β, and γ when the possibility of hearing is A, B, and C, respectively. Note that as a simpler method, only one threshold may be used for the possibility of overhearing, and a fixed pause time may be uniformly inserted when the threshold is exceeded.

  A specific example of pose insertion is shown in FIG. FIG. 6 shows that the response voice management unit 130 selects voice (a) “Destination” and (b) “Set to” from the narrator voice, and generates voice (c) “Yokohama Station” as synthesized voice. This shows a case where a response voice of “the destination is set to + Yokohama station +” is generated by connecting these in the order of (a) + (c) + (b). As a result of calculating the voice quality divergence (voice quality difference) between the narrator voices (a), (b) and (c), the overhearing possibility detecting unit 141 obtains, for example, “A” in FIG. Subsequently, referring to FIG. 5B, the signal length of the pause to be inserted is determined as “α”. As a result, as shown in the right part of FIG. 6, pauses (p1) and (p2) are responded between the audio signals (a) and (c) and between the audio signals (c) and (b). Inserted at the audio output unit. Therefore, it is finally processed into an audio signal as shown in FIG.

  In this example, the pause is shown as a silence signal. However, other than this, for example, “white noise” which is a non-stationary signal or “a stationary signal having a frequency corresponding to the fundamental frequency of the audio connected behind”. And so on. Further, when the possibility of overhearing is greater than a predetermined value, a stationary signal, a non-stationary signal, or a silence signal having a time length that is directly proportional to the overhearing possibility may be inserted into the coupling boundary of the combined audio signal. In particular, in the latter case, it is possible to expect an effect of automatically recreating a human auditory filter immediately before the next output signal (that is, an effect of preparing an ear), so it can be said that the possibility of being overheard can be further reduced.

  Regarding the voice quality control (B) of the latter combining part, specifically, the possibility of overhearing is calculated based on the difference of the fundamental frequency, the speech energy value, the AQ value, and the speech rate of the speech to be combined, that is, the output target speech signal. A specific example of a method of converting the fundamental frequency, voice energy, and speech rate so as to reduce the possibility will be described. For the fundamental frequency, voice energy, and speech rate conversion method, known pitch conversion technology, volume processing technology, and speech speed conversion technology can be applied, respectively.

  In the following example, the difference in fundamental frequency, speech energy, AQ value, and speech rate is interpreted as the possibility of being overheard directly, and the fundamental frequency, speech energy, speech rate are changed so as to eliminate the difference (the AQ value The difference is resolved by processing the fundamental frequency). However, as described above, from the viewpoint of usability, there may be a case where the voice quality is changed and output. Therefore, in this case, it is preferable to deal with the above-described method A (pause insertion). That is, the response voice output unit has a function of detecting voice quality divergences of a plurality of output target audio signals included in the combined audio signal as divergences of acoustic characteristics of the audio signals, and when this divergence is large A function for determining that the possibility of overhearing is large, and a basic function in a plurality of output audio signals so that the overhearing possibility is smaller than a predetermined value when the overhearing possibility is larger than a predetermined value. A function of processing at least one of frequency and voice energy is provided. Similarly, the response voice output unit has a function of detecting voice quality divergence in a plurality of output target audio signals included in the combined voice signal as utterance speed divergence, and when the divergence is large, it is overlooked. A function for determining that the possibility is high, and at least one of the plurality of output audio signals so that the possibility of overhearing becomes smaller than a predetermined value when the overhearing possibility is larger than a predetermined value. Has a function of adjusting the speech rate.

As in FIG. 6, (a) “Destination” and (b) “Set to” are selected from the narrator voice, and voice (c) “Yokohama Station” is generated as a synthesized voice. This shows a case where a response voice “The destination is set to + Yokohama station +” connected in the order of (a) + (c) + (b) is generated. As a result of extracting the fundamental frequency for each voice,
(A) = (b) = 170 Hz,
(C) = 150 Hz
It was detected. The difference 120 Hz is directly calculated as the possibility of being overlooked, and the fundamental frequency of the response voice is shifted so as to eliminate this difference. In the example of FIG. 7, (a) and (b ') are obtained by shifting (a) and (b) and aligning the fundamental frequency to 150 Hz of (c). After this process, (a ′) + (c) + (b ′) and voice are connected and output.

  FIG. 8 detects the possibility of overhearing based on the difference in voice energy for the combined voice similar to FIG. In this example, (a) = (b) 18 dB and (c) = 38 dB are obtained. The difference of 20 dB is regarded as the possibility of being overlooked, and the energy is shifted so as to be eliminated. Specifically, predetermined energy is set in advance, and each voice is processed so as to have this predetermined energy. In the example of FIG. 8, (a ′) (b ′) (c ′) obtained by adjusting the energy of each voice so as to be equal to 30 dB is obtained, and this is obtained as (a ′) + (c) + (b ′). ) And output in the same manner as

  In FIG. 9, for the combined speech similar to FIG. 6, the response speech output unit 140 detects the possibility of being overlooked by regarding the divergence of the utterance speed as the divergence of the speech speed. Here, when the possibility of overhearing is greater than a predetermined value, the speech rate is adjusted for at least one of the plurality of output target audio signals so that the degree of overhearing is less than the predetermined value. In this example, (a) = (b) = 6 mora / second and (c) = 12 mora / second are detected, and the difference of 6 mora is the possibility of being overlooked. In order to solve this problem, speech speed conversion is performed for (c), and 6 mora / s voice (c ') equal to (a) and (b) is acquired. This is connected and output as an array of (a ′) + (c) + (b ′) as described above.

The flow of the processing procedure described above will be described with reference to the flowchart of FIG.
First, the input voice signal of the user is recognized in step S101, and a word string is extracted as the understood content. Output speech signals to be answered are obtained from the plurality of speech generation units 130a to 130n from the word string based on this understanding (step: S102). For example, in FIG. 10, two generated audio signals (a) and (b) are acquired. The connection order is determined for the acquired audio signal (step: S103). In the example of FIG. 10, the connection order is (b) + (a).

  Next, for each of the voice signals (a) and (b), a difference in voice quality based on the above-described indices such as the fundamental frequency, voice energy, AQ value, and speech rate is calculated (step: S104). As for the voice quality difference obtained here, the degree of hearing loss is calculated using the correspondence relationship between the voice quality difference and the degree of hearing loss shown in FIG. 4 (step: S105). Further, it is determined whether or not the calculated possibility of overhearing exceeds a predetermined threshold (TH) as shown in FIG. 4 (step: S106). In this case, the threshold value may be one, or a plurality of threshold values may be set as A, B, and C in FIG. Here, when the threshold value is set to a value close to 0, it becomes severe with respect to the difference in voice quality, and the possibility that the number of objects to be processed increases. Therefore, it is preferable to set the predetermined threshold value by grasping the actual situation through experiments or the like. Here, when the overhearing possibility exceeds the threshold (step: S106: NO), the process proceeds to step: S107, and when below (step: S106: YES), the process proceeds to step: S109.

  When the above-mentioned possibility of overhearing exceeds a threshold value, an audio signal to be processed is selected (step: S107). In FIG. 10, it is assumed that the audio signal (b) is selected. Regarding this selection, any one of a plurality of generated audio signals may always be selected, or a reference voice quality is retained in advance, and all voices whose voice quality deviates from this reference are overlooked. It may be determined that there is a possibility and may be processed. In addition, when using a method of changing the fundamental frequency to eliminate the difference, the sound signal itself may be distorted as the frequency shift amount increases, and the sound quality may be deteriorated. Therefore, for example, if the maximum value of the frequency shift amount is set and the difference in the fundamental frequency exceeds the maximum value, one audio signal is not aligned with the other, but both audio signals are selected as processing targets. Preferably, the fundamental frequency is processed. This processing is performed using each of the above methods (pause insertion, fundamental frequency shift, speech energy shift, speech speed conversion, etc.) to obtain a speech signal (b ') (step: S108). After obtaining the audio signal (b ′), the process returns to step S104, where the difference is detected again, and the possibility of overhearing is verified. However, in the case of the fundamental frequency shift described above, the distortion of the fundamental frequency is extremely large considering the audio distortion, and both voices are shifted to the maximum value of the fundamental frequency shift amount (the limit value of the device). If the deviation cannot be absorbed, the process may be stopped by shifting to the maximum value. In this case, it is necessary to perform a process of getting out of the loop of step S106 with some mark such as a flag indicating that the processing process is stopped although the value of the possibility of overhearing is not optimized. When the overhearing possibility is equal to or less than the threshold (step: S106: YES), the connection order (b ′) + (a) obtained in step: S103 and the processed voice result (insert pose is obtained in step: S108). Are included (step: S109). The voice signal thus generated is output and presented to the user (step: S110).

  When the voice signal including a plurality of voice qualities is concatenated and presented to the user by the above-described configuration and processing means, the possibility of overhearing based on the voice quality divergence in each voice signal is determined, and Since the voice to be connected is processed or inserted with a pose so as to suppress the possibility and presented to the user, it is possible to provide a voice dialogue apparatus that is less likely to be overheard and can perform a smoother dialogue.

(Embodiment 2)
In the second embodiment, a response sound is presented to the user using an output sound obtained by connecting an output sound (hereinafter, system sound) generated on the system side and a sound spoken by the user (hereinafter, user sound). The application form of the present invention will be described. That is, the response voice management unit includes a system voice generation unit that generates a system voice signal that is a synthesized voice signal or a pre-recorded voice signal, and a user that extracts at least part of the voice spoken by the user as a user voice signal. And a function of generating the combined audio signal as a response audio signal by combining the output of the system audio generation unit and the output of the user audio extraction unit.

  FIG. 11 is a block diagram showing an apparatus configuration according to the second embodiment. In the second embodiment, as in the case of the first embodiment, the constituent elements include a voice input unit 110, a voice understanding unit 220, a response voice management unit 230, and a response voice output unit 240. Since the basic part is common to each function, the characteristic part of the second embodiment will be described below.

  The speech understanding unit (FIG. 11: 220) performs speech recognition of a word sequence corresponding to the speech signal obtained from the speech input unit, and selects a word sequence corresponding to the speech, It has a function to determine the understanding state of the current system using a plurality of word information included. Here, the voice recognition function has a function of giving recognition reliability to each word included in the recognized word string and outputting the result as the understanding result, and a function of recording a recognition position of each word at the time of recognition. preferable. The former reliability is the probability of a word output as an understanding candidate, and a method of calculating from the closeness of recognition likelihood with a candidate word that appears at the same time, a method of calculating from a word posterior probability, etc. have been proposed. . In the second embodiment, a predetermined threshold value for the reliability is set, and the processing proceeds as recognition success for words exceeding the threshold value, and processing such as listening is performed assuming that there is no confidence in recognition for words below the threshold value. The latter (word recognition position) can be acquired at the time of collation processing with the grammar dictionary. For example, when preparing a dictionary of “prefecture name” + “garbage” + “station name” in the recognition dictionary and matching “XX station of XX prefecture”, the utterance “XX station” and the “station name” of the dictionary Since the station name is output as a recognition word when the two match with the maximum likelihood, the matching position at that time can be acquired as the speech section of “XX station”. However, if erroneous recognition occurs due to the influence of noise or the like, there is a high possibility that this voice segment is also selected incorrectly. In the dialogue apparatus in the example of the second embodiment, the user voice is presented as it is even when the voice section is erroneously recognized as described above. As a result, the user can intuitively know that the system misunderstood the speech section, so that the subsequent dialogue becomes smooth.

A specific example is shown in FIG. FIG. 12A shows the waveform of the user voice “Go to Yokohama Station in Kanagawa”. When the dictionary shown in FIG. 3A is recognized, “Kanagawa” + “no (garbage)” + “Yokohama station” + “go to (garbage)” is recognized, and the understanding result shown in FIG. As in result (1),
When "Kanagawa (high reliability)"&"Yokohama Station (high reliability)" is obtained,
Like understanding result (2),
"Kanagawa (high reliability)"&"Yokohama Station (low reliability)"
May be obtained. The response in this case will be described later. At the same time,
"Kanagawa": A1 to A2
"No (garbage)": A2-A3
"Yokohama Station": A3-A4
"Go to (garbage)": A4-A5
Is obtained.

On the other hand, FIG. 12B shows a case where background noise exists, and the amplitude of the background noise increases from time B1. FIG. 12C shows the waveform of the same utterance in this situation. When this is recognized by a similar dictionary, “Kanagawa” is recognized in the sections C1 to C2, but the subsequent audio signal Is buried in noise and cannot be cut out correctly (determined as in sections C2 to C3), and there is a high possibility that the recognition result cannot be obtained correctly. For example, as an understanding result, as an understanding result (3) in FIG.
"Kanagawa (high reliability)"&"XX (low reliability)"
(However, XX is a word that is phonemeically different from Yokohama Station.)
May be obtained. A response example in this case will be described later.

  The response voice management unit 230 includes a system voice generation unit 231 that generates a system voice signal and a user voice extraction unit 232 that extracts a user voice signal. Based on the understanding contents of the voice signal in the voice understanding unit 220, It has a function of generating a combined voice combining a system voice signal and a user voice signal. Specifically, a system voice signal is selected for a word that is determined to have high reliability in the voice understanding unit 220, and a user voice signal is selected for a word that is determined to be low in reliability. It has a function to combine signals.

  As the system voice generation unit 231, a general voice synthesis method, a recorded voice playback method using narrator voice, or the like is used. The user voice extraction unit 232 has a function of cutting out a voice signal in a section corresponding to each recognized word using the recognition position recording function of the voice understanding unit 220.

Considering the example of FIG. 12 described above, in the case of the understanding result (1), since the word reliability of both “Kanagawa Prefecture” and “Yokohama Station” is high, the corresponding system voice is selected for both words, and the response example (1) Generate a response voice such as “I will set“ Yokohama Station (system voice) ”in“ Kanagawa (system voice) ”as the destination”.
On the other hand, in the case of the understanding result (2), although “Kanagawa Prefecture” has high reliability, “Yokohama Station” does not have sufficient reliability, so the user voices in the voice sections A3 to A4 are extracted and the response example ( 2) Generate a response voice such as “I didn't understand“ Yokohama Eki (user voice) ”in“ Kanagawa (system voice) ”.
Furthermore, in the case of the understanding result 3, although “Kanagawa Prefecture” has high reliability, the reliability of “going to Yokohama Station” is not obtained at all thereafter. Therefore, the system voice is selected for “Kanagawa Prefecture”, the user voices in the sections C2 to C3 are extracted for the subsequent sections, and the response example (3) “Yokohama of“ Kanagawa Prefecture ”(system voice) A response voice such as “I don't know the part of (user voice) going to the station. Please try again” is generated. The following response voice output unit 240 functions in the case of response examples (2) and (3), and when all are recognized with high reliability as in the case of response example (1), a response voice output is performed. The audio signal is output without using the function installed in the unit 240. That is, the response voice output unit 240 has a function of detecting voice quality divergences of a plurality of output target audio signals included in the combined audio signal as divergences of acoustic characteristics of the audio signals. It is determined that the possibility is high, and has a function of adjusting the acoustic feature so that the overhearing possibility is not more than a predetermined value.

  The missed voice detection unit 241 of the response voice output unit 240 has a function of calculating the degree of missed voice by obtaining a difference in voice quality between the system voice and the user voice selected / extracted by the response voice management unit 230, and adjusting the voice. The unit 242 has a function of processing and outputting the system voice or the user voice so as to suppress the possibility of being overheard.

Hereinafter, a specific processing flow of the specific operation of the system will be described with reference to the flowchart of FIG.
The input voice is subjected to recognition processing for the input voice from the user (step: S201). As a result of the recognition process, the understanding content (Wn, Cn) is acquired. Here, Wn is the understood word, Cn is the reliability of the word, n is the number of serial numbers of the understood word (n = 1... N), Sn is the starting position on the time axis of the nth word, En Is the end position (step: S202). Next, the reliability (Cn) is compared with the reliability threshold (TH) for all the understanding words (Wn) (step: S203). If it is larger than the threshold, that is, the reliability is high (step: S203 YES), the process proceeds to step S204. If it is smaller than the threshold, that is, the reliability is low (step: S203 NO), the process proceeds to step: S205. For the understanding word (Wn (a)) with high reliability, the corresponding system voice (SWn) is selected and acquired from the system voice generator 231 (step: S204). For the understanding word (Wn (b)) with low reliability, the corresponding speech sections Sn, En are acquired, and the user's speech signal (UWn) in the section is extracted (step: S205). The system voice generation unit 231 acquires the system voice (SWn) and the supplementary voice part (SWx) other than the user voice obtained in this way (step: S206). For example, response voices such as “I don't know” or “I want to go to my destination” correspond to supplementary system voices. The connection order of the voice signals for response voice obtained as described above, that is, the system voice (SWn), user voice (UWn) and supplementary system voice (SWx) is determined (step: S207). .

  Next, the system voice (SWn), the user voice (UWn), and the supplementary system voice (SWx) are each evaluated for at least one of the fundamental frequency, voice energy, AQ value, and speech rate. Calculate (step: S208). However, before calculating these indices, it is preferable to include a process of simply estimating a noise situation other than speech. In other words, the situation in which noise is terribly mixed into the user's voice, the situation in which the input signal overflows due to the relationship between the volume of the microphone, the mounting position, the loudness of the user's voice, the loudness of the noise, etc. In such a situation, the fundamental frequency, the AQ value, the speech rate, etc. may not be correctly evaluated. Rather, this abnormal state should be directly presented to the user. Accordingly, when such a situation is detected, exception processing (processing stop processing) is performed in the audio signal processing described later. As a noise estimation method, a known method such as a method of judging from a spectrum of a signal that emphasizes or suppresses a voice frequency band through various filters, a method of giving noise as prior knowledge, or the like can be used. Further, the overflow can be detected by monitoring the input signal.

On the other hand, the calculation of the speech rate is a process unique to the second embodiment, but it is necessary to consider the speech rate even when noise is not included. That is, for the system voice (SWn) and the supplementary system voice (SWx), the number of mora can be calculated by holding a word string corresponding to the voice to be output in advance as in the first embodiment (if speech synthesis, If the user speech (UWn) is low in reliability, the number of mora may not be determined correctly in the first place. Therefore, the number of mora of other candidate words when the understanding word (Wn (b)) is detected is compared, and if the variance is low, the number of mora of UWn is used as it is, or the average number of mora of the candidate words On the other hand, if the variance is large, for example, literature (Shinichi Kawamoto et al., “Estimation of speech rate using dynamic scales,” Proceedings of the Hokuriku Branch Joint Conference of Electrical Engineering, F-69, p.369, Oct 1999 It is preferable to directly estimate the speech rate using a known method such as literature (Japanese Patent Laid-Open No. 7-295588, speech rate estimation method).

  Step: Based on the difference in voice quality calculated in S208, referring to the correspondence as shown in FIG. 4, the degree of hearing loss is calculated (step: S209). It is determined whether or not the threshold value (HT) is exceeded (step: S210). If the threshold is exceeded (step: S210 NO), the process proceeds to step: S211. If not (step: S210 YES), the process proceeds to step: S213. In the former case, that is, when the threshold value is exceeded, an audio signal to be processed is selected (step: S211). In this example, system sound (SWx) and supplementary system sound (SWx) are selected. Subsequent to this selection operation, the selected voice signal is processed (pause insertion, fundamental frequency shift, voice energy shift, speech speed conversion, etc.) to obtain the processed voice signals SWn ′ and SWx ′. (Step: S212). After the processing, the process returns to step 208 to calculate the voice quality difference again. For example, the fundamental frequency is calculated in the first difference calculation, and in the next process, the difference in voice energy is calculated. Can do. Note that, as described above, when a situation in which a lot of noise is included in the user voice portion or an overflow situation of the input signal is detected, it is preferable to take an exception processing without outputting.

  When the above processing is executed and the possibility of overhearing the output sound (SWn ′, SWx ′, UWn) including the processed sound is equal to or less than the threshold value, the sound signals are combined based on the processing result and the combination order. That is, based on the combination order acquired in step S207, the audio signals are combined to obtain (SWn '+ UWn + SWx') (step: S213). Finally, the combined voice obtained as described above is output as a response voice (step: S214), and the series of processes is terminated.

  With the above-described series of configuration and processing, when the user's voice and the system voice are combined and output, it is possible to present the voice processed so as to suppress the occurrence of oversight.

(Embodiment 3)
In the third embodiment, speech synthesis of the corresponding word is performed for a word that is determined to have high reliability by the speech understanding unit, and user speech is recognized as a result of phoneme recognition for a word that is determined to have low reliability. Speech synthesis is performed using phoneme strings.
The basic configuration of the third embodiment is the same as that of the second embodiment, and includes the voice input unit 110, the voice understanding unit 220, the response voice management unit 330, and the response voice output unit 340 shown in FIG. .
The basic parts of each function are also common to the second embodiment. Hereinafter, different parts (response voice management unit 330 and response voice output unit 340) will be described.
The response voice management unit 330 includes a system voice generation unit 331 that generates an output word string or a phoneme string (hereinafter referred to as system phoneme string) for system voice, a user voice extraction unit 332 that extracts user voice, and the user voice extraction unit A phoneme recognition unit 333 that performs phoneme recognition on the user speech extracted by 332 and obtains a user phoneme sequence, and combines the system phoneme sequence and the user phoneme sequence based on the understanding content of the speech understanding unit 220 Has a function to generate a combined phoneme sequence.

The phoneme recognizing unit 333 is a simple recognition method called “phoneme typewriter” or “subword recognition” that is simple and has a low calculation load. Recognition. Regarding phoneme recognition, for example, non-patent literature (Hiroshi Owaki, Akira Matsushima, Harald Singer, Junichi Takami (ATR), “Phoneme typewriter using constraints on phoneme array structure,” IEICE Tech. Bulletin, SP93-113,
pp.71-78, 1993). The response speech output unit 340 includes a speech synthesis unit 341. The response speech output unit 340 outputs the response speech signal by executing speech synthesis using the combined phoneme sequence obtained from the response speech management unit 330 as an input. Output via.

FIG. 15 shows a specific operation example. 15A provides the same utterance input and understanding results (1) and (2) as in FIG. 12A, and FIG. 15B provides the same utterance input and understanding results (3) as in FIG. 12C. Is the situation.
In the case of the understanding result (1), since the reliability of both “Kanagawa Prefecture” and “Yokohama Station” is high, the phoneme strings “kanakawaken” and “yokohamaeki” corresponding to both words are acquired, and the synthesized speech of the response example 1 "Kanagawaken no yokohamaekiwo mokutekikichinisetteishishima (sets Yokohama Station in Kanagawa Prefecture as the destination)" is output.

  In the case of the understanding result (2), the reliability of “Kanagawa Prefecture” is high, and the reliability of “Yokohama Station” is low. Accordingly, for “Kanagawa Prefecture”, the corresponding phoneme string “kanagawaken” is acquired. On the other hand, the portion of “Yokohama Station” extracts the corresponding user voice and performs phoneme recognition processing. As a result, for example, “okameeki” is obtained. Combining these two, the synthesized speech “kanagawakenno“ okameeki ”nobubunga wakaramaendendita (the part of“ okameeki ”in Kanagawa Prefecture was not understood)” of the response example (2) is output.

  In the case of the understanding result (3), “Kanagawa” is obtained with high reliability, but after that, the part “going to Yokohama Station” (sections C2 to C3) has sufficient reliability due to noise. I can't. Therefore, phoneme recognition is performed for the section (C2 to C3). As a recognition result, a result greatly deviating from the input voice such as “akuooomokeueiko” is obtained. This is combined with the phoneme sequence “kanagawaken” corresponding to “Kanagawa Prefecture”, and “Kanagawaken no” is an “output” of “Akuooomoe Keiko” nobubunga wakaimaendendishita.

  Since the uncertainty of the phoneme recognition is presented to the user as it is, the user can know where and how much he / she did not know, and the subsequent dialogue becomes smooth. However, if all of the above outputs are generated using the same speech synthesizer, the phoneme that is concatenated will be output as it is, such as “Kanagawa Neno okamekie nobunbungawa kawara masendendita (I did not know the part of Kanagawa Prefecture's name). Since there is a possibility that it is not possible to accurately tell which part is unknown, for example, it is preferable to perform a process such as inserting a pose immediately before “okomeeki”.

  Hereinafter, the specific flow of these processes will be described with reference to the flowchart of FIG. First, a recognition process is performed on the user voice (step: S301), and then an understanding content (Wn, Cn) is acquired as a result of the recognition process (step: S302). Here, Wn is the understood word, Cn is the reliability of the word, n is the number of serial numbers of the understood word (n = 1... N), Sn is the starting position of the nth word on the time axis, En is also the end position on the time axis. The reliability (Cn) is compared with the reliability threshold (TH) for all the understanding words (Wn) of the acquired understanding content (step: S303). If it is larger than the threshold, that is, if the reliability is high (step: S303 YES), the process proceeds to step: S304. If it is smaller than the threshold, that is, the reliability is low (step: S303: NO), the process proceeds to step: S305.

  A system phoneme string (Spn) is acquired for an understanding word (Wn (a)) with high reliability (step: S304). For the understanding word (Wn (b)) with low reliability, the corresponding speech sections Sn and En are acquired, and the user's speech signal (UWn) in the section is extracted (step: S305). Phoneme recognition is performed on the user voice signal (UWn) extracted in this way (step: S306), and then this phoneme recognition result (UPn) is acquired (step: S307). Furthermore, the phoneme string (SPx) of the supplemental part other than the system phoneme string (SPn) and the user phoneme string (UPn) is acquired (step: S308). For example, a phoneme string such as “I don't know” or “I'm going to my destination” corresponds to this. When the system phoneme string (SPn), the user phoneme string (UPn), and the supplementary system phoneme string (SPx) are acquired in this way, each of these phoneme strings (for example, SPn → UPn → SPx) is obtained. The connection order is determined (step: S309). As described above, when each processing is completed, speech synthesis is performed based on the connected phoneme sequence in the connected sound sequence, and output (step: S310).

With the series of configurations and processing means described above, all output speech can be reproduced using the same speech synthesis method, so there is no difference in voice quality. Therefore, it is possible to greatly suppress the possibility of oversight.
In the third embodiment, the case where the output voice is generated by using only the voice synthesis has been described. However, the first or second embodiment is combined with the voice generation method using the voice recorded by the narrator as the system voice. As in the case of, processing for processing speech based on detection of the possibility of overhearing may be added.

: A block diagram showing a basic configuration of the first embodiment. : A block diagram showing a device configuration of the first embodiment. The dictionary block diagram which showed the example of the recognition dictionary in speech recognition. Correspondence diagram of difference in voice quality and possibility of oversight. Correspondence diagram of the possibility of overhearing and pause insertion time. The audio | voice waveform figure in the case of the joint audio | voice production | generation which inserted the pause. The audio | voice waveform figure in the case of the joint audio | voice production | generation via a fundamental frequency shift. The audio | voice waveform figure in the case of the joint audio | voice production | generation via an audio | voice energy shift. FIG. 6 is a speech waveform diagram in the case of combined speech generation via speech speed conversion. FIG. 3 is a flowchart showing a flow of processing in the first embodiment. FIG. 3 is a block diagram illustrating a basic configuration of a second embodiment. The wave form diagram which shows the understanding result and response example of Embodiment 2. FIG. FIG. 5 is a flowchart showing a processing flow in the second embodiment. FIG. 4 is a block diagram showing a basic configuration of a third embodiment. The figure which showed the understanding result of Embodiment 3, and the example of a response. FIG. 9 is a flowchart showing a flow of processing in the third embodiment.

Explanation of symbols

110: voice input unit 120, 220: voice understanding unit 130, 230, 330: response voice management unit 130a to 130n: voice generation unit 140, 240, 340: response voice output unit 141, 241: missed possibility detection unit 142 242: Speech adjustment unit 201: Microphone 202: AD conversion unit 203: Computing device 204: Storage device 231 and 331: System speech generation unit 232, 332: User speech extraction unit 333: Phoneme recognition unit 341: Speech synthesis unit

Claims (22)

A voice input unit for acquiring at least one voice as a voice signal;
A speech understanding unit that acquires word string information corresponding to the speech signal as an understanding result;
At least two or more sound generation units for generating output sound;
A response speech management unit that selects an output target speech signal from the plurality of speech generation units based on an understanding result in the speech understanding unit, and generates a combined speech signal obtained by connecting the output target speech signals as a response speech signal;
A response voice output unit for outputting the combined voice signal;
Comprising
The response audio output unit detects a voice quality divergence between the generated output target audio signals constituting the combined audio signal, and a listening detection that detects a degree of possibility of being missed based on the divergence. The drop possibility detection unit and the generated voice quality difference between the generated voice signals so that the drop possibility indicating the degree of the drop possibility is less than or equal to a predetermined value. A voice interactive apparatus comprising: a voice adjusting unit that adjusts a combined voice signal. The spoken dialogue apparatus according to claim 1, wherein
The response voice management unit generates a system voice signal that is a synthesized voice signal or a pre-recorded voice signal, and a user voice extraction that extracts at least part of the voice spoken by the user as a user voice signal And a function of generating the combined voice signal as a response voice signal by combining the output of the system voice generation unit and the output of the user voice extraction unit. . The voice interaction apparatus according to claim 1 or 2,
The speech understanding unit has a function of recognizing a word string corresponding to a speech signal from the speech input unit, giving a recognition reliability for each word, and outputting the result as an understanding result,
The response voice management unit selects the system voice signal for a word determined to have high reliability based on an understanding result of the voice signal, and the user voice signal for a word determined to have low reliability. And a function of generating the combined voice signal obtained by combining the system voice signal and the user voice signal. The voice interactive apparatus according to any one of claims 1 to 3,
The response voice output unit has a function of detecting a voice quality divergence of a plurality of output target voice signals included in the combined voice signal as a divergence of acoustic characteristics of the voice signal. A spoken dialogue apparatus characterized by having a function of determining that the possibility is high and adjusting the acoustic characteristic so that the possibility of overhearing is equal to or less than a predetermined value. The voice interactive apparatus according to claim 4,
The acoustic dialogue device is characterized in that the acoustic feature is at least one of a fundamental frequency, an energy value, and an AQ value (Amplitude Quotient). The voice interaction apparatus according to any one of claims 1 to 3,
The response voice output unit has a function of detecting voice quality divergence of a plurality of output target audio signals included in the combined voice signal as utterance speed divergence, and can be overlooked when the utterance speed divergence is large A speech dialogue apparatus characterized by having a function of reducing the utterance speed deviation so that the degree of missed voice is determined to be less than or equal to a predetermined value. The spoken dialogue apparatus according to any one of claims 1 to 6,
The response voice output unit inserts a stationary signal, a non-stationary signal, or a silence signal having a predetermined time length at a coupling unit boundary of the coupled voice signal when the possibility of overhearing is greater than a predetermined value. Voice dialogue device. The voice interactive apparatus according to claim 7, wherein
The response voice output unit has a stationary signal or non-stationary signal or silence of a duration that is directly proportional to the possibility of hearing at the boundary of the combined voice signal when the possibility of hearing is greater than a predetermined value. A spoken dialogue apparatus characterized by having a function of inserting a signal. The voice interaction device according to claim 1, wherein
The response audio output unit has a function of detecting a deviation in voice quality of a plurality of output target audio signals included in the combined audio signal as a deviation in acoustic characteristics of the audio signal, and when the deviation is large, A function that determines that the possibility of being overlooked is large,
A function of processing at least one of a fundamental frequency and audio energy in a plurality of output audio signals so that the overhearing possibility is smaller than a predetermined value when the overhearing possibility is larger than a predetermined value; A voice interactive apparatus characterized by comprising: The voice interaction device according to claim 1, wherein
The response voice output unit has a function of detecting a voice quality divergence in a plurality of output target voice signals included in the combined voice signal as a utterance speed divergence. The ability to determine that
A function of adjusting a speech rate for at least one of a plurality of output audio signals so that the possibility of overhearing is smaller than a predetermined value when the overhearing possibility is larger than a predetermined value; A voice interactive apparatus characterized by that. A voice input unit for acquiring at least one voice as a voice signal;
A speech understanding unit that acquires a word string corresponding to the speech signal as an understanding result;
A user voice extraction unit that extracts at least part of voice spoken by the user as user voice;
A response voice management unit that determines a response voice signal based on the understanding result;
A response voice output unit for outputting the response voice signal;
Comprising
The response voice management unit includes: a system voice generation unit that generates a word string or a phoneme string recognized by the voice understanding unit; and a phoneme recognition unit that recognizes the extracted user voice and extracts a phoneme string. Have
Having a function of generating an output phoneme sequence that combines the phoneme sequence of the user speech and the phoneme sequence of the system speech generation unit output based on the understanding result of the speech understanding unit;
The response speech output unit has a function of generating an output speech by speech synthesis based on the output phoneme string. Obtain at least one or more sounds as audio signals,
Obtaining an understanding result of word string information corresponding to the voice signal;
Generate at least two output sounds,
Based on the understanding result, the plurality of output target audio signals are selected, and a combined audio signal obtained by connecting the output target audio signals is generated as a response audio signal;
Outputting the combined audio signal;
A voice interaction method,
Detecting the possibility of being overlooked indicating the degree of possibility of being overlooked based on a voice quality divergence between the generated audio signals constituting the combined audio signal, and the overhearing possibility is less than or equal to a predetermined value A voice dialogue method comprising adjusting the generated combined voice signal so that a difference in voice quality between the generated voice signals is reduced. The voice interaction method according to claim 12,
A system voice signal, which is a synthesized voice signal or a pre-recorded voice signal, is generated, at least a part of the voice spoken by the user is extracted as a user voice signal, and the system voice signal and the user voice signal are combined Generating the combined voice signal as a response voice signal. The voice interaction method according to claim 12 or 13,
Recognizing a word string corresponding to the voice signal, giving a recognition reliability for each word included in the word string, and outputting as the understanding result;
Based on the understanding result of the speech signal, the system speech signal is selected for a recognized word with high reliability, the user speech signal is selected for a recognized word with low reliability, and the system speech signal and the user speech are selected. Generating the combined voice signal in combination with a signal. The voice interaction method according to any one of claims 12 to 14,
The voice quality divergence of the plurality of output target audio signals included in the combined audio signal is detected as the divergence of the acoustic characteristics of the audio signal. If the divergence is large, it is determined that the possibility of being overlooked is high, and the listening Adjusting the acoustic feature such that the drop possibility is equal to or less than a predetermined value. The voice interaction method according to claim 15,
The audio interaction method is characterized in that the acoustic feature is at least one of a fundamental frequency, an energy value, and an AQ value (Amplitude Quotient). The voice interaction method according to any one of claims 12 to 14,
Voice quality divergence of multiple output target audio signals included in the combined speech is detected as utterance speed divergence, and when the utterance speed divergence is large, it is judged that the possibility of being overheard is large, and the oversight is possible A voice interaction method characterized by reducing the divergence of the utterance speed so that the degree is equal to or less than a predetermined value. The voice interaction method according to any one of claims 12 to 17,
Inserting a stationary signal, a non-stationary signal, or a silence signal having a predetermined time length at the boundary of the coupled audio signal when the possibility of overhearing is greater than a predetermined value;
A voice dialogue method characterized by the above. The voice interaction method according to claim 18.
When the overhearing possibility is larger than a predetermined value, a stationary signal, a non-stationary signal, or a silent signal having a length of time that is directly proportional to the predetermined value of the overhearing possibility is inserted into a boundary of the combined voice signal. A voice dialogue method characterized by that. 15. The voice interaction method according to claim 12, wherein:
Detecting a voice quality divergence of a plurality of output target audio signals included in the combined audio signal as a divergence of acoustic characteristics of the audio signal, and if the divergence is large, determining that the possibility of being overheard is high;
Processing at least one of a fundamental frequency and audio energy in a plurality of output audio signals so that the overhearing possibility is smaller than a predetermined value when the overhearing possibility is larger than a predetermined value; A featured voice interaction method. 15. The voice interaction method according to claim 12, wherein:
Detecting voice quality divergence in a plurality of output target audio signals included in the combined audio signal as utterance speed divergence, and when the divergence is large, determining that the possibility of being overheard is high,
The speech rate is adjusted for at least one of the plurality of output audio signals so that the possibility of overhearing is smaller than a predetermined value when the overhearing possibility is larger than a predetermined value. Voice dialogue method. Obtain at least one or more sounds as audio signals,
Obtaining a word string corresponding to the speech signal as an understanding result;
Extract at least part of the voice spoken by the user as user voice,
A response voice signal is determined based on the understanding result,
Outputting the response voice signal;
Phoneme recognition of the extracted user voice to extract a phoneme string,
Generating an output phoneme sequence combining the phoneme sequence and the phoneme sequence based on the understanding result;
A speech dialogue method comprising generating output speech by speech synthesis based on the output phoneme sequence.

Images