JP7361988B2 - Voice dialogue system, voice dialogue method, and voice dialogue management device - Google Patents

Description

The present disclosure relates to a voice dialogue system, a voice dialogue method, and a voice dialogue management device.

In voice dialogue systems such as car navigation systems, smart speakers, automatic telephone answering systems, etc. that are equipped with voice recognition functions, users of the voice dialogue system may interrupt the voice dialogue system even when it is outputting a response voice. A barge-in function (hereinafter referred to as barge-in) has been developed to enable voice input. On the other hand, allowing the user to barge in may have side effects in interactive processing. For example, a voice dialogue system may not be able to properly recognize speech, and when asking the user to input another utterance, it may misrecognize the continuation of the previous utterance, or the user may only hear part of the voice response voice of the voice dialogue system. In other words, the difference in the start timing of speech recognition, or in other words, the low accuracy of barge-in acceptance judgment of speech recognition, reduces the availability (usability) of the voice dialogue system. I was letting it happen.

To address these issues, conventional voice dialogue systems use the generated response voice signal as input, calculate the length of the response voice's utterance time from the signal data file capacity, and then Based on the length, the timing of starting speech recognition is controlled before the output of response speech is completed (for example, see Patent Document 1).

Japanese Patent Application Publication No. 2007-155986

However, when applying the above-mentioned conventional voice dialogue system to a system in which the voice dialogue management section and the voice input/output section are separate and independent, the voice dialogue management section and the voice input/output section are The system operates in response to the output completion timing (output completion time) of the response voice output by the system, but the systems are often interconnected by an asynchronous communication network. In such a case, since the transmission delay of the communication network varies from moment to moment, the response voice generated by the voice dialogue management section and the output completion timing of the response voice generated by the voice input/output section differ. Therefore, it is difficult to accurately detect the output completion time of the response voice output to the user.

Furthermore, due to differences in the handling of voice data between the voice dialogue management unit and the voice input/output unit, for example, differences in the sampling frequency of voice data, it is difficult to accurately detect the output completion time of the response voice from the signal data file capacity. It is also difficult to add output setting information such as the output data file size to the output signal of the response voice.

In other words, since the output completion time of the response voice cannot be calculated from response voice data with different output timings, the voice dialogue management unit cannot accurately detect the output completion time of the response voice output by the voice dialogue system to the user. As a result, there is a problem in that the accuracy of barge-in acceptance determination by voice recognition deteriorates, and the usability of the voice dialogue system deteriorates.

The present disclosure has been made in order to solve the above-mentioned problems, and even in a voice dialogue system in which the voice dialogue management unit and the voice input/output unit are configured independently, the voice dialogue management unit can provide voice information to the user. By receiving the output completion time of the response voice output by the input/output unit, it is possible to accurately detect the output completion time of the response voice output by the voice interaction management unit to the user. The purpose of this is to improve the accuracy of barge-in acceptance determination using voice recognition and to improve the usability of voice dialogue systems.

The voice dialogue system according to the present disclosure includes:
It has a voice input/output section and a voice dialogue management section,
A voice dialogue system in which a response voice generated by the voice conversation management unit is output to a user with a delay,
The audio input/output section is
a voice input unit that acquires the user's uttered voice;
an audio output unit that outputs the response voice to the user and outputs a voice output status of the response voice to the voice dialogue management unit;
The voice dialogue management unit includes:
a voice recognition unit that performs voice recognition on the user's uttered voice and outputs a voice recognition result;
an intention understanding unit that estimates the user's utterance intention from the voice recognition result and outputs an intention understanding result;
a dialogue management unit that outputs response content information to the user based on the intention understanding result;
a voice generation unit that generates an audio signal of the response voice based on the response content information and outputs it to the audio input/output unit;
a voice output information generation unit that generates voice output information that is information indicating whether or not the response voice is being outputted from the voice output status;
The apparatus further includes an input acceptance determination section that uses the voice output information to determine whether or not input to the intention understanding section can be accepted.

Further, the voice dialogue method according to the present disclosure is executed by a voice dialogue system including a voice input/output device and a voice dialogue management device that generates response voices. The voice input/output device acquires the user's uttered voice, outputs the response voice to the user, and outputs the voice output status of the response voice to the voice dialogue management device. The voice dialogue management device performs voice recognition on the user's uttered voice, estimates the user's utterance intention from the result of the voice recognition, and responds to the user based on the intention understanding result that is the result of the estimation. is determined, and based on response content information based on the response content, an audio signal of the response voice is generated and output to the audio input/output device, and when the audio output status is input, the audio signal is determined from the audio output status. , generates audio output information that is information indicating whether or not the response voice is being outputted, and uses the audio output information to determine whether or not to perform the estimation.

Further, the voice dialogue management device according to the present disclosure is a device that generates a response voice,
a voice recognition unit that performs voice recognition on the user's uttered voice and outputs a voice recognition result;
an intention understanding unit that estimates the user's utterance intention from the voice recognition result and outputs an intention understanding result;
a dialogue management unit that outputs response content information to the user based on the intention understanding result;
a voice generation unit that generates and outputs an audio signal of the response voice based on the response content information;
an audio output information generation unit that receives an audio output status that is a status in which an audio signal of the response voice is output to the user, and generates audio output information that is information indicating whether or not the response voice is being output as audio; and,
The apparatus further includes an input acceptance determination section that uses the voice output information to determine whether or not input to the intention understanding section can be accepted.

According to the present disclosure, even in a voice dialogue system in which the voice dialogue management unit and the voice input/output unit are configured separately and independently, it is possible to accurately detect the output completion time of the response voice of the voice dialogue system. As a result, it becomes possible to improve the accuracy of barge-in acceptance determination using voice recognition, and this has the effect of improving the usability of the voice dialogue system and the voice dialogue method.

1 is a block configuration diagram of a voice dialogue system in Embodiment 1. FIG. 1 is a hardware configuration diagram of a voice dialogue system in Embodiment 1. FIG. 3 is a flowchart showing the operation of the voice dialogue system in Embodiment 1. FIG. 4 is an example of the operation of the input acceptance determination unit in the first embodiment. FIG. 2 is a block diagram of a voice dialogue system according to a second embodiment. FIG. 2 is a hardware configuration diagram of a voice dialogue system according to a second embodiment. FIG. 3 is a block configuration diagram of a voice dialogue system in Embodiment 3. FIG. 7 is a flowchart showing the operation of the voice dialogue system in Embodiment 3. FIG. 3 is a block configuration diagram of a voice dialogue system in Embodiment 4. FIG. 12 is a flowchart showing the operation of the voice dialogue system in Embodiment 4.

Embodiment 1.
<<1-1>> Configuration The voice dialogue system in Embodiment 1 will be explained using FIGS. 1 to 4. FIG. 1 is a block diagram of a voice dialogue system showing the first embodiment.

In FIG. 1, a voice dialogue system 1000 includes a voice input/output section 200, a voice conversation management section 300, and a network NW.

The voice input/output unit 200 faces the user U, and performs a process of inputting voice to the voice dialogue system 1000 and presenting a response voice from the voice dialogue system 1000 to the user U. Further, the audio input/output unit 200 is built in, for example, an audio input/output device of a smart speaker.

The voice dialogue management unit 300 obtains a voice signal uttered by the user U through the network NW described later, performs voice recognition and intention understanding of the voice uttered by the user U, and generates a response voice corresponding to the user U's intention. Perform the processing to do. The generated response voice is output to the network NW. Further, the voice dialogue management unit 300 is built in, for example, a server device in a data center located away from the user U.

The network NW is a communication device that transmits and receives data between the voice input/output unit 200 and the voice dialogue management unit 300, and is, for example, a wired or wireless digital communication device such as the Internet or a LAN (Local Area Network). Note that the network NW may be a communication device that transmits audio in analog form using a telephone line and a modem.

The audio input/output unit 200 includes an audio input unit 1 and an audio output unit 7. Furthermore, the voice dialogue management section 300 includes a voice recognition section 2, an input acceptance determination section 3, an intention understanding section 4, a dialogue management section 5, a voice generation section 6, and a voice output information generation section 8. Ru.

The voice input unit 1 uses a microphone (not shown) to acquire the voice uttered by the user U, who is the user of the voice dialogue system 1000. The obtained analog audio waveform is sampled at a sampling frequency of, for example, 16 kHz using an analog/digital converter and converted into a digital audio data string. Next, acoustic analysis of the converted digital audio data string is performed, and the data is converted into, for example, 20th-order MFCC (Mel Frequency Cepstrum Coefficients), which are feature parameters used in speech recognition. . The obtained feature parameter MFCC is output to the network NW as input audio information D1.

Note that the input audio information D1 is not limited to the feature parameter MFCC. The input voice information D1 may be any information that can be subjected to voice recognition processing in the voice recognition unit 2, which will be described later, and may be, for example, a digital voice data string representing a voice waveform or an analog voice signal. In this case, acoustic analysis in the audio input section 1 can be omitted, and the amount of processing for acoustic analysis can be reduced.

The speech recognition unit 2 inputs the input speech information D1 obtained through the network NW, detects the speech start timing and the speech completion timing of the user U through speech section detection processing, and cuts out only the speech section of the user U. The utterance content of the user U is voice recognized by performing voice recognition processing on the extracted utterance voice, and text data representing the utterance content, utterance start timing, and utterance completion timing are output as a voice recognition result D2.

The utterance content of the speech recognition result D2 may be only text data representing a specific keyword included in the user U's utterance. Alternatively, it may be numerical data representing an ID or the like indicating a predetermined keyword.

The input acceptance determination unit 3 receives the voice recognition result D2 and the voice output information D8 (described later) as input, and determines whether to accept the input of the voice uttered by the user U. If the input is accepted, the input acceptance determination unit 3 receives the voice recognition result D3 that has been accepted. Output.

The intention understanding unit 4 inputs the received speech recognition result D3, estimates the intention of the input content, and outputs it as an intention understanding result D4. Here, the intention understanding result D4 may be any information that represents the utterance intention/operation content of the user U, and may be numerical data such as text data or an ID indicating the content of the text.

The dialogue management unit 5 receives the intention understanding result D4 as input, and outputs response content information D5 when a response to the user U is required.

Note that the response content information D5 may be any information necessary to generate a response sentence, such as the type and content of the response, and can take any format such as text data or numerical data.

The voice generation unit 6 receives the response content information D5, generates a response voice, and outputs it to the network NW as an output voice D6. Here, the output audio D6 is a data string representing an audio waveform.

The audio output unit 7 inputs the output audio D6 obtained through the network NW, and converts the output audio D6 into an analog audio signal using a digital/analog converter. The output voice D6 converted into an analog voice signal is output to the user U as a response voice from the voice dialogue system 1000 using a voice notification device such as a speaker (not shown).

Furthermore, the audio output unit 7 outputs the audio output status D7, which is information indicating the audio output start time or audio output completion time of the output audio D6, to the network NW. Note that the audio output status D7 may be the audio output start time of the output audio D6 and the elapsed time from the audio output start time.

The audio output information generation unit 8 receives the audio output status D7 obtained through the network NW as input, and generates and outputs audio output information D8, which is information indicating whether or not the audio output unit 7 is outputting audio. Here, the audio output information D8 may at least be information that can express whether or not audio is being output, and is not limited to the time itself. For example, the audio output information D8 may be flag information (for example, 1 during audio output, 0 when audio is stopped) that is output at predetermined intervals (for example, 0.25 msec) indicating the timing at which audio output is completed. Bye. Alternatively, a signal that can determine the timing at which audio output is completed, such as numerical information of the relative time from the start of audio output to the completion of output, text information representing time, or the number of audio data frames counted since system startup That's fine.

<1-2> Hardware Configuration Each configuration of the voice dialogue system 1000 shown in FIG. 1 can be realized by a computer that is an information processing device with a built-in CPU (Central Processing Unit). Computers with a built-in CPU include, for example, stationary computers such as personal computers and server-type computers, portable computers such as smartphones and tablet computers, or microcomputers that are incorporated into in-vehicle information systems such as car navigation systems. Such as SoC (System on Chip).

Furthermore, each configuration of the voice dialogue system 1000 shown in FIG. LSI (Large Scale It may also be realized by an integrated circuit. Moreover, each configuration of the voice dialogue system 1000 shown in FIG. 1 may be a combination of a computer and an LSI.

FIG. 2 is a block diagram showing an example of a hardware configuration of a voice dialogue system 1000 configured using an information processing device such as a computer.

In the example of FIG. 2, the audio input/output unit 200 of the audio dialogue system 1000 includes a memory 101A, a processor 102A including a CPU 110A, a recording medium 103A, an audio interface 104 (described as audio I/F in FIG. 2), and a bus. It is provided with a signal path 108A such as.

In the example of FIG. 2, the voice dialogue management unit 300 of the voice dialogue system 1000 includes a memory 101B, a processor 102B including a CPU 110B, a recording medium 103B, a network interface 105B (described as network I/F in FIG. 2), It includes a text interface 106 (described as text I/F in FIG. 2), a display interface 107 (described as display I/F in FIG. 2), and a signal path 108B such as a bus.

The memory 101A and the memory 101B include a program memory that stores various programs for realizing the voice interaction processing of the first embodiment, a work memory that is used when the processor performs data processing, a memory that expands signal data, etc. These are storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory) used as a storage device.

More specifically, each program for the audio input section 1 and the audio output section 7 can be stored in the memory 101A. Further, the memory 101A can store intermediate data such as input audio information D1, output audio D6, and audio output status D7.

More specifically, the memory 101B stores programs for the speech recognition section 2, input acceptance determination section 3, intention understanding section 4, dialogue management section 5, speech generation section 6, and speech output information generation section 8. be able to. The memory 101B also stores input voice information D1, voice recognition results D2, received voice recognition results D3, intention understanding results D4, response content information D5, output voice D6, voice output status D7, voice output information D8, etc. Data can be stored.

Processor 102A uses CPU 110A and RAM in memory 101A as a working memory, and operates according to a computer program (ie, a voice interaction program) read from ROM in memory 101A.

More specifically, the processor 102A reads programs corresponding to each process of the voice input section 1 and the voice output section 7 from the memory 101A, and processes them with the CPU 110A, thereby creating the voice dialogue shown in the first embodiment. Audio input/output processing related to processing can be executed.

Processor 102B uses CPU 110B and RAM in memory 101B as working memory, and operates according to a computer program (ie, a voice interaction program) read from ROM in memory 101B.

More specifically, the processor 102B includes programs corresponding to the processes of the speech recognition section 2, input acceptance determination section 3, intention understanding section 4, dialogue management section 5, speech generation section 6, and speech output information generation section 8. is read from the memory 101B and processed by the CPU 110B, thereby making it possible to execute the voice dialogue management process related to the voice dialogue process shown in the first embodiment.

The recording medium 103A is used to store various data such as various setting data and signal data of the processor 102A. As the recording medium 103A, it is possible to use, for example, a volatile memory such as SDRAM (Synchronous DRAM), or a nonvolatile memory such as HDD (Hard Disk Drive) or SSD (Solid State Drive). The recording medium 103A includes, for example, a startup program including an OS (Operating System), a voice dialogue system program, initial state and various setting data, constant data for control, acoustic signal data, and various data such as error information logs. can be accumulated. Note that various data in the memory 101A can also be stored in this recording medium 103A.

The recording medium 103B is used to store various data such as various setting data and signal data of the processor 102B. As the recording medium 103B, it is possible to use, for example, a volatile memory such as SDRAM, or a nonvolatile memory such as HDD or SSD. The recording medium 103B stores various data such as a startup program including the OS, a voice dialogue system program, initial state and various setting data, control constant data, acoustic signal data, and error information logs. I can do it. Note that various data in the memory 101B can also be stored in this recording medium 103B.

The acoustic interface 104 includes a microphone that acquires an audio signal uttered by the user U, and a speaker that notifies the user U of the output audio D6.

Instead of acquiring the voice uttered by the user U using a microphone, stream data acquired from another device may be input using a network interface 105A, which will be described later. Alternatively, recorded audio data stored in an external device may be selected and read through the network interface 105A. Furthermore, instead of notifying the user U of the output audio D6 through a speaker, the output audio D6 may be sent as data to another device using the network interface 105A. Note that the acoustic interface 104 can be omitted if the system inputs and outputs audio via wired or wireless communication instead of using a microphone and a speaker.

The network interface 105A and the network interface 105B transmit and receive external data using wired or This is a communication interface that performs wireless communication.

The text interface 106 is an input device for manually inputting text such as response voice content, and is composed of input devices such as a keyboard, a touch panel, and a mouse. Note that if the system does not require human input, the text interface 106 can be omitted.

The display interface 107 is a display device for displaying the voice recognition results of the input voice, the output contents of the response voice, etc., and is composed of a display device such as a display. Note that if the system does not require display on a display device, the display interface 107 can be omitted.

As described above, the voice input unit 1, voice recognition unit 2, input acceptance determination unit 3, intention understanding unit 4, dialogue management unit 5, voice generation unit 6, voice output unit 7, voice output information shown in FIG. Each function of the generation unit 8 can be realized by a memory 101A, a memory 101B, a processor 102A, a processor 102B, a recording medium 103A, and a recording medium 103B.

Note that the program for executing the voice dialogue system 1000 may be stored in a storage device inside the computer that executes the software program, or may be stored in a computer-readable external storage medium such as a CD-ROM or flash memory. It may be maintained in a distributed format and loaded and activated when the computer starts. It is also possible to obtain programs from other computers via a wireless or wired network such as a LAN (Local Area Network).

Furthermore, the program that executes the voice dialogue system 1000 can be combined with a program that is executed externally, for example, a program that executes a car navigation system or an automatic telephone answering system, and run on the same computer. Alternatively, it is possible to perform distributed processing on multiple computers.

<<1-3>> Processing Operation Next, the processing operation of the voice dialogue system of Embodiment 1 will be described using FIG. 3. FIG. 3 is a flowchart showing the processing flow of the voice dialogue system 1000 according to the first embodiment. Note that "part" in each step below may be read as "process".

In step ST1, the voice input unit 1 acquires the input voice uttered by the user U, performs acoustic analysis, and outputs the obtained feature parameter MFCC as input voice information D1 to the voice recognition unit 2 (step ST1 ).

In step ST2, the speech recognition unit 2 first detects the speech start timing and speech completion timing of the input speech by detecting the speech section of the input speech, and extracts only the speech speech of the user U from the feature parameters of the input speech. Next, voice recognition processing is performed on the extracted speech to recognize the speech content of the user U from the input speech information D1, and the speech content recognition result, speech start timing, and speech completion timing are recorded in the speech. The recognition result D2 is output to the input acceptance determination unit 3 (step ST2). Here, the speech recognition may be performed using a known speech recognition technology. For example, as described in Non-Patent Document 1, a speech recognition method based on the HMM (Hidden Markov Model) method is used to perform word-by-word recognition. , or perform sentence-by-sentence speech recognition. Further, as a method for detecting a speech section of input speech, a known method such as a comparison of the short-term power of the speech with a predetermined threshold, or cepstral analysis of the input speech can be used.

Sadahiro Furui, "Speech Information Processing", 1st edition, Morikita Publishing Co., Ltd., published June 30, 1998, p. 96-105

In step ST3, the input acceptance determination unit 3 inputs the voice recognition result D2 and the voice output information D8, and determines whether to accept the input of the voice uttered by the user U (step ST3). Here, the audio output information D8 is information indicating whether or not a response voice is being output, and is, for example, a flag indicating whether or not a response voice is being output. In this case, the response voice is being outputted, and if the value of the flag is 0, the response voice is not being output. Alternatively, it may be the output elapsed time from the response voice output start time, and if the elapsed time is not 0, it can be determined that the response voice is being output. Note that when the output of the response voice is completed, the output result time is reset to 0.

FIG. 4 shows an example of a specific operation in the input reception determining section 3 in step ST3. Hereinafter, the response voice that the voice dialogue system 1000 outputs to the user U will be abbreviated as "system utterance", and the voice that the user U inputs to the voice dialogue system 1000 will be abbreviated as "user utterance". In this example, the timing of the start and completion of system utterances is input as audio output information D8. Furthermore, in this example, the system operates so as not to accept user utterance input in the period from the start of the system utterance to the completion of the utterance.

In order to specifically compare the effects of the first embodiment of the present invention, (a) shows an example of the operation based on the system utterance output by the voice generation unit 6, and (b) shows an example of the operation based on the system utterance output by the voice generation unit 6. An example of each operation is shown below. Note that the audio of the system utterance outputted by the audio generation unit 6 is shown as the audio content in the upper row (A), and the audio of the system utterance outputted by the audio output unit 7 to the user U is shown as the audio content in the lower row (B). Illustrated. Further, "user utterance" represents the content of the audio uttered by the user U, "utterance status" represents the output status of the system utterance, and "acceptance result" represents the reception result of the input audio at the input acceptance determination unit 3, respectively. The horizontal axis is time in the voice dialogue management section 300.

In addition, in the example of the operation shown in FIG. 4, the utterance start time and utterance completion time of the system utterance (audio content in (B)) outputted by the voice output unit 7 to the user U are the response voice outputted by the voice generation unit 6. Since the data is affected by the transmission delay of the network NW, etc., the timing is different from the system utterance (the audio content of (A)) outputted by the audio generation unit 6. Specifically, "ST(A)" shown on the time axis is the start time of system utterance obtained from the audio data of the audio generator 6, and "EN(A)" is the start time of the system utterance obtained from the audio data of the audio generator 6. This is the completion time of the resulting system utterance. Further, "ST(B)" shown on the time axis is the start time of the system utterance which is the response voice outputted by the voice output unit 7 to the user U, that is, the utterance start time in the first embodiment of the present invention, “EN(B)” is the completion time of the system utterance, which is the response voice output by the voice output unit 7 to the user U, that is, the utterance completion time in the first embodiment of the present invention.

Note that the timing at which the user U starts speaking is after the output of the response voice output by the voice output unit 7 is completed, that is, when the user U hears the system utterance (i.e., the voice content in (B)) notified to the user U. Since this is after listening, the example of the operation based on the system utterance output by the voice generation unit 6 in (a) is the same as the example of the operation according to the first embodiment of the present invention in (b).

In FIG. 4, first, the voice dialogue system 1000 outputs "Please tell me your business", which is a system utterance that prompts the user U to input voice ([1] Start of speech). After the system utterance is completed ([1] utterance completed), the user U utters, "I would like a home delivery, please."

After the voice input unit 1 acquires the user's utterance, when the voice recognition unit 2 inputs the user's utterance with the utterance section divided into "delivery," "um," and "please." The speech recognition unit 2 first receives the input "Home delivery," and the voice dialogue system 1000 understands the user's utterance intention even though the user U is in the middle of speaking, and starts the system utterance, "Please tell me your address." ([2] Start of speech).

In the example of the operation shown in (a), if the user utterances ``Um,'' and ``Please.'' are input while the system is uttering ``Please tell me your address.'' Since it can be determined that the user is speaking (between [2] "ST(A)" at the start of the speech and "EN(A)" at the end of the speech), the input acceptance of the user's utterance "Um," is rejected. Ru. However, the final part of the user's utterance "Please." is considered to have been uttered after the system utterance completion time (marked "EN(A)"). The final part of the user's utterance is mistakenly accepted together with the user's utterance "Tokyo..." after the system utterance is completed, resulting in erroneous recognition.

On the other hand, in the example of the operation of the present invention shown in (b), by receiving the input of audio output information D8 including the start and completion timing of the system utterance "Please tell me your address," the user utterance "Um," " "Please." is an input for the interval from the system utterance start time ([2] utterance start "ST(B)") to the utterance completion time ([2] utterance completion "EN(B)") Since this is known, the voice dialogue system 1000 determines that the input is for the previous system utterance ``Please tell me your business.'' and rejects the input acceptance of the user utterances ``Um,'' and ``Please.'' Then, in response to the user utterance "Tokyo..." inputted after the system utterance is completed, the input of the system utterance "Please tell me your address" can be correctly accepted, and as a result, it can be correctly recognized.

That is, as shown in Embodiment 1 of the present invention, by using the audio output information D8, the output completion time of the system utterance generated by the audio generation unit 6 and the system utterance outputted to the user U by the audio output unit 7 can be determined. Since it is possible to absorb or correct the time difference between the output completion time and the output completion time of I understand. Therefore, even if the user speaks immediately after the system utterance is completed, the user's utterance can be accepted. With this operation, even if the voice dialogue system 1000 understands the intention of the user U during the utterance and proceeds to the next dialogue, it is possible to accurately prevent erroneous recognition due to the utterance of the user U regarding the previous question. be.

Note that in the example of the operation in step ST3 described above, the operation is performed so as not to accept user utterances in the section from the start time to the completion time of the system utterance, but the system is not limited to this. For example, user utterances may not be accepted within a predetermined time after the system utterance is completed, and the utterance time length is calculated from the system utterance start time and the system utterance completion time, and a predetermined percentage of the utterance length is User utterances may not be accepted until the time period has elapsed.

In addition, although FIG. 4 shows an example of an operation that uses the system utterance start when determining input acceptance, the transmission delay of the network NW and the processing delay of voice recognition are small, and the system utterance start time is the time when voice recognition is completed. In this case, the audio output status D7 and the audio output information D8 may not include information regarding the system utterance start time, that is, the information regarding the response audio output start time may not be included.

In step ST4, the intention understanding unit 4 inputs the speech recognition result D2, estimates the user U's utterance intention/operation content with respect to the voice dialogue system 1000, and outputs the intention understanding result D4 (step ST4). Note that the intention understanding processing in the intention understanding unit 4 may be performed using a known intention understanding method. For example, for each of a plurality of intention information indicating each of a plurality of intentions, intention information of an audio signal input based on a user's utterance is An intention understanding method can be used in which a score indicating the degree of suitability for a user's utterance is calculated, and intention information indicating the intention of the user's utterance is selected from a plurality of pieces of intention information based on the calculated score.

In step ST5, the dialogue management unit 5 determines the response content based on the result of understanding the intention of the user's utterance, and outputs it as response content information D5 (step ST5). Here, the dialogue management process in the dialogue management section 5 may use a known dialogue management method. For example, a response template corresponding to the dialogue state with the user is selected from response templates corresponding to predetermined dialogue states. A dialog management method may be used that selects and outputs terminology symbols included in the selected response template.

In step ST6, the voice generation unit 6 generates a response voice to be presented to the user U according to the response content information D5, and outputs it as an output voice D6 (step ST6). When the response content information D5 is text indicating the utterance content, the speech generation unit 6 may use a known speech synthesis method, for example, a text speech based on the PSOLA (Pitch Synchronous Overlap and Add) method. A synthesis method or a waveform editing text-to-speech synthesis method as described in Non-Patent Document 2 may be used. Further, when the response content information D5 is an ID linked to voice data prepared in advance, the voice generation unit 6 reads the voice data corresponding to the ID from a built-in storage device (not shown) and outputs the voice D6. It can also be output as

Sadahiro Furui, "Speech Information Processing", 1st edition, Morikita Publishing Co., Ltd., June 30, 1998, p. 73-78

In step ST7, the audio output unit 7 notifies the user U of the generated output audio D6 as a system utterance (step ST7). Furthermore, when the audio output unit 7 completes sending the audio data of the response voice, the audio output status D7, which is information indicating the audio output completion time of the response voice that is the system utterance, is sent to the audio output information generation unit through the network NW. 8 (step ST7).

Here, the timing at which the information indicating the audio output completion time is sent as the audio output status D7 is determined, for example, when all audio data is stored in the audio output buffer when outputting from a speaker, or when transmitting data to the network NW. It is fine as long as you have finished writing. Furthermore, the timing at which the information indicating the audio output start time is sent as the audio output status D7 is the time when audio data starts to be written to the audio output buffer when outputting from the speaker or to the audio transmission buffer when transmitting data to the network NW. That's fine.

In step ST8, the audio output information generation unit 8 generates audio output information D8 from the input audio output situation D7, and outputs it to the input acceptance determination unit 3 (step ST8).

Here, as an example of the operation in step ST8, as soon as the signal indicating the audio output start time or the signal indicating the audio output completion time is received as the audio output status D7, it is sufficient to output it as is as the audio output information D8. The audio output status D7 output by the output unit 7 may be used as the audio output information D8. Furthermore, if the system is configured such that there are a plurality of audio output units 7, the audio output status of each audio output unit 7 may be distinguished, for example, the ID of the audio output unit 7, etc. What is necessary is to generate the audio output information D8 to which .

In the first embodiment, after the processing in the speech recognition section 2 in step ST2, the processing in the input acceptance determination section 3 in step ST3 is performed, but after the processing in the intention understanding section 4 in step ST4, , it may be configured to execute the process in the input reception determining unit 3 in step ST3. In this case, the intention understanding unit 4 executes the intention understanding process for all the speech recognition results D2, but the input acceptance judgment unit 3 executes the input acceptance judgment process based on the intention understanding contents. Therefore, it is possible to improve the accuracy of the input acceptance determination process.

Further, if the intention understanding result D4 obtained by the intention understanding unit 4 in step ST4 is content that corresponds to the content of the dialogue with the voice dialogue system 1000, an input acceptance determination is performed according to the audio output information D8, and the content of the dialogue is determined based on the audio output information D8. If the intention understanding result D4 is unrelated to the above, it may be operated so as to always accept input without being influenced by the audio output information D8.

As described above, in the first embodiment, the audio output information generation section generates audio output information that is information indicating whether or not system utterances are being output, and the input acceptance determination section Since the system is configured to correct the output completion time of the system utterance based on the utterance and determine whether or not to accept the user utterance, it is possible to correct the output completion time of the system utterance based on the input It becomes possible for the reception determining unit to accurately grasp the timing of completion of system utterance.

In other words, by having the configuration of the first embodiment, the input acceptance determination section can eliminate the influence of the time difference between the response voice actually heard by the user and the response voice generated by the voice generation section. This makes it possible to accurately grasp the timing of system utterance completion. In other words, even in a voice dialogue system in which the voice dialogue management section and the voice input/output section are separate and independent configurations, and there is a transmission delay of the response voice, the voice dialogue system cannot accurately detect the output completion time of the response voice. can. As a result, it becomes possible to improve the accuracy of barge-in acceptance determination using voice recognition, which has the effect of improving the usability of the voice dialogue system.

In addition, since the voice output information generation unit is configured to output information on the timing of system utterance completion as voice output information, there is no need for the input reception determination unit to receive the response voice, and the input reception determination unit Compared to calculating the speaking time by analyzing the response voice, this method has the effect of reducing the amount of processing required to analyze the response voice data.

Furthermore, if there is a transmission delay in communication on the network NW and there is a delay in receiving the response voice at the input acceptance determination unit, it is possible to obtain a more accurate system utterance completion timing than when the response voice is analyzed again. This has the effect of maintaining the judgment accuracy of input reception.

In addition, since the input acceptance determination section does not need to receive the voice data of the response voice, there is no need for the voice output section to transmit the voice data of the response voice, which has the effect of reducing the processing cost and device cost. Moreover, since it is not necessary to send and receive audio data for response voices, there may be differences in the audio input/output settings, such as different sampling frequencies between the audio data output by the audio input/output unit and the audio data received by the audio dialogue management unit. This has the effect of increasing the degree of freedom in designing the voice dialogue system.

Embodiment 2.
<<2-1>> Configuration In the first embodiment described above, voice data is transmitted and received between the voice input/output section 200 and the voice dialogue management section 300 via the network NW, but the present invention is not limited to this. For example, the voice input/output unit 200 and the voice dialogue management unit 300 are arranged in the same device, but if the voice input/output unit 200 and the voice dialogue management unit 300 are configured independently, the voice input/output unit outputs The standards (for example, sampling frequency) of the audio data received by the audio dialogue management unit are often different from that of the audio data received by the audio dialogue management unit. Even in such a case, it is also possible to directly connect the voice input/output section 200 and the voice dialogue management section 300. This will be described as a second embodiment.

The voice dialogue system in Embodiment 2 will be explained using FIG. 5. FIG. 5 is a block diagram of a voice dialogue system according to a second embodiment. In FIG. 5, the same reference numerals as in FIG. 1 indicate the same or corresponding parts. Furthermore, since their configurations are the same as those shown in Embodiment 1, their explanations will be omitted.

The voice input unit 1 uses a microphone (not shown) to acquire the voice uttered by the user U, who is the user of the voice dialogue system 1000. The obtained analog audio waveform is sampled at a sampling frequency of 16 kHz, for example, and converted into a digital audio data string. Subsequently, the converted digital audio data string is subjected to acoustic analysis and converted into, for example, 20th-order MFCC, which is a feature parameter used in speech recognition. The obtained feature parameter MFCC is output to the speech recognition section 2 in the speech dialogue management section 300 as input speech information D1.

The speech recognition unit 2 inputs the input speech information D1, performs speech recognition on, for example, cutting out the speech section of the user U and the content of the cut out speech, and extracts text data representing the content of the speech, the speech start timing, and the speech. The completion timing is output as the speech recognition result D2.

The input acceptance determination unit 3 receives the voice recognition result D2 and the voice output information D8 as input, determines whether to accept the input of the voice uttered by the user U, and outputs the accepted voice recognition result D3 if the input is accepted. .

The intention understanding unit 4 inputs the received speech recognition result D3, estimates the intention of the input content, and outputs it as an intention understanding result D4.

The dialogue management unit 5 receives the intention understanding result D4 as input, and outputs response content information D5 when a response to the user U is required.

The voice generation section 6 inputs the response content information D5, generates a response voice, and outputs it to the voice output section 7 in the voice input/output section 200 as an output voice D6.

The audio output unit 7 inputs the output audio D6 obtained from the audio generator 6, and outputs the response audio from the audio dialogue system 1000 to the user U through an audio notification device such as a speaker (not shown). The output status D7 is output to the audio output information generation section 8.

The audio output information generation unit 8 receives the audio output status D7 obtained from the audio output unit 7 as input, and generates and outputs audio output information D8, which is information indicating whether or not the audio output unit 7 is outputting audio. .

<<2-2>> Hardware Configuration Each configuration of the voice dialogue system 1000 shown in FIG. 5 can be realized by a computer, which is an information processing device with a built-in CPU, in the same way as shown in the first embodiment. Computers with a built-in CPU include, for example, stationary computers such as personal computers and server-type computers, portable computers such as smartphones and tablet computers, or microcomputers that are incorporated into in-vehicle information systems such as car navigation systems. For example, SoC.

Further, each configuration of the voice dialogue system 1000 shown in FIG. 5 may be realized by an LSI that is an electric circuit such as a DSP, an ASIC, or an FPGA. Moreover, each configuration of the voice dialogue system 1000 shown in FIG. 5 may be a combination of a computer and an LSI.

FIG. 6 is a block diagram showing an example of a hardware configuration of a voice dialogue system 1000 configured using an information processing device such as a computer. In FIG. 6, the same reference numerals as those in FIG. 2 indicate the same or corresponding parts, and their configurations are the same as those shown in Embodiment 1, so the explanation will be omitted.

In the example of FIG. 6, the voice dialogue system 1000 includes a memory 101, a processor 102 including a CPU 110, a recording medium 103, an acoustic interface 104 (indicated as acoustic I/F in FIG. 6), and a network interface 105 (indicated as acoustic I/F in FIG. 6). It includes a network I/F), a text interface 106 (described as display I/F in FIG. 6), a display interface 107 (described as text I/F in FIG. 6), and a signal path 108 such as a bus. There is.

The memory 101 is a program memory that stores various programs for realizing the voice interaction processing of the second embodiment, a work memory that is used when the processor performs data processing, and a ROM that is used as a memory that expands signal data. and a storage device such as RAM.

More specifically, the memory 101 includes a voice input section 1, a voice recognition section 2, an input acceptance determination section 3, an intention understanding section 4, a dialogue management section 5, a voice generation section 6, a voice output section 7, and a voice output section. Each program of the information generation section 8 can be stored. The memory 101 also stores intermediate information such as input speech information D1, speech recognition results D2, received speech recognition results D3, intention understanding results D4, response content information D5, output speech D6, speech output status D7, and speech output information D8. Data can be stored.

Processor 102 uses CPU 110 and RAM in memory 101 as a working memory, and operates according to a computer program (ie, a voice interaction program) read from ROM in memory 101.

More specifically, the processor 102 includes a voice input unit 1, a voice recognition unit 2, an input acceptance determination unit 3, an intention understanding unit 4, a dialogue management unit 5, a voice generation unit 6, a voice output unit 7, and voice output information. By reading the programs corresponding to each process of the generation unit 8 from the memory 101 and performing the processes with the CPU 110, the voice interaction process shown in the second embodiment can be executed.

The recording medium 103 is used to store various data such as various setting data and signal data of the processor 102. As the recording medium 103, it is possible to use, for example, a volatile memory such as SDRAM, or a nonvolatile memory such as HDD or SSD. The recording medium 103 stores various data such as a startup program including the OS, a voice dialogue system program, initial state and various setting data, control constant data, acoustic signal data, and error information logs. I can do it. Note that various data in the memory 101 can also be stored in this recording medium 103.

Instead of acquiring the voice spoken by the user U using a microphone, stream data acquired from another device may be input using the network interface 105, which will be described later. Alternatively, recorded audio data stored in an external device may be selected and read through the network interface 105. Furthermore, instead of notifying the user U of the output audio D6 through a speaker, the output audio D6 may be sent as data to another device using the network interface 105. Note that the acoustic interface 104 can be omitted if the system inputs and outputs audio via wired or wireless communication instead of using a microphone and a speaker.

The network interface 105 transmits and receives external data using wired or wireless communication, such as when referring to input audio information D1, output audio D6, and audio output status D7 from data on a network, or when inputting and outputting as stream data. It is a communication interface. Note that if external data is not transmitted or received, the network interface 105 can be omitted.

As described above, the voice input unit 1, voice recognition unit 2, input acceptance determination unit 3, intention understanding unit 4, dialogue management unit 5, voice generation unit 6, voice output unit 7, voice output information shown in FIG. Each function of the generation unit 8 can be realized by a memory 101, a processor 102, and a recording medium 103.

Note that the program for executing the voice dialogue system 1000 may be stored in a storage device inside the computer that executes the software program, or may be stored in a computer-readable external storage medium such as a CD-ROM or flash memory. It may be maintained in a distributed format and loaded and activated when the computer starts. It is also possible to obtain programs from other computers via a wireless or wired network such as a LAN.

Furthermore, the program that executes the voice dialogue system 1000 can be combined with a program that is executed externally, for example, a program that executes a car navigation system or an automatic telephone answering system, and run on the same computer. Alternatively, it is possible to perform distributed processing on multiple computers.

As described above, when the audio input/output unit 200 and the audio dialogue management unit 300 are configured independently, the standards of the audio data output by the audio input/output unit and the audio data received by the audio dialogue management unit, such as sampling Frequencies are often different. In order to interconnect the audio input/output section and the audio dialogue management section, it is necessary to make the sampling frequency of the audio data sent and received by both the same, which causes a time delay in the audio data due to sampling frequency conversion. With the configuration of the second embodiment, even if there is a time delay in system utterances, the input acceptance determination unit 3 can accurately determine the system utterance completion time (output completion timing of system utterances) by using the audio output information D8. It becomes possible to detect

As described above, in this second embodiment, the voice output information generation unit generates voice output information that is information indicating whether or not system utterances are being output, and the input acceptance determination unit Since the configuration is configured to determine whether or not to accept user utterances based on It becomes possible to understand accurately.

In other words, by having the configuration of this second embodiment, the input reception determining section can eliminate the influence of the time difference between the response voice actually heard by the user and the response voice generated by the voice generation section. This makes it possible to accurately grasp the timing of system utterance completion. In other words, even in a voice dialogue system in which the voice dialogue management section and the voice input/output section are separate and independent configurations, and there is a delay in transmission of the response voice, the time at which the voice dialogue system completes outputting the response voice (the timing at which the output of system utterances completes) ) can be detected accurately. As a result, it becomes possible to improve the accuracy of barge-in acceptance determination using voice recognition, which has the effect of improving the usability of the voice dialogue system.

In addition, since the voice output information generation unit is configured to output information on the timing of system utterance completion as voice output information, there is no need for the input reception determination unit to receive the response voice, and the input reception determination unit This method also has the effect that the amount of processing for analyzing response voice data can be reduced compared to the case where the speaking time is calculated by analyzing the response voice.

In the second embodiment, the voice input/output unit 200 and the voice dialogue management unit 300 are configured to be independent, but the configuration is not limited to this, and the voice input/output unit 200 and the voice dialogue management unit 300 are It is also possible to operate within the same system, and the same effects as in the case of independent configurations are achieved.

Embodiment 3.
<<3-1>> Configuration In the first embodiment described above, the output start time or output completion time of the response voice is detected only from the voice output situation D7 generated by the voice output unit 7, but the present invention is not limited to this. Instead, it is also possible to analyze the output voice D6 and detect the output start time or output completion time of the response voice, and this will be described as a third embodiment.

The voice dialogue system in Embodiment 3 will be explained using FIG. 7. FIG. 7 is a block diagram of a voice dialogue system according to a third embodiment. In FIG. 7, the same reference numerals as those in FIG. 1 indicate the same or corresponding parts. Furthermore, since their configurations are the same as those shown in Embodiment 1, their explanations will be omitted.

The voice generation unit 6 receives the response content information D5, generates a response voice, and outputs it to the network NW as an output voice D6. Further, the time length of the output audio D6 is calculated, for example, from the size of the audio data, and the obtained time length is output as audio length information D9.

The audio output unit 7 inputs the output audio D6 obtained from the audio generator 6 through the network NW, and outputs the response audio from the audio dialogue system 1000 to the user U using an audio notification device such as a speaker (not shown). At the same time, the audio output status D7 is output to the audio output information generation section 8.

The audio output unit 7 also inputs the audio length information D9 obtained from the audio generator 6 through the network NW, and uses an information presentation device such as a display (not shown) to provide information regarding the time length of the output audio D6. For example, it is also possible to present the remaining time until the output of the response voice is completed to the user U by displaying it in text. By presenting information regarding the time length of the output voice D6 to the user U, the user U can plan the timing of his or her own utterance, and the usability of the voice dialogue system is improved.

Alternatively, using a light emitting device such as a lamp, the utterance timing may be presented to the user U based on the flashing cycle speed of the lamp. For example, when the response voice output starts, all the lamps are turned on, and as the time remaining until the response voice output is completed, the blinking cycle becomes faster, and when the lamp goes out, the response voice output is completed, so that the user U You may also present the timing of the utterance to. By presenting information regarding the time length of the output audio D6 to the user U, the user U can plan the timing of his or her own utterances, which not only improves the usability of the voice dialogue system but also provides information that is simpler than on a display. Since the presentation device can notify the user U of the utterance timing, device costs can be reduced.

The voice output information generation unit 8 acquires the voice output start time of the response voice from the voice output situation D7 obtained through the network NW. The time obtained by adding the time length of the voice length information D9 to the acquired voice output start time of the response voice is set as the voice output completion time of the response voice, and the voice output start time and the voice output completion time are output as voice output information D8.

Furthermore, the audio output information generation unit 8 can also correct the audio output situation D7 using the audio output completion time of the response voice of the audio output situation D7 and the audio length information D9.

Here, the correction of the voice output status D7 using the voice length information D9 means, for example, the output completion time of the response voice recorded in the voice output status D7 and the voice length recorded in the voice length information D9 (i.e., The method is to measure the time difference with respect to the output completion time of the output signal at predetermined intervals, and to correct it in real time based on the measured time difference. In this way, by correcting the output completion time of the audio output status D7 in real time based on the output completion time information of the audio length information D9, the transmitted response audio caused by congestion of the network NW or retransmission can be adjusted in real time. It is possible to suppress the influence of data length fluctuations, that is, transmission "fluctuations", and it is possible to accurately detect the output completion time of the response voice of the voice dialogue system.

In addition, if the audio output status D7 cannot be received due to the influence of the network NW, or if the response audio output completion time data is corrupted due to a data transmission error, the audio output obtained from the audio length information D9 is It is also possible to correct the completion time by replacing the completion time with the audio output completion time of the audio output situation D7, and even if the audio output situation D7 is not obtained, the output completion time of the response voice of the audio dialogue system can be accurately detected.

<<3-2>> Processing Operation Next, the processing operation of the voice dialogue system according to the third embodiment will be explained using FIG. 8. FIG. 8 is a flowchart showing the processing flow of the voice dialogue system 1000 according to the third embodiment. Note that "part" in each step below may be read as "process". The operations from step ST1 to step ST6 are the same as in Embodiment 1, so the explanation will be omitted.

In step ST9, the audio generation section 6 calculates the time length of the audio data of the output audio D6, and outputs it to the audio output information generation section 8 as audio length information D9 (step ST9). At this time, the time length of the audio data can be calculated from the size of the generated audio data, the audio format such as the sampling frequency, and the file format. Furthermore, when the audio data of the output audio D6 is generated by the audio synthesis method, the synthesized audio duration specified by the audio synthesis method may be used as the audio length information D9.

Furthermore, if the voice synthesis method is capable of acquiring a silent section (silent time length) at the end of the audio data of the output audio D6, the time length obtained by removing the silent time length at the end of the audio data may be used as the audio length information D9. Further, at the end of the audio data, for example, if the amplitude value is less than or equal to a predetermined threshold value, it may be regarded as a silent section, and the time length obtained by removing the silent section may be set as the audio length information D9. In addition to the method of determining a silent section using an amplitude value that is less than or equal to a predetermined threshold, a known method for determining a silent section can be used.

Furthermore, when the voice synthesis method concatenates and outputs two or more pieces of voice data prepared in advance, the voice length information D9 may be the sum of the time lengths of the concatenated voice data. Furthermore, if the voice length information D9 can be calculated before voice generation is completed, it may be output at that point. In that case, in a configuration in which audio generation and audio output are processed in parallel, it is possible to output the audio length information D9 to the audio output information generation section 8 without delay.

In step ST10, the audio output unit 7 notifies the user U of the generated output audio D6 as a system utterance (step ST10). In addition, when the audio output unit 7 completes sending the audio data of the response voice, the audio output status D7, which is information indicating the audio output completion time or audio output completion time of the response voice, which is the system utterance, is transmitted through the network NW. It is output to the audio output information generation section 8 (step ST10).

In step ST11, the audio output information generation unit 8 acquires the audio output start time of the response voice from the audio output status D7 obtained through the network NW. The time obtained by adding the time length of the voice length information D9 to the voice output start time of the acquired response voice is the voice output completion time of the response voice, and the timing including the voice output start time and the voice output completion time is the voice output information D8. Output (step ST11).

In this third embodiment, the voice generation section 6 is configured to generate the voice length information D9, but the dialogue management section 5 generates the desired voice length information D9, and the voice generation section 6 generates the voice length information D9. The output audio D6 may be generated to have the same audio length as the length information D9. In this case, the voice generation unit 6 may adjust the voice length by increasing or decreasing the speaking speed or pause length. Alternatively, the audio length may be adjusted using a known waveform conversion method.

Further, the voice output information generation section 8 may directly input the voice length information D9 from the dialogue management section 5.

As described above, in this third embodiment, the audio output information generation section receives the audio output status obtained through the network NW and the audio length information calculated by the audio generation unit, and generates information on the audio output status. By performing the correction using the voice length information, it is possible to accurately detect the output completion time of the response voice of the voice dialogue system (the main system utterance completion timing) even if there is an influence of the network NW or data transmission error. As a result, it becomes possible to improve the accuracy of barge-in acceptance determination using voice recognition, which has the effect of improving the usability of the voice dialogue system.

Furthermore, in the third embodiment, compared to the configuration of the first embodiment in which the audio output information is generated after the output of the response voice is completed, it is possible to suppress the occurrence of a delay after the output of the response voice is actually completed. , it has a remarkable effect that the output completion time of the response voice of the voice dialogue system can be detected more accurately.

In addition, in the third embodiment, the sound generation unit is configured to use the time length obtained by removing the silence time length at the end as the sound length information. It can be considered that the system utterance has been outputted for a time of . Therefore, it becomes possible to perform input acceptance determination according to audio output information that is close to the user's U's auditory sense. Therefore, it is possible to improve the accuracy of barge-in acceptance determination by voice recognition, and the usability of the voice dialogue system is further improved.

Furthermore, in the third embodiment, the audio output section is configured to receive audio length information from the audio generation section and present information regarding the duration of the output audio to the user U on a display or the like. It is possible to time the user's own utterance, and the input acceptance determination unit can make an input acceptance determination on the premise that the user U knows the remaining time for audio output of the system utterance. Therefore, it is possible to improve the accuracy of barge-in acceptance determination by voice recognition, and the usability of the voice dialogue system is further improved.

Furthermore, in this third embodiment, the voice generation section is configured to generate output voice according to the voice length information set in the dialogue management section, so that it is possible to make an input acceptance determination that takes into account the voice length of system utterances. It becomes possible. Therefore, it is possible to improve the accuracy of barge-in acceptance determination by voice recognition, and the usability of the voice dialogue system is further improved.

Embodiment 4.
<<4-1>> Configuration As another configuration example of the first embodiment described above, the input acceptance determination unit 3 outputs a signal for checking the output status of the response voice to the voice output unit 7, and outputs an arbitrary signal to the voice output unit 7. It is also possible to configure so that the output status of the response voice can be checked at the timing, and this will be described as a fourth embodiment.

The voice dialogue system in Embodiment 4 will be explained using FIG. 9. FIG. 9 is a block diagram of a voice dialogue system according to a fourth embodiment. In FIG. 9, the same reference numerals as in FIG. 1 indicate the same or corresponding parts. Furthermore, since their configurations are the same as those shown in Embodiment 1, their explanations will be omitted.

The input acceptance determination unit 3 receives the voice recognition result D2 and the voice output information D8 as input, determines whether to accept the input of the voice uttered by the user U, and outputs the accepted voice recognition result D3 if the input is accepted. . It also outputs an output status confirmation command D10, which is a signal for inquiring the output status of the response voice, to the audio output unit 7.

The audio output unit 7 inputs the output audio D6, outputs a response audio to the user U, and outputs an audio output status D7 in response to the output status confirmation command D10 from the input acceptance determination unit 3.

<<4-2>> Processing Operation Next, the processing operation of the voice dialogue system of Embodiment 4 will be described using FIG. 10. FIG. 10 is a flowchart showing the processing flow of the voice dialogue system 1000 according to the fourth embodiment. Note that "part" in each step below may be read as "process". The operations from step ST1 to step ST2 are the same as in Embodiment 1, so the explanation will be omitted.

In step ST12, the input reception determining unit 3 determines whether the user U has started speaking, and outputs an output status confirmation command D10 to the audio output unit 7 (step ST12).

In step ST13, the audio output unit 7 receives the output status confirmation command D10, and sends information indicating whether audio is currently being output or has been completed to the audio output information generation unit 8 via the network NW as an audio output status D7. Output (step ST13).

Note that although the audio output unit 7 is configured to output the audio output status D7 indicating whether or not audio is being output in response to the output status confirmation command D10, the response voice is not output for the first time after receiving the output status confirmation command D10. When the output is in the completed state, the audio output status D7 indicating that the audio output is completed may be generated, and the amount of processing for information transmission can be further reduced.

In step ST14, the audio output information generation unit 8 generates audio output information D8 from the input audio output situation D7, and outputs it to the input acceptance determination unit 3 (step ST14).

The subsequent processes from step ST3 to step ST6 are the same as in the first embodiment, and therefore the description thereof will be omitted.

In step ST15, the audio output unit 7 notifies the user U of the generated output audio D6 as a system utterance (step ST15).

As described above, in the fourth embodiment, the input acceptance determination unit is configured to output an output status confirmation command to the audio output unit and check the output status of the response voice at any timing. Since the reception determination unit can immediately obtain information regarding the response voice output status at the time when it is necessary to process the reception determination of user utterances, it is possible to improve the reception determination accuracy of barge-in voice recognition. This has the effect of further improving the usability of the voice dialogue system.

Furthermore, in the fourth embodiment, since the voice output section does not need to transmit the response voice output completion time, an additional side effect of reducing the amount of processing such as information transmission is achieved.

In each of the embodiments described above, the sampling frequency of the input audio is 16 kHz, but the invention is not limited to this. For example, an audio signal with a different sampling frequency, such as a sampling frequency of 22 kHz, may be used, and each of the above-mentioned Similar effects are achieved in each of the embodiments.

In each of the above-described embodiments, the operations were illustrated using Japanese as the language of user utterances and system utterances, but the voice dialogue system according to the present disclosure is applicable not only to Japanese, and in that case, it is applicable. What is necessary is to use a speech recognition method, an intention understanding method, and a dialogue processing method that correspond to the language used.

In addition to the above, any component of the embodiments of the present disclosure may be modified or any component of the embodiments may be omitted within the scope of the disclosure.

The voice dialogue system according to the present disclosure is suitable for use in, for example, an automatic voice response system of a call center that accepts product deliveries. For example, in the voice dialogue system 1000 according to the first embodiment, the voice input/output unit 200 is built in the voice input/output device of a smart speaker installed facing the user U, and the voice dialogue management unit 300 is built in the voice input/output device of a smart speaker installed facing the user U. , is built in a server device in a data center located away from user U.

For example, when user U utters (user utterance) about arranging the delivery of a purchased product to the smart speaker, the voice dialogue management unit 300 performs voice recognition and intention understanding of the user utterance, and responds to user U's intention. A process of generating a response voice (system utterance) is performed, and the generated system utterance is output to the network NW.

If a user utterance is input during system utterance, the input is rejected because the user is speaking during the period from the start of system utterance to the completion of utterance. Then, it operates to accept input of user utterances input after the system utterance is completed. With this operation, even if the voice dialogue system understands user U's intention mid-speech and proceeds to the next dialogue, it is possible to prevent erroneous recognition due to user U's utterance in response to the previous question. Since it is possible to output an appropriate response voice to the user U and accept the user's utterance, it can be used as an automatic voice response system with further improved functionality.

1 speech input section, 2 speech recognition section, 3 input acceptance judgment section, 4 intention understanding section, 5 dialogue management section, 6 speech generation section, 7 speech output section, 8 speech output information generation section,
101, 101A, 101B memory,
102, 102A, 102B processor,
103, 103A, 103B recording medium,
104 acoustic interface,
105, 105A, 105B network interface,
106 text interface,
107 display interface,
108, 108A, 108B signal path,
110, 110A, 110B CPU,
200 voice input/output unit, 300 voice dialogue management unit, 1000 voice dialogue system

Claims (12)

It has a voice input/output section and a voice dialogue management section,
A voice dialogue system in which a response voice generated by the voice conversation management unit is output to a user with a delay,
The audio input/output section is
a voice input unit that acquires the user's uttered voice;
an audio output unit that outputs the response voice to the user and outputs a voice output status of the response voice to the voice dialogue management unit;
The voice dialogue management unit includes:
a voice recognition unit that performs voice recognition on the user's uttered voice and outputs a voice recognition result;
an intention understanding unit that estimates the user's utterance intention from the voice recognition result and outputs an intention understanding result;
a dialogue management unit that outputs response content information to the user based on the intention understanding result;
a voice generation unit that generates an audio signal of the response voice based on the response content information and outputs it to the audio input/output unit;
a voice output information generation unit that generates voice output information that is information indicating whether or not the response voice is being outputted from the voice output status;
A voice dialogue system comprising: an input acceptance determination unit that uses the voice output information to determine whether input to the intention understanding unit can be accepted. 2. The voice dialogue system according to claim 1, wherein the voice output information includes at least output start timing and output completion timing of the response voice. 3. The voice dialogue system according to claim 2, wherein information on the output completion timing of the response voice is corrected based on voice length information of the voice signal generated by the voice generation unit. The input acceptance determination unit outputs a signal for inquiring the output status of the response voice to the voice output unit, so that the output status of the response voice can be confirmed. 3. The voice dialogue system according to any one of 3. 4. The voice dialogue system according to claim 3, wherein the voice output unit visually presents the voice utterance timing to the user. A voice dialogue management device that generates a response voice,
a voice recognition unit that performs voice recognition on the user's uttered voice and outputs a voice recognition result;
an intention understanding unit that estimates the user's utterance intention from the voice recognition result and outputs an intention understanding result;
a dialogue management unit that outputs response content information to the user based on the intention understanding result;
a voice generation unit that generates and outputs an audio signal of the response voice based on the response content information;
an audio output information generation unit that receives an audio output status that is a status in which an audio signal of the response voice is output to the user, and generates audio output information that is information indicating whether or not the response voice is being output as audio; and,
A voice dialogue management device comprising: an input acceptance determination unit that uses the voice output information to determine whether input to the intention understanding unit can be accepted. A voice dialogue method executed in a voice dialogue system including a voice input/output device and a voice dialogue management device that generates a response voice, the method comprising:
The audio input/output device is
Obtain the user's utterance audio,
outputting the response voice to the user, and outputting the voice output status of the response voice to the voice dialogue management device ;
The voice dialogue management device includes:
voice recognition of the user 's uttered voice ,
Estimating the user's utterance intention from the voice recognition result ,
determining the content of the response to the user based on the intention understanding result that is the result of the estimation ;
generating an audio signal of the response voice based on response content information based on the response content and outputting it to the audio input/output device ;
When the audio output status is input, audio output information, which is information indicating whether or not the response voice is being outputted, is generated from the audio output status, and the estimation is performed using the audio output information. Determine whether to execute or not .
Voice dialogue method. 8. The voice interaction method according to claim 7, wherein the voice output information includes at least output start timing and output completion timing of the response voice. The voice dialogue management device includes:
generating audio length information of the audio signal;
9. The voice interaction method according to claim 8 , wherein information on the output completion timing of the response voice is corrected based on voice length information of the voice signal. The voice dialogue management device includes:
The voice interaction method according to any one of claims 7 to 9, characterized in that a signal for inquiring the output status of the response voice is output to the voice input/ output device . 10. The voice interaction method according to claim 9, wherein the voice input/output device visually presents the voice utterance timing to the user. A voice dialogue management device that generates response voices is
Recognizes the user's spoken voice ,
Estimating the user's utterance intention from the voice recognition result ,
determining the content of the response to the user based on the intention understanding result that is the result of the estimation ;
generating an audio signal of the response voice based on response content information based on the response content ;
When an audio output status is input, which is a status in which an audio signal of the response voice is being output to the user, audio output is information indicating whether or not the response voice is being output as audio, based on the audio output status. generating information and using the audio output information to determine whether to perform the estimation ;
Voice dialogue method .

Images