JP2011002535A - Voice interaction system, voice interaction method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve adaptive speed of a digital filter for suppressing a system voice component included in observation signals by a microphone array in a barge-in free voice interaction system.SOLUTION: A system voice suppressing filter section 51 and ICA (Independent Component Analysis) section 52 performs filter processing for suppressing system voice included in observation signals X(f, t) to X(f, t) by the microphone array 1 on the basis of ICA algorithm, and creates first signal groups X(f, t) to X(f, t). A user voice emphasis filter section 61 and an ICA section 62 input the first signal groups X(f, t) to X(f, t), and separates user voice Z1(f, t) on the basis of the ICA algorithm. An interaction management section 71 performs recognition processing of user voice on the basis of the separated signal Z1(f, t).

Description

  The present invention relates to a barge-in-free speech dialogue system capable of recognizing a talker voice regardless of the output timing of system voice (system self-voiced sound).

2. Description of the Related Art A voice dialogue system that performs voice interaction with a user is known (see Patent Literature 1 and Non-Patent Literature 1). These voice interactive systems themselves emit voices (hereinafter referred to as system voices) to the user, and recognize user voices (hereinafter referred to as user voices) from observation signals from microphones. In addition, when the output of the system sound and the user's utterance are simultaneously performed, the system collects the mixed sound of these sounds by the microphone.
Non-Patent Document 1 discloses a system that suppresses a reduction in the recognition rate of user speech by removing system speech from a mixed sound of user speech and system speech. A system that allows a user's utterance during system voice output, such as the system disclosed in Non-Patent Document 1, is generally called a “barge-in free voice dialogue system”.

  More specifically, the speech dialogue system disclosed in Non-Patent Document 1 uses a blind source separation (BSS) technology based on independent component analysis (ICA) to separate and suppress system speech. To use. Note that ICA-based BSS is a signal processing technique for separating a desired signal (such as a user voice) that has arrived from an unknown sound source (blind sound source) from observation signals collected by a microphone array.

  Non-Patent Document 2 and Patent Document 2 disclose an example in which a BSS based on ICA similar to Non-Patent Document 1 is applied to an acoustic echo canceller. Non-Patent Document 2 and Patent Document 2 use BCA based on ICA in order to remove the acoustic echo generated when the output signal from the speaker wraps around the microphone from the observation signal from the microphone. The observation signal by the microphone is a mixed sound including the speech sound of the near-end speaker (user) and the output sound from the speaker (received signal from the far-end speaker). In Non-Patent Document 2 and Patent Document 2, the observation signal from the microphone and the received signal from the far-end speaker are input to the digital filter, and the speech of the near-end speaker (user) included in the mixed sound is suppressed.

JP 2007-248534 A JP 2004-48253 A

Sigeyuki Miyabe et al., "Barge-in- and noise-free spoken dialogue interface based on sound field control and semi-blind source separation", Proc. 15th European Signal Processing Conference (EUSIPCO 2007), pp.232-236, Sep . 2007 Ted Wada et al., "Use of decorrelation procedure for source and echo suppression", Proc. International Workshop for Acoustic Echo and Noise Control (IWAENC'08), # 9086 in Online Proceedings or CD-ROM, Sep. 2008

  FIG. 4 is a diagram illustrating a configuration example of a barge-in free speech dialogue system 800 to which the technology disclosed in Non-Patent Document 1 is applied. In FIG. 4, the microphone array 1 is composed of K macrophone elements, and observes a mixed sound in which user voice, background noise, and system voice are mixed.

  The speaker 2 outputs system sound. A sound source signal for system sound output is generated by the sound synthesizer 72, converted to an analog signal by the DA converter (DAC) 4, and then supplied to the speaker 2.

The K AD converters (ADCs) 31 to 3K perform sampling of the K observation signal groups X _j (t) (j = 1, 2,... K) by the microphone array 1. On the other hand, the ADC 30 samples the sound source signal X ₀ (t) supplied to the speaker 2 to generate a system audio sample string.

Note that the system 800 performs ICA in the frequency domain. For this reason, the sample sequence of the observation signal group X _j (t) (j = 1, 2,... K) is converted into a time / frequency domain signal group X _j (f, t) (j = 1, 2,... K). Similarly, the sample sequence of the system sound X ₀ (t) is also converted into a time / frequency domain signal X ₀ (f, t) by the short-time DFT.

The adaptive filter unit 81 receives the observation signal group X _j (f, t) and the system voice X ₀ (f, t), and three separated signals Z ₁ (f, t), Z ₂ ( f, t) and Z ₃ (f, t). Here, Z ₁ (f, t) is a separated signal estimated as user speech, Z ₂ (f, t) is a separated signal estimated as background noise, and Z ₃ (f, t) is a system signal. It is a separated signal estimated as speech.

The signal separation process based on ICA performed in the adaptive filter unit 81 can be expressed by the following equation (1). Here, W ₁₁ (f), W ₂₁ (f), W _1K (f), W _2K (f), W ₁₀ (f), W ₂₀ (f), and W ₃₀ (f) are the adaptive filter unit 81. Each of the separation filters.

Based on the ICA algorithm, the ICA unit 82 is configured so that the separated signal groups (separated signal vectors) Z ₁ (f, t), Z ₂ (f, t), and Z ₃ (f, t) are independent from each other. The filter coefficients of the respective separation filters W ₁₁ (f), W ₂₁ (f), W _1K (f), W _2K (f), W ₁₀ (f), W ₂₀ (f), W ₃₀ (f) are updated. .

The dialogue management unit 71 performs voice recognition based on the separated signal Z ₁ (f, t) corresponding to the user voice or the signal Z ₁ (t) that is returned to the time domain. Furthermore, the dialogue management unit 71 executes information processing corresponding to the recognized content of the user's utterance, and controls the speech synthesis unit 72 to generate a response message to the user.

As described above, the barge-in free speech dialogue system disclosed in Non-Patent Document 1 separates user speech, background noise, and system speech by one adaptive algorithm, that is, one ICA algorithm. However, in this case, the filter lengths (the number of filter coefficients and the number of taps) of the separation filters W ₁₀ (f), W ₂₀ (f), and W ₃₀ (f) for suppressing the system voice are set as the user voice enhancement filter. The filter length must be the same as W ₁₁ (f) and W _1K (f). Therefore, there are problems that the number of filter coefficients as optimization parameters increases, (a) the time required for filter optimization increases, and (b) it is difficult to avoid local minimum in the optimization process. .

  The present invention has been made on the basis of the above-mentioned knowledge by the inventor of the present application, and is an observation signal by a microphone array in a barge-in-free speech dialogue system capable of recognizing a dialogue voice regardless of the output timing of the system voice. It is an object of the present invention to improve the adaptive speed of a digital filter for suppressing the system sound component contained in.

A first aspect of the present invention is a barge-in-free voice interactive system capable of recognizing user voice under a situation where system voice is output from a speaker. The system includes a first filter unit, a first ICA unit, a second filter unit, a second ICA unit, and a dialogue management unit.
The first filter unit performs a filtering process on the observation signal group by the microphone array, and generates a first signal group after the filtering process.
The first ICA unit includes the first filter unit so that a sound source signal supplied to the speaker for outputting the system sound and each of the first signal groups are independent from each other. The adaptive filter group included in is optimized based on the independent component analysis algorithm.
The second filter unit performs a filtering process on the first signal group to generate a second signal group after the filtering process.
The second ICA unit optimizes the adaptive filter group included in the second filter unit based on an independent component analysis algorithm so that the second signal group is independent from each other.
The dialogue management unit performs dialogue management including recognition processing of the user voice based on a signal included in the second signal group and generation processing of a new system voice based on a recognition processing result.

  In the first aspect of the present invention described above, the optimization of the filter coefficient group included in the first filter unit for system speech suppression is performed independently from the second filter unit for user speech enhancement. Based on the algorithm. As a result, the filter length of the adaptive filter group included in the first filter unit that suppresses system speech can be made shorter than that of the adaptive filter group included in the second filter unit. Therefore, according to the first aspect of the present invention, it is possible to optimize the filter coefficient of the first filter unit at high speed. In addition, the probability of being caught by the local minimum in the optimization process can be reduced.

  According to the first aspect of the present invention described above, in the barge-in free speech dialogue system, it is possible to improve the adaptive speed of the digital filter for suppressing the system speech component included in the observation signal by the microphone array.

It is a block diagram which shows the whole structure of the voice dialogue system concerning embodiment of this invention. It is a block diagram which shows the structural example of the system audio | voice suppression filter part which the audio | voice dialog system shown in FIG. 1 has. It is a block diagram which shows the structural example of the user audio | voice emphasis filter part which the speech dialogue system shown in FIG. 1 has. It is a block diagram which shows the structure of the barge-in free voice interactive system concerning background art.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted as necessary for the sake of clarity.

<Embodiment 1 of the Invention>
FIG. 1 is a block diagram showing an overall configuration of a barge-in free speech dialogue system 100 according to the present embodiment. Note that, among the components included in the voice interaction system 100, the same components as those in the voice interaction system 800 illustrated in FIG. 4 are denoted by the same reference numerals as those in FIG. Here, a duplicate description of these common components is omitted.

In FIG. 1, a system sound suppression filter unit 51 and an ICA unit 52 perform system sound suppression processing based on ICA on the observation signal groups X _{1 to} X _K obtained by the microphone array 1. Then, the user speech enhancement filter unit 61 and the ICA unit 62 perform user speech enhancement processing based on ICA on the signal groups Y _{1 to} Y _K after the suppression of the system speech. That is, the voice interaction system 100 performs “system voice suppression” and “user voice enhancement” by two independent ICA algorithms.

By the way, many of the filter coefficients that require a lot of calculation time in adaptive learning of the digital filter depend on the length (that is, time length) of the spatial transfer function to be estimated. Empirically, the length of the spatial transfer function between the microphone array 1 and the sound source has the following two properties. That is,
(1) The sound source closest to the microphone array 1 is the speaker 2 included in the system 100.
(2) Although the distance from the microphone array 1 to the sound source of the user and background noise is relatively short, it is far compared to the distance from the microphone array 1 to the speaker 2.
Therefore, the adaptive filter length for estimating the system voice is essentially shorter than the adaptive filter length for estimating the user voice and the background noise.

  Therefore, in the spoken dialogue system 100 that performs “system voice suppression” and “user voice enhancement” by two independent ICA algorithms, the filter length of the adaptive filter group included in the filter unit 51 for system voice suppression is set to the user's filter length. It can be made shorter than the adaptive filter group included in the filter unit 61 for speech enhancement. Therefore, according to the spoken dialogue system 100, it is possible to optimize the filter coefficient group of the system voice suppression filter unit 51 at high speed. In addition, the probability of being caught by the local minimum in the optimization process can be reduced.

  Below, the specific structural example of the filter part 51, the ICA part 52, the filter part 61, and the ICA part 62 is demonstrated. Here, a case where frequency domain ICA is performed will be described.

FIG. 2 is a block diagram illustrating a configuration example of the filter unit 51 and the ICA unit 52. The filter unit 51 illustrated in FIG. 2 includes K adaptive filters B ₁₀ (f) to B _K0 (f). Filters B ₁₀ (f) to B _K0 (f) and ICA units 521 to 52K, which will be described later, estimate the transfer function of the propagation path of the system sound reaching the microphone elements 11 to 1K from the speaker 2.

The K adaptive filters B ₁₀ (f) to B _K0 (f) are arranged in association with the _K observation signals X ₁ (f, t) to X _K (f, t). Each of the filters B ₁₀ (f) to B _K0 (f) receives the sound source signal X ₀ (f, t) corresponding to the system sound, and changes the frequency characteristics of X ₀ (f, t). Outputs of the filters B ₁₀ (f) to B _K0 (f) are supplied to adders (subtracters) 511 to 51K, respectively.

The adders 511 to 51K subtract the output signals of the filters B ₁₀ (f) to B _K0 (f) from the observation signals X ₁ (f, t) to X _K (f, t), so that the signal Y ₁ ( f, t) to Y _K (f, t) are generated. The signals Y ₁ (f, t) to Y _K (f, t) are supplied to the subsequent user speech enhancement filter unit 61 as an observation signal group after system speech suppression.

The filtering process by the filter unit 51 shown in FIG. 2 can be expressed by the following equation (2).

The ICA unit 52 updates the filter coefficients of the adaptive filters B ₁₀ (f) to B _K0 (f) based on the ICA algorithm. In the example of FIG. 2, the ICA unit 52 includes K ICA units 521 to 52K corresponding to K adaptive filters B ₁₀ (f) to B _K0 (f).

For example, the ICA unit 521 uses the adaptive filter B ₁₀ so that the sound source signal X ₀ (f, t) corresponding to the system sound and the filtered output signal Y ₁ (f, t) are independent from each other according to the ICA algorithm. The filter coefficient group in (f) is updated. Similarly, the ICA unit 52K follows the ICA algorithm so that the sound source signal X ₀ (f, t) corresponding to the system sound and the filtered output signal Y _K (f, t) are independent from each other. The filter coefficient group of _K0 (f) is updated.

Mutual information (Kullback-leibler divergence), higher-order statistics (kurtosis: Kurtosis), and the like are used as evaluation criteria for independence in ICA. The update of the filter coefficients in the ICA units 521 to 52K may be performed by ICA using mutual information or higher-order statistics. As an example, Equation (3) shows a filter coefficient update formula that applies a method of maximizing mutual information known as the Infomax method. In equation (3), the function φ (Y) is a probability density function of the sound source signal. α is an update coefficient (learning rate). In the formula (3), <A> _t represents the time average of A. [I] represents the number of updates of the filter coefficient. J in Formula (3) is an integer of 1 to K. When an audio signal is handled as in this embodiment, φ (Y) may be approximated by a sigmoid function.

In the case of the frequency domain ICA, the observation signals X ₁ (t) to X _K (stored in the memory) are used for adaptive learning (that is, update of the filter coefficients) of the adaptive filters B ₁₀ (f) to B _K0 (f). After the sample data sequence of t) and X ₀ (t) is divided in a short time frame unit, the data sequence in the frequency domain after DFT is performed on the divided sample sequence is used. That is, it is necessary to store a sample data string for at least one frame in advance. Therefore, when performing speech recognition in real time, the adaptive filters B ₁₀ (f) to B _K0 (f) whose filter coefficients are updated based on the sample data sequence one or several frames before are used to A system voice suppression process may be performed on the sample data string.

Next, a configuration example of the user voice enhancement filter unit 61 and the ICA unit 62 will be described with reference to FIG. The filter unit 61 illustrated in FIG. 3 includes 2K adaptive filters V ₁₁ (f), V ₂₁ (f) to V _1K (f), and V _2K (f). These filters are arranged in association with the K output signals Y ₁ (f, t) to Y _K (f, t) from the filter unit 51.

For example, the adaptive filters V ₁₁ (f) and V ₂₁ (f) receive the signal Y ₁ (f, t) and change the frequency characteristics of Y ₁ (f, t). The adaptive filters V _1K (f) and V _2K (f) change the frequency characteristics of the signal Y _K (f, t).

The adder 611 adds the K signals output from the filters V ₁₁ (f) to V _1K (f) to generate a separated signal Z ₁ (f, t). The adder 612 adds K signals output from the filters V ₂₁ (f) to V _2K (f) to generate a separated signal Z ₂ (f, t). The separated signal Z ₁ (f, t) is an estimated signal of user speech, and the separated signal Z ₂ (f, t) is an estimated signal of background noise.

The filtering process by the filter unit 61 shown in FIG. 3 can be expressed by the following equation (4).

The ICA unit 62 performs 2K adaptive filters V ₁₁ (f) and V ₂₁ (f) so that the separated signals Z ₁ (f, t) and Z ₂ (f, t) are independent of each other according to the ICA algorithm. Update processing of filter coefficients of .about.V _1K (f) and V _2K (f) is performed. As an example, an equation for updating the filter coefficient using the Infomax method is shown in Equation (5). The function φ (Y) in equation (5) may be approximated by a sigmoid function.

  As shown in FIGS. 2 and 3, by performing “suppression of system speech” and “emphasis of user speech” using the frequency domain ICA, an effect of reducing the amount of calculation can be obtained. The acoustic echo canceller based on ICA disclosed in Patent Literature 2 performs adaptive filter learning using ICA in the time domain. For this reason, in the method of Patent Document 2, it is necessary to handle a mixing problem including time delay, that is, convolutional mixing. On the other hand, since the specific example described with reference to FIGS. 2 and 3 uses the frequency domain ICA, the instantaneous mixing problem may be solved by applying ICA for each frequency bin. For this reason, compared with the method of patent document 2 using time domain ICA, the amount of calculations can be reduced significantly and the convergence speed of an adaptive filter can be improved.

  By the way, the filter processing by the filter units 51 and 61 and the filter coefficient update processing by the ICA units 51 and 62 can be realized by using a semiconductor processing device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). Good. These filter processing and filter coefficient update processing may be realized by causing a computer system including a DSP (Digital Signal Processor), a microprocessor, and the like to execute a program.

  A program including an instruction group for causing a computer system to perform filter processing and filter coefficient update processing can be stored in various types of storage media, and can be transmitted via a communication medium. It is. Here, the storage medium includes, for example, a flexible disk, a hard disk, a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD, a ROM cartridge, a battery-backed RAM memory cartridge, a flash memory cartridge, a nonvolatile RAM cartridge, and the like. . In addition, the communication medium includes a wired communication medium such as a telephone line, a wireless communication medium such as a microwave line, and the Internet.

  Furthermore, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention described above.

100 Spoken Dialogue System 1 Microphone Array 11-1K Microphone 30, 31-3K AD Converter (ADC)
4 DA converter (DAC)
51 System Voice Suppression Filter Unit 52 ICA Unit 61 User Voice Enhancement Filter Unit 62 ICA Unit 71 Dialogue Management Unit 72 Speech Synthesis Units 511 to 51K Adders 521 to 52K ICA Units 611 and 612 Adders B _{10 to} B _K0 Adaptive Filter V ₁₁ (F), V ₂₁ (f) to V _1K (f), V _2K (f) Adaptive filter

Claims (6)

A barge-in free voice interactive system capable of recognizing user voice in a situation where system voice is output from a speaker,
A first filter unit that performs a filtering process on the observation signal group by the microphone array and generates a first signal group after the filtering process;
The adaptive filter group included in the first filter unit is independent so that the sound source signal supplied to the speaker for outputting the system sound and each of the first signal group are independent from each other. A first ICA unit that is optimized based on a component analysis algorithm;
A second filter unit that performs a filtering process on the first signal group and generates a second signal group after the filtering process;
A second ICA unit that optimizes an adaptive filter group included in the second filter unit based on an independent component analysis algorithm so that the second signal group is independent of each other;
A dialogue management unit for performing dialogue management including recognition processing of the user voice based on a signal included in the second signal group and generation processing of a new system voice based on a recognition processing result;
Barge-in free spoken dialogue system.   The barge-in free spoken dialogue system according to claim 1, wherein the first and second ICA units optimize an adaptive filter group by independent component analysis in a frequency domain. A first filter step of performing a filtering process on the observation signal group by the microphone array to generate a first signal group after the filtering process;
The adaptive filter group used in the first filter step is independent so that the sound source signal supplied to the speaker for outputting system sound and the first signal group are independent from each other. A first ICA step to optimize based on a component analysis algorithm;
Performing a filtering process on the first signal group to generate a second signal group after the filtering process; and
The second filter so that the second signal groups are independent of each other. A second ICA step of optimizing the adaptive filter group used in the step based on an independent component analysis algorithm;
A dialogue management step for performing dialogue management including recognition processing of the user voice based on a signal included in the second signal group and generation processing of a new system voice based on a recognition processing result;
A voice interaction method comprising:   The method according to claim 3, wherein the optimization of the adaptive filter group in the first and second filter steps is performed by independent component analysis in the frequency domain. A program for causing a computer to execute information processing related to a barge-in-free voice interactive system capable of recognizing user voice under a situation where system voice is output from a speaker,
The information processing
A first filter step of performing a filtering process on the observation signal group by the microphone array to generate a first signal group after the filtering process;
Adaptive filter group used in the first filter step so that a sound source signal supplied to the speaker for outputting the system sound and each of the first signal group are independent from each other. A first ICA step that optimizes based on an independent component analysis algorithm;
Performing a filtering process on the first signal group to generate a second signal group after the filtering process; and
The second filter so that the second signal groups are independent of each other. A second ICA step of optimizing the adaptive filter group used in the step based on an independent component analysis algorithm;
A dialogue management step for performing dialogue management including recognition processing of the user voice based on a signal included in the second signal group and generation processing of a new system voice based on a recognition processing result;
Including the program.   The program according to claim 5, wherein the optimization of the adaptive filter group in the first and second filter steps is performed by independent component analysis in a frequency domain.

Images