JP2011203414A - Noise and reverberation suppressing device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically suppress both noise and reverberation by adapting for environmental change.SOLUTION: A noise and reverberation suppressing device is equipped with: a filter creation section for creating a separation filter matrix for each frequency by using an input signal which is converted to a frequency domain; a noise estimation section for calculating estimation noise; a direct voice/initial reflection voice estimation section which obtains each element of a reverse matrix of the separation filter matrix by converting it to a time domain, as an estimation space transfer characteristic, and which extracts a section corresponding to direct voice and initial reflection of voice from the estimation space transfer characteristic, creates the direct voice and an initial reflection filter matrix for each frequency by converting the extracted section to the frequency domain, and calculates estimation direct voice and initial reflection voice; a late stage reverberation creation section which calculates a filter coefficient of a late stage reverberation characteristic for each frequency from reverberation time of a given space and a power amount of the section extracted by the direct voice/initial reflection voice estimation section, and which calculates a pseudo late stage reverberation. Noise and late stage reverberation in a mixed observation signal is suppressed thereby.

Description

  The present invention relates to a noise and dereverberation apparatus and method.

  A hands-free voice command recognition system (hereinafter simply referred to as a voice recognition system) in which a user inputs voice commands to a robot has been developed. Here, as shown in FIG. 7, the sound picked up by the robot R in the real environment includes the direct sound of the user P's voice (hereinafter referred to as user voice), the late reverberation sound of the user voice, and noise. Including. In the figure, the direct sound of the user voice is indicated by a solid line, the late reverberation sound of the user voice is indicated by a broken line, and the noise is indicated by a chain line.

  In the voice recognition system, the robot recognizes the user's voice from the observation signal from the microphone. However, as a factor that deteriorates the voice recognition performance, (1) mixed noise other than the user voice, and (2) late reverberation component of the user voice. Impact, etc.

  Although the reverberation of the user voice is a cause of deterioration of the speech recognition performance, as shown in the prior art described later, for the initial reverberation component, by using the “reverberation model” in the acoustic model in the speech recognition system, It is known that the effect can be removed.

Conventionally, in the environment where noise and reverberation occur, a technique for suppressing either noise alone or reverberation alone has been developed.
For example, Non-Patent Documents 1 and 2 disclose techniques aimed at suppressing late reverberation components. FIG. 8 shows a functional configuration of the speech recognition system disclosed in Non-Patent Documents 1 and 2. In the figure, a line having a normal thickness means that a signal is transmitted, and a line thicker than that means that a filter is transmitted.

  As shown in FIG. 8, the late reverberation estimation unit 501 receives the observation signal from the microphone 1 and estimates the late reverberation component from the reverberation characteristics of the environment measured in advance. In addition, a correction coefficient is obtained from the reverberation characteristics of the environment measured in advance, and the gain correction unit 502 corrects the estimated amplitude of the late reverberation component using the given correction coefficient. The reverberation suppression processing unit 503 suppresses the late reverberation included in the observation signal from the microphone 1 using the late reverberation component whose amplitude is corrected. Thus, the late reverberation component is estimated from the observed signal, and the late reverberation component of the user voice is suppressed by subtracting the estimated late reverberation component.

  Further, for example, Non-Patent Document 3 discloses a technique aimed at noise suppression. FIG. 9 shows a functional configuration of the speech recognition system disclosed in Non-Patent Document 3. Also in FIG. 9, each thickness line means the same as in FIG.

  In FIG. 9, blind sound source separation (BSS) 601, speech / noise selection unit 602, and multi-channel noise estimation unit 603 receives the observation signal from the microphone array 2 composed of a plurality of microphone elements and separates the mixed sound. To estimate the noise component. The mask generation unit 604 and the noise suppression processing unit 605 create a mask from the estimated noise component and suppress noise contained in the observation signal using the created mask. Further, the direct speech enhancement unit 606 enhances the direct sound of the user speech included in the observation signal after noise suppression. As described above, the noise component mixed in is estimated by using the blind sound source separation (BSS) (or blind sound source extraction (BSE)) algorithm, and the user filter is performed by performing the Wine Filter process using the observation signal and the noise estimation amount as input. Is extracted to perform voice enhancement processing for hands-free operation.

JP 2003-066986 A JP 2009-509673 A JP 2003-334785 A JP 2003-305670 A JP 2004-098252 A

Randy Gomez, Jani Even, Hiroshi Saruwatari, Kiyohiro Shikano, "Robustness in Microphone-speaker Location under Reverberant Conditions for Speech Recognition," Proceedings of the Acoustical Society of Japan, pp. 159-160, March 2008. Randy Gomez, Jani Even, Hiroshi Saruwatari, Kiyohiro Shikano, "DISTANT-TALKING ROBUST SPEECH RECOGNITION USING LATE REFLECTION COMPONENTS OF ROOM IMPULSE RESPONSE," ICASS, pp. 4581-4584, 2008. Jani Even, Hiroshi Saruwatari, Kiyohiro Shikano, Tomoya Takatani, "Speech enhancement in presence of diffuse background noise using sparsity based blind signal extraction," Proceedings of the Acoustical Society of Japan, pp. 765-768, September 2009. Jani Even, Randy Gomez, Hiroshi Saruwatari, Kiyohiro Shikano, "Combining blind signal separation and spectral subtraction of late impulse response effect for dereverberation in noisy and highly reverberant environment," Proceedings of the Acoustical Society of Japan, pp. 839-842, 2008 September. Takuya Yoshioka, Tomohiro Nakatani, Hiroshi Okuno, “Efficient optimization method of MIMO dereverberation filter in weighted prediction error method”, Proc. 651-654, September 2009. Takuya Yoshioka, Tomohiro Nakatani, and Masato Miyoshi, "FAST SLGORITHM FOR CONDITIONAL SEPARATION AND DEREVERBERATION," EURASIP, pp. 1432-1436, 2009.

On the other hand, the inventor of the present application has created a technique shown in FIG. 10 in order to suppress both noise and reverberation.
In the technique shown in FIG. 10, first, a blind sound source separation (BSS) 701, a voice / noise selection unit 702, a multi-channel noise estimation unit 703, a mask generation unit 704, a noise suppression processing unit 705, and a direct speech enhancement unit 706 The speech is directly enhanced after suppressing noise in the same manner as in the technique shown in FIG. Then, the late reverberation estimation unit 707, the gain correction unit 708, and the reverberation suppression processing unit 709 suppress the late reverberation using the reverberation characteristics of the environment measured in advance, similarly to the technique shown in FIG. Thereby, both noise and reverberation can be suppressed.

  FIG. 11 is a table showing functions provided in the technologies shown in FIGS. As shown in the figure, the technique shown in FIG. 8 cannot suppress noise, the technique shown in FIG. 9 cannot suppress late reverberation, and the technique shown in FIG. Although both reverberations can be suppressed, it is necessary to collect operational environment data in advance in order to obtain a gain correction coefficient. When the environment changes, it is necessary to collect operational environment data again. Since the gain correction coefficient is used to suppress the late reverberation component, the technique shown in FIG. 9 does not require the gain correction coefficient.

  However, even in the technology capable of suppressing both noise and reverberation shown in FIG. 10, the reverberation characteristic is measured in advance in each space for the spatial transfer characteristic indicating the reverberation characteristic of a space such as a room, The user has to give the robot reverberation characteristics according to the space.

  The model used by the robot for speech recognition was created in an ideal environment without reverberation. If sound other than the direct sound of the user's sound is mixed due to reverberation, it will cause a mismatch with the model. End up.

  Such a mismatch can be avoided by creating a model after obtaining necessary data in a sound environment in advance. This is not realistic because of cost and time. For this reason, there is a strong demand for a technique that can automatically suppress both noise and reverberation in response to environmental changes without requiring collection of operational environment data in advance.

  There are other techniques for suppressing both noise and reverberation, as disclosed in Non-Patent Documents 4 to 6. However, in either technique, collection of operational environment data in advance is possible. The point of automatically creating the reverberation characteristics of the space is not disclosed.

  In addition, as other noise suppression techniques, there are techniques disclosed in Patent Documents 1 to 5 and the like. For example, the techniques disclosed in Patent Document 1 and Patent Document 2 only allow noise suppression. .

  SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a noise and dereverberation apparatus and method capable of automatically suppressing both noise and reverberation by solving the above-described problems and adapting to environmental changes. .

  The noise and dereverberation apparatus according to the first aspect of the present invention uses an input signal obtained by converting a mixed observation signal including voice and noise into a frequency domain at each frequency for separating the voice from the mixed observation signal. A noise estimation unit that calculates estimated noise using a filter creation unit that creates a separation filter matrix, an input signal, the separation filter matrix, and an inverse matrix of the separation filter matrix, and the separation filter matrix Each element of the inverse matrix is converted to the time domain to obtain an estimated spatial transfer characteristic, and a section corresponding to the direct voice and initial reflection of the voice is extracted from the estimated spatial transfer characteristic, and the extracted section is converted to the frequency domain. Transform to create a direct speech and initial reflection filter matrix at each frequency, the input signal, the separation filter matrix, and the direct speech and initial reflection filter matrix. The direct voice / initial reflected voice estimation unit for calculating the estimated direct voice and the initial reflected voice, the reverberation time of the given space, and the power amount of the section extracted by the direct voice / initial reflected voice estimation unit, A late period of calculating pseudo-late reverberation using the estimated direct speech and early reflected sound, the separation filter matrix, and the filter coefficient of the late reverberation characteristic, calculating a filter coefficient of the late reverberation characteristic at each frequency Reverberation generation unit, and suppresses noise and late reverberation in the mixed observation signal using the estimated noise calculated by the noise estimation unit and the pseudo late reverberation calculated by the late reverberation generation unit To do.

  As a result, even when the environment changes, the estimated direct speech and early reflection sound are calculated based on the observation signal in the new space, the late reverberation characteristics are automatically created, and the calculated estimated direct speech and initial reflection are calculated. Since the late reverberation can be calculated from the voice and the created late reverberation characteristics, both noise and reverberation can be automatically suppressed in accordance with the change in the environment.

  Further, a noise suppression mask generation unit that calculates a noise suppression mask using the input signal and the estimated noise calculated by the noise estimation unit, and a noise suppression mask in the mixed observation signal using the noise suppression mask A noise suppression processing unit that suppresses noise; and a gain correction unit that corrects the amplitude of the pseudo late reverberation calculated by the late reverberation generation unit using the noise suppression mask calculated by the noise suppression mask generation unit; Further, the late reverberation in the mixed observation signal may be suppressed using the pseudo late reverberation after the amplitude is corrected by the gain correction unit.

  Furthermore, a late reverberation sound suppression mask generation unit that calculates a late reverberation sound suppression mask using the input signal and the pseudo late reverberation after the amplitude is corrected by the gain correction unit, and the late reverberation sound A late reverberation suppression unit that suppresses reverberation in the mixed observation signal by using a suppression mask may be further included.

  In addition, the filter creation unit performs adaptive learning processing using the input signal and creates a separation filter matrix at each frequency for separating the speech from the mixed observation signal, and the blind sound source separation A voice / noise selector that performs replacement so that the first element of the generated separation filter matrix becomes the speech, and an inverse that calculates an inverse matrix of the separation filter matrix after replacement by the speech noise / selection unit. A matrix operation unit.

  Further, with respect to the signal after suppression of the noise in the mixed observation signal and the late reverberation using the estimated noise calculated by the noise estimation unit and the pseudo late reverberation calculated by the late reverberation generation unit A direct voice emphasis unit that directly emphasizes the voice may be further provided.

  The method for suppressing noise and reverberation according to the second aspect of the present invention includes a step of converting a mixed observation signal including speech and noise into a frequency domain to be an input signal, and using the input signal, the mixed observation signal. Generating a separation filter matrix at each frequency for separating the speech from the above, calculating an inverse matrix of the created separation filter matrix, the input signal, the created separation filter matrix, and the created Using the inverse matrix of the separation filter matrix to calculate the estimated noise, converting each element of the created inverse matrix of the separation filter matrix into the time domain, and obtaining the estimated spatial transfer characteristics; Cutting out the section corresponding to the direct voice and initial reflection of the voice from the estimated spatial transfer characteristics, and converting the cut out section into a frequency domain Estimating direct speech and initial reflection filter matrix using the step of creating a direct speech and initial reflection filter matrix at wave number, the input signal, the created separation filter matrix, and the created direct speech and initial reflection filter matrix. The step of calculating reflected speech, the step of calculating the filter coefficient of the late reverberation characteristic at each frequency from the reverberation time of the given space and the power amount of the extracted section, the calculated estimated direct speech and The step of calculating the pseudo late reverberation using the early reflection sound, the created separation filter matrix, and the calculated filter coefficient of the late reverberation characteristic, the calculated estimated noise, and the calculated pseudo late reverberation And suppressing noise and late reverberation in the mixed observation signal.

  According to the present invention, it is possible to provide a noise and dereverberation apparatus and method that can automatically suppress both noise and reverberation in response to environmental changes.

1 is a functional configuration diagram of a noise and dereverberation device according to a first exemplary embodiment; FIG. 3 is a functional configuration diagram of a filter creation unit according to the first embodiment. FIG. 6 is a diagram for explaining a cut-out process according to the first embodiment; It is a functional block diagram of the noise and the dereverberation apparatus concerning other embodiment. It is a functional block diagram of the noise and the dereverberation apparatus concerning other embodiment. It is a functional block diagram of the noise and the dereverberation apparatus concerning other embodiment. It is a figure which shows the sound collected in a real environment. It is a functional block diagram of the dereverberation technique relevant to this invention. It is a functional block diagram of the noise suppression technique relevant to this invention. It is a functional block diagram of the noise and the dereverberation technique relevant to this invention. It is a table | surface which shows the subject of each technique relevant to this invention. It is a figure which illustrates the matrix used in this invention. It is a figure which illustrates the matrix used in this invention.

Embodiment 1
Embodiments of the present invention will be described below with reference to the drawings. 1 and 2 are block diagrams showing a system configuration of a noise and dereverberation apparatus according to an embodiment of the present invention. In the figure, a line having a normal thickness means that a signal is transmitted, and a line thicker than that means that a filter is transmitted.

  The noise and dereverberation apparatus 100 according to the present embodiment includes a microphone array 2, a filter creation unit 10, a noise estimation unit 21, a noise suppression mask generation unit 22, a noise suppression processing unit 23, and a direct voice / initial stage. A reflected speech estimation unit 24, a late reverberation generation unit 25, a gain correction unit 26, a late reverberation suppression mask generation unit 27, a late reverberation suppression unit 28, and a direct speech enhancement unit 29 are provided. .

  The noise and dereverberation apparatus 100 has, as main hardware configurations, a CPU (Central Processing Unit) that performs control processing, arithmetic processing, and the like, and a ROM (control program, arithmetic program, and the like that are executed by the CPU. The microcomputer includes a read only memory (RAM) and a random access memory (RAM) that temporarily stores processing data and the like. In addition, the filter creation unit 10, the noise estimation unit 21, the noise suppression mask generation unit 22, the noise suppression processing unit 23, the direct speech / initial reflection speech estimation unit 24, the late reverberation generation unit 25, and the gain correction unit 26, the late reverberation suppression mask generation unit 27, the late reverberation suppression processing unit 28, and the direct speech enhancement unit 29, for example, are realized by a program stored in the ROM and executed by the CPU. May be.

  The noise and dereverberation apparatus 100 performs processing related to noise suppression in a portion including the filter creation unit 10, the noise estimation unit 21, the noise suppression mask generation unit 22, and the noise suppression processing unit 23. In addition, the filter creation unit 10, the direct speech / early reflection speech estimation unit 24, the late reverberation generation unit 25, the gain correction unit 26, the late reverberation suppression mask generation unit 27, and the late reverberation suppression unit 28 In the portion including, processing related to the suppression of late reverberation is performed. Further, in a portion including the direct speech enhancement unit 29, processing related to direct sound enhancement is performed. Furthermore, a speech recognition process is performed based on the signal directly output from the speech enhancement 29.

  Further, the filter creation unit 10 performs batch processing, accumulates a predetermined amount of observation signals (voice data), and then performs filter creation processing using the accumulated voice data. The noise estimation unit 21 and the like excluding the filter creation unit 10 perform real-time processing using the filter created by the filter creation unit 10.

  An outline of processing by the noise and dereverberation apparatus 10 will be described. First, the observation signal observed by the microphone array 2 includes user voice and noise. The user voice includes a direct voice, an early reflection sound, and a late reverberation sound. The filter creation unit 10 separates user speech and noise from the observation signal, and creates a separation filter at this time. Using the estimated noise separated by the separation filter, the noise of the observation signal is suppressed. On the other hand, the direct sound / initial reflected sound estimation unit 24 estimates the direct sound and the initial reflected sound of the user sound from the observation signal using this separation filter. Then, the late reverberation generation unit 25 creates an artificial late reverberation characteristic, and creates pseudo late reverberation from the estimated direct sound and initial reflected sound of the user voice and the created late reverberation characteristic. Then, by using the created pseudo late reverberation, the late reverberation of the user voice of the observation signal is suppressed.

  The microphone array 2 is composed of a plurality of microphone elements, and observes mixed sound in which user voice and noise are mixed. The plurality of microphones are provided on the head of the robot R, for example, and are arranged in the horizontal direction. The observation signal of each microphone corresponds to each channel i.

  An observation signal from each microphone is converted into digital data (hereinafter referred to as audio data) by an AD converter (not shown). Furthermore, audio data of each microphone is accumulated for a predetermined time and divided into frames. Discrete Fourier transform processing is performed on the audio data in units of frames, and converted to an input signal vector X (f, t) in the time / frequency domain.

  As shown in FIG. 2, the filter creation unit 10 includes a blind sound source separation (BSS) 11, a voice / noise selection unit 12, and an inverse matrix calculation unit 13. The filter creation unit 10 creates a separation filter matrix at each frequency for separating user speech from the input signal vector X (f, t) using the input signal vector X (f, t). Hereinafter, the blind sound source separation (BSS) 11 and the like will be described with reference to FIG.

The blind sound source separation (BSS) 11 performs an adaptive learning process using the input signal vector X (f, t) and creates a separation filter matrix W (f) at each frequency. The blind sound source separation (BSS) 11 outputs an output signal vector Y (f, t) = W (f) X (f, t) using the created separation filter matrix W (f).
Here, the blind sound source separation (BSS) 11 performs a process capable of adaptive learning without using prior information by using conventionally proposed independent component analysis and principal component analysis.

The voice / noise selection unit 12 uses the separation filter matrix W (f, t) so that the first element (Y ⁽¹⁾ (f, t)) of the output signal vector Y (f, t) becomes voice data. Swap elements.
This is because the output signal of the blind sound source separation (BSS) 11 is clustered as user speech or noise, which is a clustering result for each frequency bin, but the user speech and noise are interchanged. This is because there is a possibility.

  Note that the processing by the voice / noise selection unit 12 is also called a permutation solution. For example, a conventionally proposed solution (a joint density probability distribution between output signal vectors Y (f, t) is obtained and this combination is performed. A method of distributing user speech and noise based on the shape of the probability density distribution may be used. Further, since there are many solutions used here, other solutions may be used. For example, (1) a solution using the kurtosis of the peripheral probability density distribution, (2) a solution using the envelope of the separation signal of the sound source signal, (3) a solution using the continuity of the phase information, (4) space A solution using a spectrum may be used.

The inverse matrix calculation unit 13 calculates an inverse matrix H (f) = W ⁻¹ (f) for the separation filter matrix W (f) used for calculating the output signal vector Y (f, t). The obtained inverse matrix H (f) is an estimated value of the spatial transfer characteristic matrix, and means a reverberation characteristic of a space such as a room.

  The relationship in which an observation signal in which user speech and noise are mixed corresponds to the mapping of the user speech and noise from each sound source to the observation signal. On the other hand, the observation signal mixed with noise is the user speech and the noise. Separation into noise corresponds to mapping from the observation signal to each sound source. Since the separation matrix filter matrix W (f) created in the blind sound source separation (BSS) 11 means a mapping from the observed signal to each sound source, the inverse matrix H (f) of the separation matrix filter matrix W (f) is obtained. Means obtaining a mapping of the user voice and noise from each sound source to the observation signal.

Returning to FIG. 1, the description will be continued.
The noise estimation unit 21 uses the following calculation formula from the input signal vector X (f, t), the separation filter matrix W (f, t), and the estimated value H (f) of the spatial transfer characteristic matrix: A noise signal vector X _N (f, t) included in the observation signal (input signal vector X (f, t)) is calculated. That is, the noise estimation unit 21 estimates a noise component included in the observation signal (input signal).
X _N (f, t) = H (f) D _N (f) W (f) X (f, t)
Here, D _N (f) indicates a diagonal matrix whose element is i row i column (1 <i). In the general diagonal matrix, all diagonal elements are 1, but D _N (f) used here is set to 0 in the 1st row and 1st column so as not to obtain a user voice. The remaining diagonal elements are all 1. Therefore, for example, in the case of 3 rows and 3 columns, the matrix is as shown in FIG. 12A, and in the case of 4 rows and 4 columns, the matrix is as shown in FIG.

The direct speech / early reflection speech estimation unit 24 performs inverse fast Fourier transform processing on each element of the matrix H (f) obtained by the inverse matrix calculation unit 13 to obtain an estimated value of the spatial transfer characteristic after conversion into the time domain. . Then, from this estimated value, a section corresponding to the direct sound and initial reflected sound to each microphone is cut out by a window function, and the cut-out section is subjected to Fourier transform processing to be converted into the frequency domain, whereby each frequency is obtained. A direct speech / early reflection speech estimation filter matrix is created. The matrix after Fourier transform processing obtained here is denoted by H _E (f). Note that the window length h _E used in the cut-out is given as a parameter. For example, FIG. The value of _{h E} obtained in 3 (70 [taps]) may be used. Cutout process using sound and initial reflected sound estimation unit 24 directly, for example, in the case shown the waveform after converting the estimate of the spatial transfer characteristic in the time domain in the left diagram of FIG. 3, which in the h _E length of It is to cut out by the window as shown in the right figure of FIG.

Further, the direct speech / initial reflection speech estimation unit 24 uses the following calculation formula to calculate the direct speech / initial reflection speech estimation filter matrix H _E (f) after the Fourier transform and the separation matrix filter matrix W (f): From the observation signal (input signal vector X (f, t)), the direct sound / initial reflected sound signal vector X _E (f, t) of the user voice included in the observation signal is calculated. That is, the direct sound / initial reflected sound estimation unit 24 estimates the direct sound / initial reflected sound component of the user sound included in the observation signal.
X _E (f, t) = H _E (f) D _S (f) W (f) X (f, t)
However, D _S (f) represents a matrix in which only the element in i row and i column (1 <i) is 1, and all other elements are 0. DS _S (f) used here is set so that only the element in the first row and first column is set to 1 and all the remaining elements are set to 0 in order not to obtain noise. Therefore, for example, in the case of 3 rows and 3 columns, the matrix is as shown in FIG. 13A, and in the case of 4 rows and 4 columns, the matrix is as shown in FIG. 13B.

The late reverberation generation unit 25 uses the following calculation formula to calculate the late reverberation estimation signal vector from the direct sound / early reflected sound signal vector X _E (f, t) of the user voice and H _E (f) described later. X _L (f, t) is calculated. That is, the late reverberation generation unit 25 generates a pseudo late reverberation component. X _L ⁽ⁱ⁾ (f, t) indicates the i-th element of the vector X _L (f, t). The same H _L (f) is used for each microphone.
X _L ⁽ⁱ⁾ (f, t) = X _E ⁽ⁱ⁾ (f, t) H _L (f)
Here, H _L (f) represents the filter coefficient of the late reverberation characteristic, and the reverberation time (T60) of the room given by the user and the amount of power cut by the direct voice / early reflected voice estimation unit 25 (ie, cut out) And the power amount of the section). More specifically, the ratio between the power amount of the direct sound and the early reflection sound and the power amount of the late reverberation is not extracted from the power amount of the section cut out in the direct sound / early reflection sound estimation unit 25 and the others. Since this corresponds to the ratio to the power amount of the section, the filter coefficient of the late reverberation characteristic can be calculated by correcting the amplitude of the late reverberation obtained from the reverberation time based on this ratio. In addition, since calculation of _HL (f) should just be performed using the conventionally well-known calculation formula, the detailed description is abbreviate | omitted here.

Noise suppression mask generating unit 22 uses the following equation, the input signal vector X (f, t) from the noise signal vector _X N (f, t) and the noise suppression mask vector _M N (f, t) Is calculated. Note that M _N ⁽ⁱ⁾ (f, t) represents the i-th element of the vector M _N (f, t). The coefficient α ₁ is a parameter for adjusting the degree of noise suppression, and an appropriate value is given by the user.
M _N ⁽ⁱ⁾ (f, t) = sqrt (| X ⁽ⁱ⁾ (f, t) | ² / (| X ⁽ⁱ⁾ (f, t) | ² + α ₁ | X _N ⁽ⁱ⁾ (f, t) | ² ))

The noise suppression processing unit 23 uses the following calculation formula to calculate an intermediate output after noise suppression from the observation signal (input signal vector X (f, t)) and the noise suppression mask vector M _N (f, t). A signal vector V (f, t) is calculated. That is, the noise suppression processing unit 23 suppresses environmental noise included in the observation signal. V ⁽ⁱ⁾ (f, t) indicates the i-th element of the vector V (f, t).
V ⁽ⁱ⁾ (f, t) = M _N ⁽ⁱ⁾ (f, t) X ⁽ⁱ⁾ (f, t)

The gain correction unit 26 uses the following calculation formula to calculate the intermediate output signal vector V _L ⁽ⁱ ) from the late reverberation estimation signal vector X _L (f, t) and the noise suppression mask vector M _N (f, t). ⁾ (F, t) is calculated. That is, the gain correction unit 26 corrects the amplitude of the late reverberation estimation component using the noise suppression mask. The noise suppression mask vector M _N (f, t) obtained by the noise suppression mask generation unit 22 is used in common by the noise suppression processing unit 23 and the gain correction unit 26, so that the noise suppression processing unit 23 In accordance with the suppression, the gain correction unit 26 can correct the amplitude of the late reverberation estimation component. V _L ⁽ⁱ⁾ (f, t) represents the i-th element of the vector V _L (f, t).
V _L ⁽ⁱ⁾ (f, t) = M _N ⁽ⁱ⁾ (f, t) X _L ⁽ⁱ⁾ (f, t)

The late reverberation suppression mask generation unit 27 uses the following calculation formula to suppress dereverberation from the intermediate output signal vector V (f, t) after noise suppression and the intermediate output signal vector V _L (f, t). A mask vector M _L (f, t) is calculated. M _L ⁽ⁱ⁾ (f, t) represents the i-th element of the vector M _L (f, t). The coefficient α ₂ is a parameter that adjusts the degree of late reverberation suppression, and is given an appropriate value by the user.
M _L ⁽ⁱ⁾ (f, t) = sqrt (| V ⁽ⁱ⁾ (f, t) | ² / (| V ⁽ⁱ⁾ (f, t) | ² + α ₂ | V _L ⁽ⁱ⁾ (f, t) | ² ))

Late dereverberation processing unit 28, using the following equation, the intermediate output signal vector V (f, t) after the noise suppression and, dereverberation mask vector M _{L (f,} t) and a, late reverberation suppression The subsequent output signal vector Y (f, t) is calculated. That is, the late reverberation suppression processing unit 28 uses the reverberation suppression mask to suppress the late reverberation included in the intermediate output after noise suppression. Y ⁽ⁱ⁾ (f, t) represents the i-th element of the vector Y (f, t).
^{_{Y (i) (f, t}} ) = M L (i) (f, t) V (i) (f, t)

The direct speech enhancement unit 29 calculates the output signal vector O (f, t) after direct speech enhancement from the output signal vector Y (f, t) using the following calculation formula. That is, the direct voice emphasis unit 29 directs the beam toward the user direction θ and directly enhances the voice. The user orientation θ is obtained when the speech / noise selector 12 estimates a noise component.
O (f, t) = Σ _i Y ⁽ⁱ⁾ (f, t) H _DS ⁽ⁱ⁾ (f, t)
H _DS ⁽ⁱ⁾ (f, t) is a delay and sum filter coefficient, and Σ _i indicates that the averaging process is performed for all channels i (all microphone elements). Delay and Sum is a method of correcting the arrival time difference between microphone elements using the user orientation θ estimated from the output signal vector Y (f, t) and forming a beam in the user orientation.

  As described above, according to the noise and dereverberation apparatus 100 according to the present embodiment, the estimated direct speech and the initial reflected speech are calculated based on the new spatial observation signal, and the late reverberation characteristics are converted into the spatial reverberation characteristics. It is possible to calculate late reverberation from the estimated direct sound and early reflection sound that are automatically created and calculated, and the late reverberation characteristics that are created, so even if the environment changes, it adapts to environmental changes Thus, both noise and reverberation can be automatically suppressed.

Other embodiments.
In the above-described embodiment, the example in which the filter creation unit 10 performs blind sound source separation (BSS) has been described, but the present invention is not limited to this. For example, as shown in FIG. 4, blind signal extraction (BSE) may be applied instead of blind sound source separation (BSS). That is, as illustrated in FIG. 4, the noise and dereverberation apparatus 200 includes a microphone array 2, a blind signal extraction (BSE) 201, a projection vector estimation unit 202, a noise estimation unit 203, and a noise suppression mask generation unit 204. A noise suppression processing unit 205, a direct speech / early reflection speech estimation unit 206, a late reverberation generation unit 207, a gain correction unit 208, a late reverberation suppression mask generation unit 209, and a late reverberation suppression unit 210. And a direct voice emphasis unit 211.

  In the blind signal extraction (BSE) 201 shown in FIG. 4, user speech is extracted from the observation signal and output, and the projection vector estimation unit 202 outputs an estimated value of the spatial transfer characteristic matrix based on this. In addition, the user orientation θ estimated by the noise estimation unit 203 is output to the speech enhancement unit 211. Since the processing performed by the blind signal extraction (BSE) 201 and the projection vector estimation unit 202 is known, detailed description thereof is omitted here.

  In the above-described embodiment, an example in which a noise suppression mask and a late dereverberation suppression mask are generated by separate mask generation units has been described, but the present invention is not limited to this. For example, as shown in FIG. 5, the noise suppression mask and the late dereverberation suppression mask may be generated by one mask generation unit, and the noise suppression process and the late dereverberation suppression process may be performed by one suppression processing unit. That is, as shown in FIG. 5, the noise and dereverberation apparatus 300 includes a microphone array 2, a blind sound source separation (BSS) 301, a speech / noise selection unit 302, an inverse matrix calculation unit 303, and a noise estimation unit 304. A direct speech / early reflection speech estimation unit 305, a late reverberation generation unit 306, a noise / late reverberation suppression mask generation unit 307, a noise / late reverberation suppression unit 308, a direct speech enhancement unit 309, It is good also as a structure provided with.

  Further, for example, as shown in FIG. 6, blind signal extraction (BSE) is applied instead of blind sound source separation (BSS), and a noise suppression mask and a late reverberation suppression mask are generated by one mask generation unit. The noise suppression process and the late dereverberation suppression process may be performed by one suppression processing unit. That is, as shown in FIG. 6, the noise and dereverberation apparatus 400 includes a microphone array 2, a blind signal extraction (BSE) 401, a projection vector estimation unit 402, a noise estimation unit 403, a direct voice and an early reflection voice. The estimation unit 404, the late reverberation generation unit 405, the noise / late reverberation suppression mask generation unit 406, the noise / late reverberation suppression processing unit 407, and the direct speech enhancement unit 408 may be provided.

Here, when compared with the technique related to the present invention shown in FIG. 10 and the like, the points different from the present invention and advantageous effects will be further described.
(1) Late Reverberation Characteristics In the technique shown in FIG. 10, it is necessary to give the reverberation characteristics of the space in advance.
On the other hand, in the present invention, the reverberation characteristic can be automatically created from the reverberation time of a room or the like.
(2) Reverberation Sound Estimation Method In the technique shown in FIG. 10, reverberation sound is estimated for a signal including a direct user sound, an early reflection sound, and a late reverberation sound. .
On the other hand, in the present invention, the reverberation sound is estimated for a signal including the direct sound of the user voice and the initial reflected sound. This is achieved by further providing a direct sound and early reflection sound estimation function.
(3) Gain Correction Before Late Reverberation Suppression In the technique shown in FIG. 10, since the gain correction coefficient is given in advance and the correction coefficient cannot be estimated automatically, the characteristics of the room are measured and corrected in advance. It is necessary to find the coefficient.
On the other hand, in the present invention, the generated late reverberation is automatically corrected. In the present invention, the power amount of the transfer characteristic cut when estimating the direct sound and the early reflection sound (that is, the power amount of the section not cut out) and the power amount of the late reverberation characteristic to be created are the same. Correct to the amount.
(4) Reducing processing distortion by direct speech enhancement processing Both the technique shown in FIG. 10 and the present invention employ the Delay and Sum (DS) processing in the direct speech enhancement processing. The DS process includes an averaging process, which has a side effect that the suppression process distortion generated in each channel is reduced.
In the technique shown in FIG. 10, since the dereverberation process is performed after the direct speech enhancement process, the distortion of the dereverberation process of each channel is not alleviated.
On the other hand, in the present invention, since the speech enhancement process is directly performed after suppressing both noise and reverberation, distortion of the dereverberation process of each channel can be reduced.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.

100, 200, 300, 400 Noise and dereverberation device,
1 microphone, 2 microphone array, 10 filter creation unit,
11 Blind sound source separation (BSS), 12 Voice / noise selection unit,
13 inverse matrix calculation unit, 21 noise estimation unit, 22 noise suppression mask generation unit,
23 noise suppression processing unit, 24 direct speech / early reflection speech estimation unit,
25 late reverberation generation unit, 26 gain correction unit, 27 late reverberation suppression mask generation unit,
28 Late reverberation suppression processing section 29 Direct speech enhancement section

201 Blind signal extraction (BSE), 202 Projection vector estimation unit,
203 noise estimation unit, 204 noise suppression mask generation unit,
205 Noise suppression processing unit, 206 Direct speech / early reflection speech estimation unit,
207 late reverberation generation unit, 208 gain correction unit,
209 Late reverberation suppression mask generation unit, 210 Late reverberation suppression processing unit,
211 Direct speech enhancement unit,

301 blind sound source separation (BSS), 302 voice / noise selection unit,
303 Inverse matrix calculation unit, 304 Noise estimation unit,
305 direct speech / early reflection speech estimation unit, 306 late reverberation generation unit,
307 noise / late reverberation suppression mask generation unit, 308 noise / late reverberation suppression unit,
309 Direct speech enhancement unit,

401 Blind signal extraction (BSE), 402 Projection vector estimation unit,
403 noise estimation unit, 404 direct speech / early reflection speech estimation unit,
405 Late reverberation generator, 406 Noise / late reverberation suppression mask generator,
407 noise / late reverberation suppression processing unit, 408 direct speech enhancement unit,

501 late reverberation estimation unit, 502 gain correction unit, 503 reverberation suppression processing unit,

601 Blind sound source separation (BSS), 602 voice / noise selection unit,
603 multi-channel noise estimation unit, 604 mask generation unit,
605 noise suppression processing unit, 606 direct speech enhancement unit,

701 Blind sound source separation (BSS), 702 Voice / noise selection unit,
703 multi-channel noise estimation unit, 704 mask generation unit,
705 noise suppression processing unit, 706 direct speech enhancement unit, 707 late reverberation estimation unit,
708 gain correction unit, 709 dereverberation processing unit,

P user, R robot

Claims (6)

A filter creation unit that creates a separation filter matrix at each frequency for separating the speech from the mixed observation signal, using an input signal in which the mixed observation signal including speech and noise is converted into a frequency domain;
A noise estimation unit that calculates an estimated noise using the input signal, the separation filter matrix, and an inverse matrix of the separation filter matrix;
Each element of the inverse matrix of the separation filter matrix is converted into the time domain and acquired as an estimated spatial transfer characteristic, and a section corresponding to the direct voice and initial reflection of the voice is extracted from the estimated spatial transfer characteristic, and the extracted section Is converted into the frequency domain to create a direct speech and initial reflection filter matrix at each frequency, and the estimated direct speech is estimated using the input signal, the separation filter matrix, and the direct speech and initial reflection filter matrix. And a direct speech / initial reflection speech estimator for calculating early reflection speech,
From the reverberation time of the given space and the power amount of the section extracted by the direct speech / initial reflected speech estimation unit, the filter coefficient of the late reverberation characteristics at each frequency is calculated, and the estimated direct speech and early reflected speech are calculated. A late reverberation generation unit that calculates pseudo late reverberation using the separation filter matrix and the filter coefficient of the late reverberation characteristic,
Noise and late reverberation in the mixed observation signal are suppressed using the estimated noise calculated by the noise estimation unit and the pseudo late reverberation calculated by the late reverberation generation unit. Reverberation suppressor. A noise suppression mask generation unit that calculates a noise suppression mask using the input signal and the estimated noise calculated by the noise estimation unit;
Using the noise suppression mask, a noise suppression processing unit that suppresses noise in the mixed observation signal;
A gain correction unit that corrects the amplitude of the pseudo late reverberation calculated by the late reverberation generation unit using the noise suppression mask calculated by the noise suppression mask generation unit;
2. The noise and dereverberation apparatus according to claim 1, wherein late reverberation in the mixed observation signal is suppressed using pseudo late reverberation after amplitude is corrected by the gain correction unit. A late reverberation suppression mask generation unit that calculates a late reverberation suppression mask using the input signal and the pseudo late reverberation after the amplitude is corrected by the gain correction unit;
The noise and dereverberation apparatus according to claim 2, further comprising: a later-stage reverberation suppression unit that suppresses reverberation in the mixed observation signal using the latter-stage reverberation suppression mask. The filter creation unit
Blind sound source separation for performing adaptive learning processing using the input signal and creating a separation filter matrix at each frequency for separating the speech from the mixed observation signal;
A voice / noise selector that performs replacement so that the first element of the separation filter matrix created by the blind sound source separation is the voice;
The noise and reverberation according to any one of claims 1 to 3, further comprising: an inverse matrix calculation unit that calculates an inverse matrix of the separation filter matrix after the replacement by the voice noise / selection unit. Suppressor. Using the estimated noise calculated by the noise estimation unit and the pseudo late reverberation calculated by the late reverberation generation unit, the noise in the mixed observation signal and the signal after the late reverberation are directly suppressed The noise and dereverberation apparatus according to any one of claims 1 to 4, further comprising a direct speech enhancement unit that enhances speech. Converting a mixed observation signal including speech and noise into a frequency domain to be an input signal;
Using the input signal to create a separation filter matrix at each frequency that separates the speech from the mixed observation signal;
Calculating an inverse matrix of the created separation filter matrix;
Calculating estimated noise using the input signal, the created separation filter matrix, and an inverse matrix of the created separation filter matrix;
Transforming each element of the inverse matrix of the created separation filter matrix into the time domain to obtain an estimated spatial transfer characteristic;
Cutting out a section corresponding to the direct voice and initial reflection of the voice from the acquired estimated spatial transfer characteristic;
Transforming the extracted section into a frequency domain to create a direct speech and initial reflection filter matrix at each frequency; and
Calculating estimated direct speech and initial reflected speech using the input signal, the created separation filter matrix, and the created direct speech and initial reflection filter matrix;
From the reverberation time of the given space and the amount of power of the extracted section, calculating a filter coefficient of the late reverberation characteristics at each frequency;
Calculating the pseudo late reverberation using the calculated estimated direct sound and early reflected sound, the created separation filter matrix, and the calculated filter coefficient of the late reverberation characteristic;
Using the calculated estimated noise and the calculated pseudo late reverberation to suppress the noise and the late reverberation in the mixed observation signal, and a method for suppressing noise and reverberation.

Images