JP3484112B2 - Noise component suppression processing apparatus and noise component suppression processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform noise suppression processing which has a small calculation volume and eliminates an unexpected noise. SOLUTION: A device is provided with a means 12 which performs frequency (f) analysis of each of reception position-classified sound signals obtained by detecting sounds in plural positions to obtain channel (ch)-classified (f) components, a first beam form processing means (B1) 13 which suppresses a noise (N) in the direction of a speaker to obtain an objective sound (0a) component by F processing based on a filter (F) coefficient which reduces sensitivity in directions other than the desired direction with respect to the (f) component of each channel, a second beam form processing means (B2) 16 which suppresses the voice of the speaker to obtain an N component by F processing which reduces sensitivity in directions other than the desired direction with respect to the (f) component of each channel obtained by the means 12, estimating means 17 and 18 which estimate the direction of N by the F coefficient of B1 and estimates the direction of the voice 0a by that of B2, means 14 and 15 which correct the coming direction of 0a of an input object 0j and that of N of 0j in B1 and B2 in accordance with estimated directions respectively, a means 30 for spectrum subtraction(SS) processing based on outputs of B1 and B2, a means 40 which obtains a directivity index D according with the time difference and the amplitude difference of coming sounds from the output of the means 12, and a means 50 for SS processing control based on D and the direction of 0a.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a noise component suppressing apparatus and a noise component suppressing method for suppressing noise by using a plurality of microphones and extracting a target voice.

[0002]

2. Description of the Related Art Since there are various noise sources in the environment, it is difficult to avoid noise sneaking in from the surroundings even when capturing a voice signal with a microphone. However, when a voice signal mixed with noise is reproduced, the target voice becomes difficult to hear, and therefore noise component reduction processing is necessary.

By the way, as a technique for reducing noise mixed in voice, there is a technique known in the prior art for suppressing noise by using a plurality of microphones. Further, this microphone processing technology has been focused on technical development by many researchers for the purpose of inputting voice from a voice recognition device or a video conference device. Above all, regarding a microphone array using an adaptive beamformer processing technology that can obtain a large effect with a small number of microphones, refer to Reference 1 (Electronic Information and Communication Society: Audio Systems and Digital Processing) or Reference 2 (Heykin; Adaptive Filt).
er Theory (Plentice Hall), various methods such as a generalized side rope canceller (GSC), a frost type beamformer, and a reference signal method are known.

The adaptive beamformer process is generally a process of suppressing noise by a filter having a blind spot formed in the direction of arrival of interference noise.

However, in this adaptive beamformer processing technique, if the actual arrival direction of the target signal is different from the assumed arrival direction, the target signal is regarded as noise and is removed, so that the performance is reduced. It has a problem of deterioration.

Therefore, in order to improve this, for example, Document 3
(Hojuzan et al .: "Robust Generalized Sidelobe Canceller Using Leak Adaptive Filter for Blocking Matrix", IEICE Transactions A Vol. J79-AN
o. 9 pp1516-1524 (1996.9)), a technique has been developed to allow a deviation between an assumed arrival direction and an actual arrival direction. In this case, however, a target signal is removed. Even if is reduced, the target signal may be distorted due to the deviation between the actual arrival direction and the assumed arrival direction.

[0007] In contrast, for example, Japanese Application flat 9-979
In JP-A-4, by using a plurality of beam formers, the speaker direction is sequentially detected and the input direction of the beam former is corrected in that direction, thereby tracking the speaker direction and reducing the distortion of the target signal. Methods are also disclosed.

However, the method disclosed in Japanese Unexamined Patent Publication Application flat 9-9794 is, because a adaptive filtering in the time domain, when estimating the speaker direction from the filter coefficients, the filter coefficients of the time domain Since the conversion to the frequency domain is necessary, the amount of calculation becomes large.

[0009]

As a technique for suppressing voice noise, a plurality of microphones are used, and the microphones are used to capture the voice of the speaker and to provide a filter that forms a blind spot in the arrival direction of the interfering noise. There is an adaptive beamformer processing technique that suppresses a noise component by passing the signal.

In this adaptive beamformer processing technique, when the actual arrival direction of the target signal, that is, the direction of the speaker is different from the assumed arrival direction, the target signal is regarded as noise and removed. There is a problem that the voice collection performance deteriorates.

Therefore, in order to improve this, a technique has been developed which allows a deviation between the assumed arrival direction and the actual arrival direction, but in this case, even if the removal of the target signal is reduced, There is a concern that the target signal may be distorted due to the difference between the arrival direction of the received signal and the assumed arrival direction, leaving a problem of the quality of the obtained speech.

Further, a method of tracking the direction of the speaker and reducing the distortion of the target signal by sequentially detecting the direction of the speaker using a plurality of beam formers and correcting the input direction of the beam former in the direction. Is also proposed. However, since this method performs adaptive filtering in the time domain, when estimating the speaker direction from the filter coefficient, it is necessary to convert the filter coefficient in the time domain into the frequency domain, resulting in a large amount of calculation. There was a problem.

Therefore, all of the conventional techniques have merits and demerits, and there is a strong demand for the development of a beam former processing technique capable of collecting a target signal with high quality and requiring a short processing time. In addition, for example, when considering a voice recognition device installed in a vehicle, on the street, or at a station, sudden noise encountered while the vehicle is traveling, noise from an oncoming vehicle moving at high speed, or noise of a passing vehicle. There are sounds, and in a video conference, etc., a sudden ringing sound of the telephone installed in the room, a sudden noise such as the sound of the door opening and closing of people entering and leaving the room, etc. are also considered, so the duration such as these is very short Regarding noise, we would like to have sufficient noise suppression performance.

Therefore, the object of the present invention is to
By using a beamformer that operates in the frequency domain,
The amount of calculation can be greatly reduced, and noise suppression for noise with a very short duration and noise suppression for moving sound sources can be expected, and noise components can be suppressed even for sudden noise. A processing device and a noise component suppression processing method are provided.

[0015]

In order to achieve the above object, the present invention is configured as follows.

[1] First, two different voices of the speaker are used.
A voice input means for receiving sound at a position of more than one position and outputting as a sound signal respectively, and a frequency analysis means for performing frequency analysis for each channel of the sound signal corresponding to the sound receiving position and outputting a frequency component for each channel. And a first beamformer processing means for suppressing incoming noise other than the target voice by adaptive filter processing using frequency components of a plurality of channels output from the frequency analysis means, and outputting a signal of the target voice component. Second beamformer processing means for outputting a noise component signal by performing target voice suppression processing by adaptive filter processing using frequency components of a plurality of channels output from the frequency analysis means, and the first beam Noise direction estimating means for estimating a noise direction from the filter coefficient calculated by the former processing means, and the second beam former processing The target sound direction estimating means for estimating the target sound direction from the filter coefficient calculated in the stage, and the first input direction which is the arrival direction of the target sound to be input in the first beamformer processing means, First input direction correcting means for sequentially correcting based on the target sound direction estimated by the sound direction estimating means, and second input direction which is the arrival direction of noise to be input in the second beamformer processing means. Second input direction correcting means for sequentially correcting the above based on the noise direction estimated by the noise direction estimating means, the output of the first beamformer processing means and the second
Spectrum subtraction means for performing non-linear noise suppression processing based on the output of the beam former processing means, and a direction based on the time difference and the amplitude difference of the incoming sound from the frequency components output from the frequency analysis means. Directionality detecting means for calculating a soundness index, and spectrum subtraction control means for controlling the spectrum subtraction processing of the spectrum subtraction means based on the directionality index and the target sound direction output from the target sound direction estimating means. It comprises and is comprised.

[2] Secondly, two different voices of the speaker are used.
A voice input means for receiving sound at a position of more than one position and outputting as a sound signal respectively, and a frequency analysis means for performing frequency analysis for each channel of the sound signal corresponding to the sound receiving position and outputting a frequency component for each channel. And the frequency component of the plurality of channels obtained by the frequency analysis means is subjected to adaptive filter processing using a filter coefficient calculated so that sensitivity outside the desired direction becomes low The first beamformer processing means for performing the incoming noise suppression processing for suppressing speech other than the above to obtain the target speech component, and the frequency components of the plurality of channels obtained by the frequency analysis means have low sensitivity outside the desired direction. The voice from the speaker direction is suppressed by performing adaptive filter processing using the filter coefficient calculated as follows. Second beamformer processing means for obtaining a noise component, and adaptive filter processing using the filter coefficient calculated so that the sensitivity outside the desired direction becomes low for the frequency components of the plurality of channels obtained by the frequency analysis means. By estimating the noise direction from the second beamformer processing means for suppressing the voice from the speaker direction and obtaining the second noise component, and the filter coefficient calculated by the first beamformer processing means. Noise direction estimating means, first target sound direction estimating means for estimating a first target sound direction from the filter coefficient calculated by the second beamformer processing means, and third adaptive beamformer processing means. The second target sound direction estimating means for estimating the second target sound direction from the filter coefficient calculated by The first input direction, which is the arrival direction of the target sound of interest, is estimated by the first target sound direction estimated by the first target sound direction estimation means and by the second target sound direction estimation means. When a first input direction correcting means for sequentially correcting based on either or both of the second target sound directions and a noise direction estimated by the noise direction correcting means are within a predetermined first range, A second input direction correction means for sequentially correcting the second input direction, which is the arrival direction of the noise to be input in the second beamformer processing means, based on the noise direction, and the noise direction correction means for estimation. When the generated noise direction is within the predetermined second range, the third input direction, which is the arrival direction of noise to be input in the third beamformer processing means, is sequentially corrected based on the noise direction. Third input direction correction Means and one of the first and second output noises based on whether the noise direction estimated by the noise direction estimating means comes from a predetermined first range or a predetermined second range. One of them is determined to be a true noise output and either one of the noises is output, and at the same time, which of the estimation results of the first voice direction estimation means and the second voice direction estimation means is valid is determined. Effective noise determining means for outputting one of the speech direction estimation results to the first input direction correcting means, and nonlinear noise based on the outputs of the first beamformer processing means and the second beamformer processing means. Spectral subtraction means for performing spectral subtraction processing, which is suppression processing, and a directional finger based on the time difference and the amplitude difference of the incoming sound from the frequency components output from the frequency analysis means. And a spectral subtraction control means for controlling the spectral subtraction processing of the spectral subtraction means based on the directivity index and the target sound direction output from the target sound direction estimating means. Configure.

[3] Thirdly, the spectrum subtraction means divides the obtained voice frequency into frequency bands to calculate voice power for each band, and the voice band power calculation means. The obtained noise frequency component is divided for each frequency band to calculate noise power for each band, and the frequency of voice and noise obtained from the voice band power calculation unit and the noise band power calculation unit. Based on the band power and the output of the spectrum subtraction control means, a band weight calculation means for obtaining a band weight coefficient, and a spectrum subtraction means for suppressing background noise by multiplying the voice signal by the band weight coefficient for each frequency band. A noise component suppressor characterized by being configured.

[4] Fourthly, the spectral subtraction means converts the obtained audio frequency into
Voice band power calculating means for calculating voice power for each band by dividing for each frequency band, and noise band power calculation for dividing the obtained noise frequency component for each frequency band to calculate noise power for each band Means, input band power calculation means for dividing the frequency components of the input signal obtained by frequency-analyzing the input signal obtained from the voice input means into frequency bands, and calculating the input power for each band, and the voice band power calculation Means and the noise band power calculation means, the frequency band power of voice and noise, the input power for each band obtained by the input band power calculation means, and the output of the spectrum subtraction control means, to obtain a band weighting coefficient. A band weight calculation means and a modified spectrum reduction for suppressing background noise by applying the band weight coefficient for each frequency band of the voice signal. Characterized in that it comprises means.

According to the present invention having such a configuration, in the case of the configuration [1], the voice input means receives the voice uttered by the speaker at two or more different positions, and the frequency analysis means receives the voice. Frequency analysis is performed for each channel of the audio signal corresponding to the sound receiving position, and frequency components of a plurality of channels are output. Then, the first beamformer processing means performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. Incoming noise suppression processing for suppressing voices other than the voice from the speaker direction is performed to obtain a target voice component, and the second beamformer processing means is for the plurality of channels of the plurality of channels obtained by the frequency analysis means. The frequency component is subjected to adaptive filter processing using a filter coefficient calculated so that sensitivity outside the desired direction becomes low, whereby the voice from the speaker direction is suppressed and a noise component is obtained. The noise direction estimating means estimates the noise direction from the filter coefficient calculated by the first beamformer processing means, and the target sound direction estimating means calculates the noise direction by the filter coefficient calculated by the second beamformer processing means. The target sound direction is estimated from. The target sound direction correcting means sequentially corrects the first input direction, which is the arrival direction of the target sound to be input in the first beam former, based on the target sound direction estimated by the target sound direction estimating means. Therefore, the first beam former suppresses noise components coming from other than the first input direction, and extracts the speaker's voice component with low noise. Further, the noise direction correction means sequentially corrects the second input direction, which is the arrival direction of noise to be input in the second beamformer, based on the noise direction estimated by the noise direction estimation means. , The second beam former suppresses the components coming from other than the second input direction and extracts the remaining noise components in which the speaker's voice component is suppressed.

As described above, the present system can separately obtain the voice frequency component in which the noise component is suppressed and the noise frequency component in which the voice component is suppressed, but the greatest feature of the present invention is the first and second aspects. The beamformer operating in the frequency domain is used as the beamformer of the above, and in the present invention, the incoming sound comes from the target direction by using short-time data so as to deal with sudden noise. Incorporating directionality detection means to obtain a directionality index for deciding whether or not to be accurate, and suppressing spectral noise by controlling the spectral subtraction based on the directionality index and the speaker direction in conventional processing. It is in.

According to this, the target sound / noise is determined based on both the directionality index and the speaker direction obtained from the directivity of the beamformer filter,
It is possible to remove voices from areas other than the set speaker range, and it is also possible to remove signals with short durations, such as sudden noise, with high accuracy, making it possible to perform noise suppression processing in an actual environment with extremely high accuracy. Become.

Further, since the beam formers operating in the frequency domain are used as the first and second beam formers, the amount of calculation can be significantly reduced.

According to the present invention, in addition to the processing amount of the adaptive filter being greatly reduced, the frequency analysis processing other than the frequency analysis on the input voice can be omitted, and it is necessary for the filter calculation. The conversion processing from the time domain to the frequency domain is also unnecessary, and the overall calculation amount can be significantly reduced.

That is, in the prior art, a spectral subtraction (hereinafter abbreviated as SS) process is performed after the beamformer process in order to suppress the spreading noise that cannot be suppressed by the beamformer. Takes the frequency spectrum as input, so FFT
Frequency analysis such as (Fast Fourier Transform) has been conventionally required, but when a beamformer operating in the frequency domain is used, the frequency spectrum is output from the beamformer, and this can be diverted to SS. Conventional FFT processing steps that perform FFT for SS can be omitted. Therefore, the total calculation amount can be significantly reduced.

Further, the conversion processing from the time domain to the frequency domain, which is necessary for the direction estimation using the filter of the beamformer, becomes unnecessary, and the total calculation amount can be greatly reduced.

Further, in the case of the configuration [2], the voice input means receives the voice uttered by the speaker at two or more different positions, and the frequency analysis means receives the voice corresponding to the sound receiving position. Frequency analysis is performed for each channel of the signal, and frequency components of a plurality of channels are output. Then, the first beamformer processing means performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. Incoming noise suppression processing for suppressing voices other than the voice from the speaker direction is performed to obtain a target voice component, and the second beamformer processing means is for the plurality of channels of the plurality of channels obtained by the frequency analysis means. The frequency component is subjected to adaptive filter processing using a filter coefficient calculated so that sensitivity outside the desired direction becomes low, whereby the voice from the speaker direction is suppressed and a noise component is obtained. The noise direction estimating means estimates the noise direction from the filter coefficient calculated by the first beamformer processing means, and the target sound direction estimating means calculates the noise direction by the filter coefficient calculated by the second beamformer processing means. The target sound direction is estimated from.

The first target sound direction estimating means estimates the first target sound direction from the filter coefficient calculated by the second beamformer processing means, and the second target sound direction estimating means calculates the target sound direction. The second target sound direction is estimated from the filter coefficient calculated by the third adaptive beam former processing means.

The first input direction correcting means estimates the first input direction, which is the arrival direction of the target sound to be input in the first beam former, by the first target sound direction estimating means. The first target sound direction and the second target sound direction estimated by the second target sound direction estimating means are sequentially corrected based on one or both of them. When the noise direction estimated by the noise direction correcting unit is within a predetermined first range, the second input direction correcting unit determines the arrival direction of noise to be input in the second beamformer. A certain second input direction is sequentially corrected on the basis of the noise direction, and the third input direction correction means, when the noise direction estimated by the noise direction correction means is within a predetermined second range, The third input direction, which is the arrival direction of noise to be input in the third beam former, is sequentially corrected based on the noise direction.

Therefore, the second beam former whose second input direction is corrected by the output of the second input direction correcting means suppresses components coming from other than the second input direction and extracts the remaining noise components. In addition, the third beam former whose third input direction is corrected by the output of the third input direction correcting means suppresses the components arriving from other than the third input direction and remaining noise components. Will be extracted.

Then, the effective noise determining means determines whether the noise direction estimated by the noise direction estimating means comes from the predetermined first range or the predetermined second range. Either one of the output noise and the second output noise is determined to be a true noise output, and either one of the noises is output, and at the same time, one of the first voice direction estimating means and the second voice direction estimating means is output. It is determined whether the estimation result is valid and the valid voice direction estimation result is output to the first input direction correcting means.

As a result, the target sound direction correction means obtains the first input direction, which is the arrival direction of the target sound to be input in the first beam former, by the determined target sound direction estimation means. The first beamformer suppresses the noise components coming from other than the first input direction and extracts the speaker's voice component with low noise, because the correction is sequentially performed based on the target sound direction.

As described above, the present system can separately obtain the voice frequency component in which the noise component is suppressed and the noise frequency component in which the voice component is suppressed, but the greatest feature of the present invention is the first and second aspects. The beamformer operating in the frequency domain is used as the beamformer of the above, and in the present invention, the incoming sound comes from the target direction by using short-time data so as to deal with sudden noise. Incorporating directionality detection means to obtain a directionality index for deciding whether or not to be accurate, and suppressing spectral noise by controlling the spectral subtraction based on the directionality index and the speaker direction in conventional processing. It is in.

According to this, by determining the target sound / noise based on both the above-mentioned directionality index and the speaker direction obtained from the directivity of the filter of the beamformer,
It is possible to remove voices from areas other than the set speaker range, and it is also possible to remove signals with short durations, such as sudden noise, with high accuracy, making it possible to perform noise suppression processing in an actual environment with extremely high accuracy. Become.

Further, since the beam formers operating in the frequency domain are used as the first and second beam formers, the amount of calculation can be significantly reduced.

According to the present invention, the processing amount of the adaptive filter is greatly reduced, and frequency analysis processing other than the frequency analysis on the input voice can be omitted, and it is necessary for the filter calculation. The conversion processing from the time domain to the frequency domain is also unnecessary, and the overall calculation amount can be significantly reduced.

Further, according to the present invention, a noise tracking beamformer having completely different monitoring areas for noise tracking is provided to estimate the voice direction from each output and which is effective from each estimation result. It is determined whether or not the noise tracking is performed properly, and the result of estimation of the voice direction by the filter coefficient of the beamformer that is determined to be effective is given to the first target sound direction correction means to correct the first target sound direction. The means sequentially corrects the arrival direction of the target sound as the input target in the first beam former, based on the target sound direction estimated by the target sound direction estimation means. The beamformer No. 1 can suppress the noise component coming from other than the first input direction and extract the speaker's voice component with low noise, and even if the noise source moves, this is lost. In which it is possible to suppress and tracking without.

In the prior art, 2ch, that is,
In order to enable tracking of the target sound source with only two microphones, one beamformer for noise tracking is used separately from the beamformer for noise suppression. For example, the noise source has moved across the direction of the target sound. In such a case, noise tracking accuracy may decrease.

However, according to the present invention, since a plurality of beam formers for tracking noise are used to handle separate tracking ranges, it is possible to prevent the tracking accuracy from deteriorating even in the above case.

Further, in the case of the configuration of the item [3], the voice band power calculation means divides the obtained spectrum component of the voice frequency into frequency bands to calculate the voice power for each band, and calculates the noise band power. The calculating means divides the obtained spectral component of the noise frequency into frequency bands to calculate noise power for each band. Then, the band weight calculating means, based on the frequency band power of the voice and noise obtained from the voice band power calculating means and the noise band power calculating means, and the output of the spectrum subtraction controlling means, obtains a weighting coefficient for each band, The spectrum subtraction unit suppresses the background noise by applying this weighting coefficient for each frequency band of the audio signal.

According to this configuration, the nondirectional noise that cannot be suppressed by the beamformer (background noise) uses the target voice component and noise component that can be obtained by the beamformer of the system of the present invention, and uses this for spectral subtraction. It is suppressed by processing. That is, this system is provided with two beam formers for extracting a target voice component and a noise component as beam formers. The target voice component and the noise component, which are outputs of these beam formers, are used to perform spectral subtraction. By processing, background noise components having no directionality are suppressed. Spectral subtraction (SS) processing is known as noise suppression processing, but generally performed spectral subtraction (SS) processing uses a one-channel microphone (that is, one microphone), and the output of this microphone is used to output audio. Since the noise power is estimated in a non-existing section, it cannot be dealt with when non-stationary noise is superimposed on the speech. In addition, when two-channel microphones (that is, two microphones) are used and one is used for noise collection and one is used for noise-superimposed voice collection, it is necessary to separate the installation locations of both microphones. ,
The noise superimposed on the voice and the noise captured by the noise collecting microphone are out of phase with each other, and the effect of improving noise suppression is not significantly improved even if the spectral subtraction process is performed.

However, in the present invention, since the beam former for extracting the noise component is prepared and the output of this beam former is used, the phase shift is corrected. Therefore, even in the case of non-stationary noise, the accuracy is high. It is possible to realize various spectrum subtraction processing. Furthermore, since the output of the beamformer in the frequency domain is used, frequency analysis can be omitted and spectral subtraction can be performed, and non-stationary noise can be suppressed with a smaller amount of calculation than before.

Further, the invention of [4] is, in the noise suppressor of the invention of [3], frequency components of the input signal obtained by frequency-analyzing the input signal obtained from the voice input means are divided for each frequency band, An input signal band power calculation means for calculating the input power for each band is provided, and the processing for suppressing the background noise is performed by weighting each frequency band of the voice signal. Band power calculation means, the spectral component of the obtained voice frequency,
Divide each frequency band and calculate the voice power for each band,
The noise band power calculation means calculates the noise power for each band by dividing the obtained spectral component of the noise frequency for each frequency band.

There is also input band power calculation means, which receives the frequency spectrum component of the input voice obtained by frequency-analyzing the input signal obtained from the voice input means, and outputs this frequency frequency component. Divide into each band and calculate the input power for each band.

Then, the band weight calculating means is the voice input frequency band power and the noise input frequency band power obtained from the voice band power calculating means, the noise band power calculating means and the input signal band power calculating means, and Based on the input frequency band power mixed with voice and noise and the output of the spectral subtraction control means, the weighting coefficient for each band is obtained, and the spectrum subtracting means applies the weighting coefficient for each frequency band of the voice signal to the background noise. Suppress.

In the invention of the above item [4], in the spectral subtraction (SS) processing of the invention of the item [3], the power of the noise component is further corrected, so that the noise suppression can be performed with higher accuracy. It makes it possible to do. That is, in the invention of the item [3], it is assumed that the power N of the noise source is small. Therefore, when the spectral subtraction (SS) process is performed, the distortion may be large in the portion where the noise source component is superimposed on the voice. However, here, the power of the input signal is used to correct the calculation of the band weight in the spectral subtraction processing in the third invention.

As a result, it becomes possible to extract only the voice component with a small distortion by suppressing the noise component having a direction and the noise component having no direction.

[0048]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

(Basic Configuration Example) FIG. 1 is a basic configuration diagram of a noise component suppressing apparatus according to the present invention. The feature of this system is that microphones for at least two channels having directivity are arranged as voice input sections with the axial directions of the directivities inclined to each other, and the voice signals obtained by these microphones are subjected to frequency analysis for each channel. By performing adaptive beamformer processing using a filter coefficient calculated so that sensitivity outside the desired direction becomes low, noise from the speaker direction is suppressed to obtain a noise component, and processing for suppressing this noise component is performed. By using a noise suppression processing device that is said to obtain a speaker voice component with less noise, a noise-suppressed speaker voice component and a noise component are obtained, and this noise suppression processing device is used for a short time. A direction detection unit based on frequency analysis is added to detect the arrival of sudden noise or high-speed moving sound source that cannot be suppressed by the beamformer processing.
By controlling the spectral subtraction performed using this detection result and the speaker voice component and noise component obtained by the noise suppression processing device, a speaker tracking function that can set the allowable range in the speaker direction with high accuracy is secured. In addition, it is possible to suppress sudden noise, high-speed moving sound source, and the like.

That is, in FIG. 1, 10 is a speaker tracking microphone array including a voice input section, a frequency analysis section, and a noise suppression processing apparatus, 30 is a spectrum subtraction processing section, 40 is a direction detection section, and 50 is a spectrum. It is a subtraction control unit.

The voice input unit, which is a component of the speaker tracking microphone array 10, is for receiving the voice uttered by the speaker at at least two different positions to obtain a voice signal. By disposing two microphones having directivity with their axes inclined to each other, the sound collection directions are shifted. Also,
The frequency analysis unit, which is a constituent element of the speaker tracking microphone array 10, performs frequency analysis for each channel of the audio signal corresponding to the sound receiving position and outputs frequency components of a plurality of channels.

In addition, the noise suppression processing device, which is a component of the speaker tracking microphone array 10, calculates, for example, the frequency components of the plurality of channels obtained by the frequency analysis unit so that the sensitivity outside the desired direction becomes low. First beamformer processing means for obtaining a target voice component by performing incoming noise suppression processing for suppressing voices other than the voice from the speaker direction by performing adaptive filter processing using the filtered filter coefficient; With respect to the frequency components of the plurality of channels obtained by the analysis unit, the voice from the speaker direction is suppressed by performing adaptive filter processing using a filter coefficient calculated so that the sensitivity outside the desired direction becomes low, Noise is obtained from the second beamformer processing means for obtaining a noise component and the filter coefficient calculated by the first beamformer processing means. Noise direction estimating means for estimating the direction, target sound direction estimating means for estimating the target sound direction from the filter coefficient calculated by the second beamformer processing means, and an object to be an input target in the first beamformer A first sound input direction, which is a sound arrival direction, is sequentially corrected based on the target sound direction estimated by the target sound direction estimating means, and an input target is input in the second beam former. The second direction of noise arrival
And a noise direction correcting means for sequentially correcting the input direction of 1 based on the noise direction estimated by the noise direction estimating means.

The direction detecting section 40 calculates the index D of directionality based on the time difference and the amplitude difference of the incoming sound from the frequency component output from the frequency analyzing section, and the spectral subtraction processing section 30 This is for performing a spectral subtraction process which is a nonlinear noise suppression process based on the first beam former output and the second beam former output.

Further, the spectral subtraction control unit 50, based on the directionality index D obtained by the direction detection unit 40 and the target sound direction output from the target sound direction estimating means of the noise suppression processing device, the spectral subtraction. It controls the processing of the means.

FIG. 2 shows the flow of the entire noise component suppressing process.

First, the basic noise suppression processing will be described. 2 channels here, ie microphone 2
An example using two audio signals captured in the book will be explained.
The processing method is the same even when there are three or more channels.

The voice input from the voice input section of the speaker tracking microphone array 10 having two microphones is
It is sent to the frequency analysis unit, for example, fast Fourier transform (FF
The frequency component is calculated by T) or the like. Next, in the first beamformer, which is one of the constituent elements in the noise suppression processing device of the speaker tracking microphone array 10, the noise is suppressed by the frequency domain adaptive filter from the frequency components for the inputs of the two channels, and the purpose is Outputs the frequency component in the direction of sound. Here, the operation of adjusting the phase by the input direction correction unit 1 is performed using the output from the target sound direction estimation unit so that the direction of the target sound is in front of the microphone.

Further, in the second beam former 16 which is one of the constituent elements in the noise suppression processing device of the speaker tracking microphone array 10, the target sound is obtained from the frequency components corresponding to the inputs of the two channels by the adaptive filter in the frequency domain. Suppress and output the frequency component in the noise direction. Here, it is assumed that the direction of noise is in front of the microphones, and the output from the noise direction estimation unit is used to determine the phase in the second input direction correction unit so that it can be considered that the noises have arrived at the two microphones at the same time. Perform the adjustment operation (phase adjustment).

Here, the speaker tracking microphone array 10
In the noise direction estimation unit, which is one of the constituent elements of the noise suppression processing device, the noise direction is estimated from the adaptive filter of the first beam former, and the target sound direction estimation unit and the adaptive filter of the second beam former are used. Estimate the sound direction.
Note that these processes are performed at fixed time intervals such as 8 [msec].

Next, the direction detecting section 40 and the spectrum subtraction control section 50, which are important points of the present invention, will be described. In the direction detection unit 40, the direction index is calculated using not only the phase difference between the two microphones but also the power ratio of the input signal of each channel, which will be described later in detail, based on frequency analysis such as short-time FFT. To do. The spectral subtraction control unit 50 includes a target sound direction (speaker direction) estimated by the target sound direction estimation unit, which is one of the components of the noise suppression processing device of the speaker tracking microphone array 10, and the directionality detection unit 40. As shown in FIG. 2, based on the directionality index D calculated in step 1, any one of the three types of signals corresponding to the value of the directionality index D is sent to the spectrum subtraction processing section 30, and the spectrum subtraction processing section 30 At 30, the spectral subtraction processing, which is the diffusion noise suppression processing, is performed according to the three types of signals.

Here, the three types of signals output from the spectrum subtraction control unit 50 refer to three types of signals of "0", "1", and "2".
The signal "0" represents a section containing almost only noise, the signal "1" represents a section in which large sudden noise is superimposed on the speech section, and the signal "2" represents a section containing almost only speech. It means that.

First, as initial settings, for example, the permissible range in the speaker direction is set to ± 20 ° from the center of the two microphones, and the threshold value of the directionality index D is set to "1.0" (step S20 in FIG. 2). ). This threshold is an example, and it is actually preferable to adjust it by experimentation in the usage environment.

Here, it is determined whether or not the directionality index D sent from the directionality detecting section 40 is less than or equal to the threshold value (1.0) (steps S21, S22, S23 in FIG. 2), and If the result is less than or equal to the threshold value, then it is determined whether the target sound direction (speaker direction) is within the setting range (step S24 in FIG. 2). If it is within the setting range, the spectral subtraction processing unit 30 Send signal 2 "(step S in FIG. 2)
25). If it is outside the set range, the signal "0" is sent (step S26 in FIG. 2).

If the directionality index D is equal to or more than the threshold value and the target sound direction is within the set range (step S21 in FIG. 2,
(S22, S23, S27), it is determined that sudden noise is superimposed on the voice coming from the speaker direction, and a signal "1" is sent (step S28 in FIG. 2), and the directionality index D is a threshold value. If the target sound direction is out of the setting range (see FIG. 2).
Of steps S21, S22, S23 and S27), and the signal "0" is sent (step S226 of FIG. 2).

In the spectral subtraction processing unit 30, when the signal from the spectral subtraction control unit 50 is "0", it can be regarded almost as a noise section, so that the output signal is cut by applying the minimum weight. Then, this is output as the final audio frequency component.

When the signal from the spectrum subtraction control unit 50 is "1", it is considered that sudden noise is superimposed on the voice section, and the output from the second beamformer is treated as a noise component in two channels. The spectral subtraction process of is performed. Then, this is output as the final audio frequency component.

Further, in the spectrum subtraction processing section 30, when the signal from the spectrum subtraction control section 50 is "2", it is considered as a voice-only section, and the spectrum of one channel is output for the output of the first beamformer. Perform subtraction. Then, this is output as the final audio frequency component.

As another control method, when the signal is "1", a large sudden noise is superposed on the voice, so that it may be regarded as a noise section and may be processed in the same manner as the signal "0".

As a result, it is possible to obtain a system capable of suppressing the sudden noise, the high-speed moving sound source, etc. while securing the speaker tracking function capable of setting the allowable range of the speaker direction with high accuracy.

The above is an outline of the present invention. The details of each component necessary for constructing such a system will be described.

<Configuration Example 1 of Speaker Tracking Microphone Array 10> A configuration example of the speaker tracking microphone array 10 will be described. This example corresponds to the content of claim 0A1.

FIG. 3 is a block diagram showing a configuration example of a system as a configuration example 1 of the speaker tracking microphone array 10, and is a block diagram showing a basic configuration of a noise suppressing device according to an embodiment of the present invention. . The present invention is a technique for enabling speaker tracking even when the number of microphones is 2 ch (ch; channel), that is, a minimum of two microphones. Even in the case, the processing method is the same.

In FIG. 3, 11 is a voice input unit, 12 is a frequency analysis unit, 13 is a first beam former, 14 is a first input direction correction unit, 15 is a second input direction correction unit, 1
6 is a second beam former, 17 is a noise direction estimation unit, 1
Reference numeral 8 denotes a target sound direction estimation unit (voice direction estimation unit).

Of these, the voice input unit 11 is, for example, for receiving the voice (target voice) uttered by the speaker whose voice is to be collected at two or more different positions. 2 were installed at different points
A microphone is used to capture voice and convert it into an electrical signal. The frequency analysis unit 12 performs frequency analysis for each channel of the audio signal corresponding to the sound receiving position of the microphone and outputs frequency components of a plurality of channels. Specifically, here, the first microphone is used. Signal component in the time domain by separately performing fast Fourier transform on the captured audio signal (the audio signal of the first channel ch1) and the audio signal captured by the second microphone (the audio signal of the second channel ch2), respectively. Is converted into frequency domain component data, and converted into frequency spectrum data for each channel and output.

The first beam former 13 uses the frequency component outputs of a plurality of channels from the frequency analysis unit 12, in this case, the audio signals of ch1 and ch2, to extract the frequency component of the target audio from this. By using the frequency components of each of ch1 and ch2 (frequency spectrum data) to suppress incoming noise other than the target voice by adaptive filter processing, the frequency components from the target sound source direction can be detected. The second beam former 16 is a processing means for performing extraction, and the second beam former 16 outputs frequency components of a plurality of channels from the frequency analysis unit 12.
In this case, the audio signal of ch1 and ch2 is used to extract the frequency component from the noise source direction, and the adaptive filter is performed by using the frequency component (frequency spectrum data) of each of the ch1 and ch2. It is a processing unit that performs processing of suppressing components other than voice from the noise source direction by processing, thereby extracting data of frequency spectrum components from the noise source direction.

Further, the noise direction estimation unit 17 performs a process of estimating the noise direction from the filter coefficient calculated by the first beam former 13, and specifically, the first beam The noise direction is estimated using parameters such as filter coefficients for filtering processing obtained from the adaptive filter of the former 13, data corresponding to the estimated amount is output, and a target sound direction estimation unit (speech direction estimation unit) 18 Is a process of estimating the target sound direction from the filter coefficient calculated by the second beamformer 16, and is specifically used by the adaptive filter of the second beamformer 16. The noise direction is estimated from parameters such as existing filter coefficients, and data corresponding to the estimated amount is output.

The first input direction correcting section 14 is for correcting the input direction of the beamformer to the original target sound direction, and is an object to be an input target in the first beamformer 13. A first input direction that is a sound arrival direction is generated based on the target sound direction estimated by the target sound direction estimation unit 18, and an output for sequentially correcting the generated direction is generated and given to the first beamformer 13. Is. Specifically, the first input direction correction unit 14 converts the data corresponding to the estimated amount output from the target sound direction estimation unit 18 into the current target sound source direction angle information α to obtain target angle information α. It is output to the first beam former 13.

The second input direction correcting section 15 is for correcting the input direction of the second beam former 16 to the noise direction, and in the second beam former 16,
An output for sequentially correcting the second input direction, which is the arrival direction of the noise to be input, based on the noise direction estimated by the noise direction estimation unit 17, is generated, and is output to the second beamformer 14. To give. Specifically, the second input direction correction unit 15 converts the data corresponding to the estimated amount output from the noise direction estimation unit 17 into the current target angle information of the noise source direction to obtain the target angle information α. The beam is output to the second beam former 16.

Here, a configuration example of the beam formers 13 and 16 will be shown.

<Example of Configuration of Beamformer> The beamformers 13 and 16 used in the system of the present invention have a configuration as shown in FIG. That is, the beam formers 13 and 16 used in the system of the present invention, in order to be able to obtain the target signal component to be extracted from the input voice, in the arrival direction of the target signal component to be extracted, It comprises a phase shifter 100 for setting the input direction of the former, and a beamformer main body 101 for suppressing components from directions other than the arrival direction of the signal component to be extracted.

The phase shifter 100 is the correction vector generator 100.
a and multiplication means 100b and 100c, the beamformer main body 101 includes addition means 101a and 101b,
101c and an adaptive filter 101d.

The correction vector generation unit 100a receives the angle information α from the input direction correction unit 14 or 15 as the input direction information and generates a correction vector corresponding to α from this, and the multiplication means 100b performs the frequency analysis. The data of the frequency spectrum component of ch1 output from the unit 12 is multiplied by the correction vector and output.
The multiplication means 100c outputs c from the frequency analysis unit 12.
The data of the frequency spectrum component of h2 is multiplied by the correction vector and output.

The adding means 101a is the multiplying means 100.
The output of b and the output of the addition means 100c are added and output, the addition means 101b outputs the difference between the output of the multiplication means 100b and the output of the addition means 100c, and the addition means 101c is the addition means 101a. Is output as the output of the beamformer.
d is a digital filter for performing filtering calculation processing on the output of the adding means 101b and outputting the result.
The filter coefficient (parameter) is sequentially changed so that the output of the adding means 101c becomes the minimum.

In this example, the microphone configuration is 2
Book, that is, the first and second microphones m1 and m2
In this case, the input direction of the beamformer is set to the audio signal from the direction in which the input target exists, as shown in FIG. 4B. , So that they can be considered equivalently arriving at both microphones m1 and m2 at the same time.
This refers to delaying the frequency components of the two audio channels of 1 and ch2 so that their phases are aligned (phased). This is because in the case of the configuration of FIG.
It is realized by adjusting the phase shift in the phase shift unit 100 in accordance with the angle information α output by 15.

That is, in the case of the configuration of FIG.
For 0, the correction vector corresponding to the input direction (angle information α) desired to be corrected is generated by the correction vector generation unit 100a, and the multiplication means 100b and 100 for multiplying the signals of the channels of 1ch and 2ch respectively by this correction vector.
The phase shift unit 100 configured to multiply by c aligns the phases as follows.

For example, in an omnidirectional microphone arrangement as shown by adding symbols m1 and m2 in FIG.
Consider that the speaker, which is the target sound source at one point, corrects the phase of the signal as if he / she were at point P2. In such a case, the phase of the speaker voice signal (ch1) detected by the first microphone m1 separated by the distance d and the second phase
Speech signal (ch2) detected by microphone m2
So as to have the same phase, the complex number W1 W1 = (cos jωτ in the speaker voice signal (ch1) of the first microphone m1 corresponding to the propagation time difference τ τ = rc = rsinα r = dsinα , Sin j ω τ) complex conjugate. Here, c is the speed of sound, d is the distance between the microphones, α is the moving angle of the speaker as the sound source of the target sound seen from the microphone m1, j is the imaginary number, and ω is the angular frequency.

That is, focusing on the voice of the target sound source moved to the angle α by applying the complex conjugate of W1, the signal (ch) captured by the first microphone m1
1) means that the phase shift control is performed so that the signal has the same phase as the signal captured by the second microphone m2.

The signal of the second microphone m2 (ch
It is assumed that the complex conjugate of the complex number W2 = (1,0) is applied to 2). That is, this means that the signal (ch2) of the second microphone m2 is not subjected to angle correction.

Here, the vector {W1, W2} in which the complex numbers W1 and W2 are arranged is generally called a direction vector, and the vector conjugate {W1 *, W2 *} of the complex conjugate in {W1, W2} is It is called a correction vector.

If a correction vector is generated corresponding to the angle information α and the frequency spectrum components of ch1 and ch2 are multiplied by this correction vector, the first microphone m1
The output of is corrected so as to have the same phase as that of the second microphone m2, even though the sound source has moved from P1 to P2. As far as the first microphone m1 is concerned, the second microphone m1 , M2 have the same distance to the P2 position sound source.

In this embodiment, there are two beam formers. Of these two beam formers, the first beam former 13 uses the phase shifter 100 to set the sound source direction of the target sound as the input target direction. , Ch1 (or c
The frequency component of h2) is delayed by the above-described method, and the second beam former 16 uses the phase shifter 100 to set ch1 (or ch1) so that the noise source direction is the input target direction.
The frequency component of 2) is delayed by the above method to align the phases of both. However, with respect to the sound component from the direction other than the arrival direction of the target sound S, that is, the noise component N, the phases of the first and second microphones m1 and m2 are completely uncorrected, so that the first microphone m1 and the second microphone m There is a time difference in the timing detected by the microphone m2.

In this way, the output of the first microphone m1 whose phase is corrected by the phase shifter 100 with respect to the voice signal detected from the sound source in the target sound direction (the frequency spectrum of ch1 composed of the target voice component S and the noise component N) Data) and the uncorrected output of the second microphone m2 (the frequency spectrum data of ch2 consisting of the target voice component S and the noise component N ') are respectively added by the adding means 101a, 1a.
01b is input. Then, in the adding means 101a, c
By adding the output of h1 and the output of ch2, the power component of the signal of twice the target voice S and the noise component N + N 'is obtained, and the adding means 101b outputs ch1 (S + N) and ch2 (S + N). ′) Difference ((S
+ N)-(S + N ') = NN'), that is, the power component of noise is obtained. Then, the adding means 101c
Then, the adaptive filter 101 for the output of the adding means 101a
The difference between the outputs of d is calculated, and this difference is used as the output of the beamformer and fed back to the adaptive filter 101d.

The adaptive filter 101d is addition means 101b.
Is a digital filter for performing a filtering calculation process so as to extract the frequency spectrum of the sound component arriving from the direction corresponding to the current search direction with respect to the output of The search angle is variable, and the maximum output is output when the search angle matches the input signal direction. Therefore, if the incident direction of the incoming signal and the search angle match, the output of the adaptive filter 101d (N-
N ') is maximum. Since the output (N-N ') of the adaptive filter 101d is the power of the noise component, the output when it is maximum is given to the adding means 101c and subtracted from the output (2S + N + N') from the adding means 101a. , The noise component N is maximally canceled to suppress the noise. Therefore, in this state, the output of the adding means 101c is the minimum.

Therefore, the adaptive filter 101d sequentially changes the signal arrival direction search angle in 1 ° increments (sensitivity by direction in 1 ° increments) and the filter coefficient (parameter) so that the output of the adding means 101c is minimized. As a result, the incident direction of the incoming signal and the search angle (the incoming direction of the incoming signal and the sensitivity to the direction) match, so that the adaptive filter 101d controls them and the output of the adding means 101c becomes the minimum. To do so.

That is, as a result of this control, the beam former can extract the voice component from the target direction. Further, when the noise component is extracted as the target sound, the above-mentioned control may be performed in such a manner that the above-mentioned target sound is regarded as noise.

Regarding the beamformer main body 101, in addition to the generalized sidelobe canceller (GSC),
Various types such as a frost type beam former can be applied in the same idea as described above, and therefore, the present invention is not particularly limited.

The operation of the present system having such a configuration will be described. This system is characterized in that the voice frequency component and the noise frequency component of the target sound are separately extracted and output.

First, the voice input unit 11 having a plurality of microphones, in this example, the voice input unit 11 having a total of two microphones m1 and m2, captures the voices of ch1 and ch2. The two channels of audio signals ch1 and ch2 input from the audio input unit 11 (that is, the first channel ch1 is the audio from the first microphone m1 and the second channel ch2 is the second microphone m).
(Corresponding to the voice from 2) is sent to the frequency analysis unit 12, where a frequency component (frequency spectrum) is obtained for each channel by performing processing such as fast Fourier transform (FFT).

The frequency components for each channel obtained by the frequency analysis unit 12 are given to the first and second beam formers 13 and 16, respectively.

In the first beam former 13, the noise components are processed by adjusting the phases of the frequency component inputs for two channels according to the direction of the target sound and then processing the same as described above by the adaptive filter in the frequency domain. Suppress and output the frequency component in the direction of the target sound.

Here, specifically, the first input direction correction section 14 gives the following angle information (α) to the first beam former 13. That is, the first input direction correction unit 14 adjusts the input phase of the frequency components of the two channels by using the output from the given voice direction estimation unit 18 so that the direction of the target sound is the front direction of the microphone. The angle information (α) required for the first beam former 13 is given to the first beam former 13 as an input direction correction amount.

As a result, the first beam former 13 corrects the target sound direction in accordance with the correction amount (α), and suppresses the voice coming from the direction other than the target sound direction, thereby reducing the noise component. Is suppressed and the target sound is extracted.

That is, the target sound direction estimating unit 18 knows the noise source direction by using the parameter of the adaptive filter in the second beam former 16 for extracting the noise component,
An output reflecting that is output, and the first input direction correction unit 1
In 4, the input direction correction amount (α) is generated corresponding to the output from the target sound direction estimation unit 18, and the target sound direction in the first beam former 13 is corrected corresponding to the correction amount (α),
As a result, the first beam former 13 suppresses the voice coming from a direction other than the target sound direction, thereby suppressing the noise component and extracting the target sound.

That is, in the case of the second beam former 16, since the noise is the target sound, the phase is matched with the noise. As a result, the sound source of the speaker is treated as a noise source by the second beamformer 16, and the adaptive filter incorporated in the beamformer carries out the process of extracting the sound from the speaker sound source. An output reflecting the direction of the speaker sound source is obtained from the parameter of the adaptive filter of the beam former 16. Therefore, by the target sound direction estimation unit 18,
If the noise source direction is known by using the parameter of the adaptive filter in the second beam former 16, it reflects the direction of the speaker sound source which is the target sound. Therefore, the target sound direction estimation unit 18 causes the second beam former 16 to
An output reflecting the parameter of the adaptive filter in is output, and the first input direction correction unit 14 generates an input direction correction amount (α) corresponding to the output from the target sound direction estimation unit 18, and corresponds to this correction amount. If the target sound direction in the first beam former 13 is corrected, the first beam former 13
In addition, it is possible to suppress a voice coming from a direction other than the target sound direction.

Further, in the second beam former 16, 2
For the frequency component input for the channel, the target sound is suppressed by the adaptive filter in the frequency domain, and the frequency component in the noise direction is output. Here, specifically, assuming that the direction of the noise is in front of the microphones, it is possible to use the output from the noise direction estimating unit 17 so that the noises arrive at the two microphones at the same time. The input direction correction unit 5 performs an operation for adjusting the phase (phase adjustment).

That is, the noise direction estimation unit 17 knows the noise source direction by using the parameter of the adaptive filter in the first beam former 13 for extracting the speaker voice component, and outputs the reflected noise direction. The second input direction correction unit 15 generates an input direction correction amount (α) corresponding to the output from the noise direction estimation unit 17 and supplies the input direction correction amount (α) to the second beam former 16. Only the noise component is extracted by correcting the noise direction according to the correction amount and suppressing the voice coming from directions other than this direction.

Here, the noise direction estimation unit 17 estimates the noise direction from the adaptive filter of the first beam former 13, and the target sound direction estimation unit 18 uses the adaptive filter of the second beam former 16 to determine the target sound direction. To estimate.

Note that these processes are performed by, for example, 8 [mse
c] etc. every short fixed time. Hereinafter, the fixed time is called a frame.

In this way, the first beam former 1
3 makes it possible to extract the voice component of the target sound (speaker), and the second beam former 16 makes it possible to extract the noise component.

If the environment in which the apparatus is installed is a quiet conference room and the video conference system is installed in this conference room and is used for extracting the speaker voice of the video conference system, remove it. Even if it is the noise that must be
Since it is considered that the problem is not a large disturbing sound, in such a case, the first beamformer 13 performs an inverse Fourier transform on the extracted target sound (speaker) component and returns it to the time domain. By returning to a voice signal and outputting it as voice with a speaker or transmitting it, it can be used as a speaker voice with reduced noise.

Here, the processing procedure of the direction estimating units 17 and 18 will be touched upon.

[Processing Procedure of Direction Estimating Unit] FIG. 5 shows a processing procedure of the direction estimating units 17 and 18.

This processing is performed for each frame. First,
Initial settings are made (step S1). As the contents of this initial setting, as shown by enclosing with a dotted line frame in FIG. 5, the “target sound tracking range” is set to “0 ° ± θr (for example, 20 °)”, and the other range is searched for noise. Set as a range.

When the initial setting is completed, the process proceeds to step S2. In this step S2, a process of generating a direction vector is performed. Then, after the direction-specific sensitivity calculation is performed, the direction-specific sensitivity frequency is accumulated (steps S3 and S).
4).

Then, after carrying out this for all frequencies and directions, the one having the minimum value is obtained, and the direction of the one having the minimum accumulated value is set as the signal arrival direction (steps S5 and S6). ).

That is, specifically, steps S2 to S
4, the filter coefficient W (k) and the directional vector S
A process of calculating the inner product with (k, θ) for each frequency component in steps of 1 ° in a predetermined range, obtaining the sensitivity in the corresponding direction, and then adding the sensitivity for all frequency components. I do. Then, in steps S7 and S8, of the cumulative values for each direction obtained as a result of adding the sensitivities of all frequency components, the direction in which the value is the minimum value is called the signal arrival direction. .

The processing procedure shown in FIG. 5 is the same for both the noise direction estimating section 17 and the target sound estimating section 18.

In this way, the noise direction estimation unit 17 estimates the noise direction, and the target sound estimation unit 18 estimates the target sound direction. Then, this estimation result is given to the corresponding input direction correction units 14 and 15.

The first input direction correction unit 14 having received the noise direction estimation result averages the direction estimation results of the input direction up to the previous frame and the current frame, calculates a new input direction, and shifts the beamformer. Output to the phase section 100,
Second input direction correction unit 15 that has received the target sound estimation result
Also, the input direction up to the previous frame and the direction estimation result of the current frame are averaged, a new input direction is calculated, and the result is output to the phase shifter 100 of the beamformer.

The averaging is performed, for example, using the coefficient β as in the following equation.

Θ1 (n) = θ1 (n−1) · (1−
α) + E (n) · β where θ1 is the input direction of the sound, n is the number of the processing frame, and E is the direction estimation result of the current frame. The coefficient β may be variable based on the output power of the beam former.

When the beam former is GSC, conventionally, it is necessary to transform the filter coefficient in the time domain into the frequency domain in the direction estimation, but in the present invention, the adaptive filter of GSC is applied to the frequency spectrum. The filter coefficient used for the filter calculation processing is originally obtained in the frequency domain. The process of transforming the filter coefficient in the time domain into the frequency domain is unnecessary. Therefore,
Even if GSC is used in the system of the present invention, it is possible to speed up the processing because the conversion from the filter coefficient in the time domain to the frequency domain is unnecessary.

[Overall Processing Procedure] FIG. 6 shows the overall processing procedure of the system. This process is performed for each frame.

First, initialization is performed (step S11).
As the initial setting content, the tracking range in the direction of the target sound is 0 ° ±
θr (for example, θr = 20 °), and the noise direction estimation unit 17
Is set to θr <φ1 <180 ° −θr, −180 ° + θr <φ1 <−θr, and the search range of the target sound direction estimation unit 18 is set to −θr <φ2 <θr.

Then, the initial value of the input direction of the target sound is θ1.
= 0 °, and the initial value of the noise input direction is θ2 = 90 °.

When the initial setting is completed, first, the process of the first beam former 13 is performed (step S12), the noise direction is estimated (step S13), and if the noise direction is within the range of φ2, the The input direction of the second beam former 16 is corrected (steps S14 and S15), and otherwise it is not corrected (step S14).

Next, the process proceeds to the second beam former 16 (step S16), and the direction of the target sound is estimated (step S17). Then, if the estimated direction of the target sound is within the range of φ1, the input direction of the first beam former 13 is corrected (steps S18 and S19), and otherwise, the next frame is processed without doing anything. Move on to.

The above example is characterized in that the beamformer operating in the frequency domain is used as the beamformer, which makes it possible to greatly reduce the amount of calculation.

That is, voice input means for receiving the voice uttered by the speaker at at least two different positions,
Frequency analysis means for performing frequency analysis for each channel of the audio signal corresponding to the sound receiving position to output frequency components of a plurality of channels, and frequency components of the plurality of channels obtained by the frequency analysis means are outside the desired direction. The incoming noise suppressing process for suppressing the voice other than the voice from the speaker direction is performed by performing the adaptive filter process using the filter coefficient calculated so that the sensitivity of The speaker is obtained by performing adaptive filter processing on the frequency components of the plurality of channels obtained by the beamformer processing means and the frequency analysis means using filter coefficients calculated so that sensitivity outside the desired direction becomes low. A second beamformer processing means for suppressing voice from a direction and obtaining a noise component, and the first beamformer processing means. Noise direction estimation means for estimating the noise direction from the calculated filter coefficient, target sound direction estimation means for estimating the target sound direction from the filter coefficient calculated by the second beamformer processing means, and the first beam Target sound direction correcting means for sequentially correcting the first input direction, which is the arrival direction of the target sound to be input in the former, based on the target sound direction estimated by the target sound direction estimating means; The beam direction further includes a noise direction correction unit that sequentially corrects a second input direction, which is a direction of arrival of noise to be input, based on the noise direction estimated by the noise direction estimation unit.

Then, the voice input means receives the voice uttered by the speaker at two or more different positions, and the frequency analyzing means frequency-analyzes this for each channel of the voice signal corresponding to the sound receiving position. To output frequency components of multiple channels. Then, the first beamformer processing means performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. By the incoming noise suppression processing for suppressing voices other than the voice from the speaker direction to obtain a target voice component, and
The beamformer processing means of (1) performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means by using filter coefficients calculated so that sensitivity outside the desired direction becomes low. Suppresses the voice coming from the person and obtains a noise component. The noise direction estimating means estimates the noise direction from the filter coefficient calculated by the first beamformer processing means, and the target sound direction estimating means calculates the noise direction by the filter coefficient calculated by the second beamformer processing means. The target sound direction is estimated from. The target sound direction correcting means sequentially corrects the first input direction, which is the arrival direction of the target sound to be input in the first beam former, based on the target sound direction estimated by the target sound direction estimating means. Therefore, the first beam former suppresses noise components coming from other than the first input direction, and extracts the speaker's voice component with low noise. Further, the noise direction correction means sequentially corrects the second input direction, which is the arrival direction of noise to be input in the second beamformer, based on the noise direction estimated by the noise direction estimation means. , The second beam former suppresses the components coming from other than the second input direction and extracts the remaining noise components in which the speaker's voice component is suppressed.

As described above, the present system can separately obtain the voice frequency component in which the noise component is suppressed and the noise frequency component in which the voice component is suppressed, but the greatest feature of the present invention is that the first and second The point is that a beamformer that operates in the frequency domain is used as the beamformer. And by doing so, the amount of calculation can be significantly reduced.

According to the present invention, the processing amount of the adaptive filter is significantly reduced, and frequency analysis processing other than the frequency analysis on the input voice can be omitted, which is necessary for the filter calculation. The conversion processing from the time domain to the frequency domain is also unnecessary, and the overall calculation amount can be significantly reduced.

That is, in the prior art, the spectral subtraction (hereinafter abbreviated as SS) process is performed after the beamformer process in order to suppress the spreading noise that cannot be suppressed by the beamformer. Takes the frequency spectrum as input, so FFT
Frequency analysis such as (Fast Fourier Transform) has been conventionally required, but when a beamformer operating in the frequency domain is used, the frequency spectrum is output from the beamformer, and this can be diverted to SS. Conventional FFT processing steps that perform FFT for SS can be omitted. Therefore, the total calculation amount can be significantly reduced.

Further, the conversion processing from the time domain to the frequency domain, which is necessary for the direction estimation using the filter of the beamformer, becomes unnecessary, and the total calculation amount can be greatly reduced.

Next, an example will be described in which tracking can be performed with high accuracy even when the noise source moves across the range of the target sound direction.

<Structure Example 2 of Speaker Tracking Microphone Array 10> Another structure example of the speaker tracking microphone array 10 will be described.

In this example, tracking can be performed with high accuracy even when the noise source moves across the range of the target sound direction.
An example of using two beam formers that track noise will be described.

FIG. 7 shows an overall configuration diagram as a configuration example 2 of the speaker tracking microphone array 10. In FIG. 7, 1
1 is a voice input unit, 12 is a frequency analysis unit, 13 is a first beamformer, 14 is a first input direction correction unit, 15 is a second input direction correction unit, 16 is a second beamformer, 1
7 is a noise direction estimation unit, 18 is a first voice direction estimation unit (target sound direction estimation unit), 21 is a third input direction correction unit, 22 is a third beam former, and 23 is a second voice. The direction estimating unit, 24 is an effective noise determining unit.

Of these, the third input direction correction unit 21
Is for correcting the input direction of the third beam former 22 to the noise direction, and
2 is the arrival direction of noise to be input, the third
The output direction for sequentially correcting the input direction of (1) based on the noise direction estimated by the noise direction estimating unit 17 is generated and given to the third beam former 22. Specifically, the third input direction correction unit 21 converts the data corresponding to the estimated amount output from the noise direction estimation unit 17 into the current target angle information of the noise source direction, and sets it as the target angle information α. 3 to the beam former 22.

The third beam former 22 uses the frequency component outputs of a plurality of channels from the frequency analysis unit 12, in this case, the frequency spectra of the audio signals of 1ch and 2ch, and from this, the frequency spectrum components from the noise source direction. Is performed to suppress the frequency spectrum components other than the noise source direction by the adaptive filter processing in which the sensitivity adjustment for each direction is performed on the frequency components (frequency spectrum data) of each of the 1ch and 2ch. Thus, it is a processing means for extracting data of the frequency spectrum component from the noise source direction. Like the first and second beam formers 13 and 16, the third beam former 22 also employs the configuration as described with reference to FIG.

The second voice direction estimating unit 23 is similar to the target voice estimating unit (voice direction estimating unit) 18, and determines the target sound direction from the filter coefficient calculated by the third beam former 22. This is a process for estimating, and specifically, the voice direction is estimated from the adaptive filter of the third beamformer 22 and the data corresponding to the estimated amount is output.

The effective noise determining section 24 is the voice direction estimating section 1
8, 23 and the information on the voice direction and the noise direction estimated by the noise direction estimation unit 17, it is determined which of the second beamformer 16 and the third beamformer 22 is effectively tracking noise, The output of the beam former determined to be effectively tracking is output as a noise component. In addition, since the components denoted by the same reference numerals as those of the configuration of FIG. 3 indicate the same components, the details will be referred to the above description, and will not be described again here.

As can be seen from the figure, the difference between the speaker tracking microphone array 10 of the configuration example 2 and the speaker tracking microphone array 10 of the configuration example 1 is that in the configuration example 1, the third input direction correction unit 21 is further provided. And the third beam former 22
And a second voice direction estimating unit 23 and an effective noise determining unit 24 are added.

Then, the second and third beam formers 1
6, 22 and the output of the noise direction estimation unit 17, and the outputs of the first and second voice direction estimation units 18, 23 are passed to the effective noise determination unit 24, and the output of the effective noise determination unit 24 is transferred. It is configured to be passed to the first input direction correction unit 14.

The operation of the present system having such a configuration will be described.

First, the voice input unit 11 having a plurality of microphones, in this example, the voice input unit 11 having a total of two microphones m1 and m2, captures the voices of ch1 and ch2. The two channels of audio signals ch1 and ch2 input from the audio input unit 11 (that is, the first channel ch1 is the audio from the first microphone m1 and the second channel ch2 is the second microphone m).
(Corresponding to the voice from 2) is sent to the frequency analysis unit 12, where a frequency component (frequency spectrum) is obtained for each channel by performing processing such as fast Fourier transform (FFT).

The frequency components for each channel obtained by the frequency analysis unit 12 are respectively the first, second and third frequency components.
Beam formers 13, 16 and 22.

In the first beam former 13, the frequency components of the two channels are matched in phase according to the direction of the target sound, and processed by the adaptive filter in the frequency domain as described above to reduce noise. Suppress and output the frequency component in the direction of the target sound. Here, specifically describing, the first input direction correction unit 14 gives the following angle information (α) to the first beam former 13. That is, the first input direction correction unit 14 includes the effective noise determination unit 24.
It is necessary to adjust the input phase of the frequency components of the two channels by using the output from the voice direction estimation unit 18 or the voice direction estimation unit 23 given via the so that the direction of the target sound is as if it were the front direction of the microphone. The first beam former 1 with the angle information (α) as the input direction correction amount
Give to 3.

As a result, the first beam former 13 corrects the target sound direction in accordance with the correction amount (α), and suppresses the voice coming from the direction other than the target sound direction, whereby the noise component Is suppressed and the target sound is extracted.

That is, in the case of the second and third beam formers 16 and 22, since the noise is the target sound, the phase is matched with the noise. As a result, the sound source of the speaker is treated as a noise source in the second and third beamformers 16 and 22,
Since the adaptive filter built in each beamformer performs the process of extracting the sound from the speaker sound source, the direction of the speaker sound source can be determined from the parameters of the adaptive filters of the second and third beamformers 16 and 22. Information that reflects is obtained.

Therefore, if the noise source direction is known by the first or second speech direction estimation unit 18 or 23 using the parameter of the adaptive filter in the second or third beam former 16 or 22, it is the target sound. It reflects the direction of a speaker's sound source. Therefore, the first or second voice direction estimation unit 18 or 23 outputs an output reflecting the parameter of the adaptive filter in the second or third beam former 16 or 22, and the first input direction correction unit 14 outputs the output. If an input direction correction amount (α) is generated in response to this output and the target sound direction in the first beam former 13 is corrected in accordance with this correction amount, the first beam former 13 will move from a direction other than the target sound direction. Since the incoming voice is suppressed, in this case, the component from the speaker sound source can be extracted.

On the other hand, since the parameter is controlled by the adaptive filter of the first beam former 13 so that the noise component is extracted, the noise direction estimation unit 17 estimates the noise direction from this parameter and outputs the information. Second and third
To the effective noise determining unit 24 and the input direction correcting units 15 and 21.

Then, the second input direction correction unit 15 receiving the output from the noise direction estimation unit 17 generates the input direction correction amount (α) corresponding to the output from the noise direction estimation unit 17, and If the target sound direction in the second beamformer 16 is corrected according to the correction amount, the second beamformer 16 suppresses the voice coming from directions other than the target sound direction. Therefore, the noise component that is the component of can be extracted.

At this time, since the parameters are controlled by the adaptive filter of the second beam former 16 so as to extract the speaker voice component which is the target sound, the first voice direction estimation unit 18 uses this parameter from the parameters. , The speaker voice direction can be estimated. Then, the first voice direction estimating unit 18 gives the estimated information to the effective noise determining unit 24.

The output from the noise direction estimation unit 17 is also given to the third input direction correction unit 21, and the third input direction correction unit 21 having received this output, the noise direction estimation unit 17 concerned. The input direction correction amount (α) is generated in response to the output from and is given to the third beam former 22. This allows
The third beam former 22 corrects its own target sound direction in accordance with the given correction amount.

As a result, the third beam former 22 suppresses a voice coming from a direction other than the target sound direction, and in this case, a component other than the speaker sound source, that is, a noise component can be extracted.

At this time, since the parameters are controlled by the adaptive filter of the third beam former 22 so that the speaker voice component which is the target sound is extracted, the second voice direction estimation unit 23 uses this parameter from the parameters. , The speaker voice direction can be estimated. Then, this estimated information is given to the effective noise determination unit 24.

In the effective noise determining section 24, the first and second
Of the second beamformer 1 based on the speaker voice direction estimation information given from the voice direction estimation units 18 and 23 and the noise direction estimation information given from the noise direction estimation unit 17.
It is determined which of the sixth beamformer 22 and the third beamformer 22 is effectively tracking noise. Then, based on the result of this determination, the parameter of the adaptive filter in the beamformer that is determined to be effectively tracking is given to the first input direction correction unit 14.

Therefore, the first input direction correction unit 14 outputs an output reflecting the parameter, and the first input direction correction unit 14 generates the input direction correction amount (α) corresponding to this output. Since the target sound direction in the first beam former 13 is corrected according to the correction amount, the first beam former 13 is supposed to suppress the voice coming from a direction other than the target sound direction. In this case, the speaker It is possible to extract a component from a sound source, and when noise from a widely moving noise source is targeted, it is possible to reliably capture and remove the noise without losing the moving noise source.

That is, in this embodiment, the first beam former 1 is used for extracting the voice frequency component of the speaker.
3 is provided, and second and third beam formers 16 and 22 are provided for extracting noise frequency components. Then, as shown in FIG. 8 when viewed from the observation point, if the speaker is located in the 0 ° direction and should be monitored in the angle range of 0 ° ± θ, the voice frequency component of the speaker is extracted. Change range φ of the first beam former 13 provided for
1, that is, the change range in steps of 1 ° in the direction of increasing the sensitivity in the adaptive filter is set to −θ <φ1 <θ at most, and this range is used for filtering. In this case, of the second and third beam formers 16 and 22 provided to extract the noise frequency component, the change range φ2 of the second beam former 16 is −180 ° + θ <φ2 <−θ, and The change range φ3 of the beam former 22 of No. 3 is set to θ <φ3 <180 ° −θ. However, 180 ° indicates a facing position of 0 ° through the center point, − indicates counterclockwise rotation in the figure when viewed from the 0 ° position, and + indicates clockwise rotation.

Therefore, in this way, the second beam former 16 and the third beam former 22 track noises coming from different ranges with the target sound arrival range φ1 in between. Therefore, even if the noise source in the range of φ2 suddenly moves to the range of φ3 across the range of φ1, the third beamformer 22 having the region of φ3 as its place moves the noise source. Can be immediately captured, so that the direction of noise is never lost.

In the case of this configuration, the second beam former 1
A total of two outputs, that is, the output of No. 6 and the output of the third beamformer beam 22 are obtained as the output of noise. Based on the result of the noise direction estimation unit 17, the effective noise determination unit 24 outputs the second beam. It is determined which of the former 16 and the third beam former 22 is effectively tracking the noise, and based on this determination result, the output of the one which is effectively tracked is used as the noise component.

[Overall Processing Flow in Speaker Tracking Microphone Array 10 of Configuration Example 2] FIG. 9 shows the overall processing flow described above. This process is performed for each frame. After setting the change range and the initial value of the input direction of each beamformer (step S31), the process of the first beamformer 13 is performed (step S32), and after estimating the noise direction (step S33), the noise direction As an input, the effective noise determination unit 24 determines whether the noise direction is φ2 or φ3, and determines which of the second beamformer 16 and the third beamformer 22 to select. (Step S34).

Then, the estimated noise direction is sent to either the second input direction modification unit 15 or the third input direction modification unit 21, the noise direction is modified, and the processing of the selected beam former is executed. To be done.

That is, if the estimated noise direction is in the region of φ2, the noise direction is sent to the second input direction correction section 15, the noise direction is corrected, and the processing of the second beam former 16 is executed. The target sound direction is estimated (step S
34, S35, S36, S37).

If the estimated noise direction is in the area of φ3, the noise direction is sent to the third input direction correction unit 21,
The noise direction is corrected, the processing of the third beam former 22 is executed, and the target sound direction is estimated (step S3).
4, S38, S39, S40, S41).

Next, it is judged whether or not the voice direction (target sound direction) estimated by the selected beamformer is within the range of φ1, and if it is within the range, the estimated voice direction is determined by the first beamformer 13. Is sent to the first input direction correction unit 14 and the input direction is corrected (step S4).
2, S43). If it is out of the range, the correction process is not executed,
The process proceeds to the next frame (steps S42, S
31).

This processing is performed for each frame, and noise suppression is performed while tracking the voice and noise directions.

As described above, in this example, the voice inputting means for receiving the voice uttered by the speaker at at least two different positions and the frequency analysis for each channel of the voice signal corresponding to the sound receiving position. Frequency analysis means for performing frequency output of a plurality of channels and adaptation of the frequency components of the plurality of channels obtained by the frequency analysis means using a filter coefficient calculated so that sensitivity outside the desired direction becomes low. A first beamformer processing unit that obtains a target voice component by performing an incoming noise suppression process that suppresses voices other than the voice from the speaker direction by performing a filter process, and the plurality of units obtained by the frequency analysis unit. By applying adaptive filter processing to the frequency components of the channel, using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. Suppressing the voice from the speaker direction,
The second beamformer processing means for obtaining the first noise component and the filter coefficient calculated for the frequency components of the plurality of channels obtained by the frequency analysis means so that the sensitivity outside the desired direction becomes low. Second beamformer processing means for suppressing the voice from the speaker direction by performing adaptive filter processing and obtaining a second noise component;
A noise direction estimating means for estimating a noise direction from the filter coefficient calculated by the first beamformer processing means, and a first target sound direction is estimated from a filter coefficient calculated by the second beamformer processing means. First target sound direction estimating means, second target sound direction estimating means for estimating a second target sound direction from the filter coefficient calculated by the third adaptive beam former processing means, and the first beam The first input direction, which is the arrival direction of the target sound to be input in the former, is defined by the first target sound direction estimated by the first target sound direction estimating means and the second target sound direction estimating means. First input direction correction means for sequentially correcting based on either or both of the estimated second target sound directions,
When the noise direction estimated by the noise direction correction means is within a predetermined first range, the second input direction, which is the direction of arrival of noise to be input in the second beamformer, is set to the noise direction. Second input direction correcting means for sequentially correcting the noise on the basis of the noise and the noise direction estimated by the noise direction correcting means is within a predetermined second range. Third input direction correcting means for sequentially correcting a third input direction which is an arrival direction based on the noise direction, and whether the noise direction estimated by the noise direction estimating means has come from a predetermined first range. Based on whether it comes from a predetermined second range, one of the first output noise and the second output noise is determined to be a true noise output, and either one of the noises is output, and at the same time,
Effective noise for determining which estimation result of the first voice direction estimation means and the second voice direction estimation means is valid and outputting either one of the voice direction estimation results to the first input direction correction means. And a determining means.

In such a configuration, the voice input means receives the voice uttered by the speaker at two or more different positions, and the frequency analysis means receives the voice signal corresponding to the sound receiving position. The frequency analysis is performed for each channel and the frequency components of a plurality of channels are output. Then, the first beamformer processing means performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. The incoming beam noise suppressing process for suppressing a voice other than the voice from the speaker direction is performed to obtain a target voice component, and the second beamformer processing means
The frequency components of the plurality of channels obtained by the frequency analysis means are subjected to adaptive filter processing using filter coefficients calculated so that sensitivity outside the desired direction becomes low, thereby suppressing voice from the speaker direction. Then, the noise component is obtained. The noise direction estimating means estimates the noise direction from the filter coefficient calculated by the first beamformer processing means, and the target sound direction estimating means calculates the noise direction by the filter coefficient calculated by the second beamformer processing means. The target sound direction is estimated from. The first target sound direction estimating means estimates the first target sound direction from the filter coefficient calculated by the second beamformer processing means, and the second target sound direction estimating means calculates the third target sound direction. The second target sound direction is estimated from the filter coefficient calculated by the adaptive beam former processing means.

The first input direction correcting means estimates the first input direction, which is the arrival direction of the target sound to be input in the first beam former, by the first target sound direction estimating means. Sequential correction is performed based on either or both of the first target sound direction thus determined and the second target sound direction estimated by the second target sound direction estimating means. When the noise direction estimated by the noise direction correcting unit is within a predetermined first range, the second input direction correcting unit determines the arrival direction of noise to be input in the second beamformer. A certain second input direction is sequentially corrected on the basis of the noise direction, and the third input direction correction means, when the noise direction estimated by the noise direction correction means is within a predetermined second range, The third input direction, which is the arrival direction of noise to be input in the third beam former, is sequentially corrected based on the noise direction.

Therefore, the second beam former whose second input direction is corrected by the output of the second input direction correcting means suppresses components coming from other than the second input direction and extracts the remaining noise components. In addition, the third beam former whose third input direction is corrected by the output of the third input direction correcting means suppresses the components arriving from other than the third input direction and remaining noise components. Will be extracted.

Then, the effective noise determining means determines whether the noise direction estimated by the noise direction estimating means comes from the predetermined first range or the predetermined second range. Either one of the output noise and the second output noise is determined to be a true noise output, and either one of the noises is output, and at the same time, one of the first voice direction estimating means and the second voice direction estimating means is output. It is determined whether the estimation result is valid and the valid voice direction estimation result is output to the first input direction correcting means.

As a result, the target sound direction correction means obtains the first input direction, which is the arrival direction of the target sound to be input in the first beam former, by the determined target sound direction estimation means. The first beamformer suppresses the noise components coming from other than the first input direction and extracts the speaker's voice component with low noise, because the correction is sequentially performed based on the target sound direction.

As described above, the present system can separately obtain the voice frequency component in which the noise component is suppressed and the noise frequency component in which the voice component is suppressed. The greatest feature of the present invention is that the first to third The point is that a beamformer that operates in the frequency domain is used as the beamformer. And by doing so, the amount of calculation can be significantly reduced.

According to the present invention, the processing amount of the adaptive filter is significantly reduced, and frequency analysis processing other than the frequency analysis on the input voice can be omitted, and it is necessary for the filter calculation. The conversion processing from the time domain to the frequency domain is also unnecessary, and the overall calculation amount can be significantly reduced.

Further, according to the present invention, a noise tracking beamformer having completely different monitoring areas for noise tracking is provided, and the voice direction is estimated from each output, and which is effective from each estimation result. It is determined whether or not the noise tracking is performed properly, and the result of estimation of the voice direction by the filter coefficient of the beamformer that is determined to be effective is given to the first target sound direction correction means to correct the first target sound direction. The means sequentially corrects the arrival direction of the target sound as the input target in the first beam former, based on the target sound direction estimated by the target sound direction estimation means. The beamformer No. 1 can suppress the noise component coming from other than the first input direction and extract the speaker's voice component with low noise, and even if the noise source moves, this is lost. In which it is possible to suppress and tracking without.

In the prior art, 2ch, that is,
In order to enable tracking of the target sound source with only two microphones, one beamformer for noise tracking is used separately from the beamformer for noise suppression. For example, the noise source has moved across the direction of the target sound. In such a case, noise tracking accuracy may decrease.

However, according to the present invention, since a plurality of beam formers for tracking noise are used to take charge of separate tracking ranges, it is possible to prevent the tracking accuracy from deteriorating even in the above case.

The speaker tracking microphone array 10 of the configuration examples 1 and 2 described above, while reducing the calculation load,
This is an example that mainly suppresses noise having a direction. These are suitable for use in environments where the sound source of the speaker is known, such as in video conferencing systems, and where there is little environmental noise, but the levels and characteristics are mixed and diverse. There is a possibility that it may not be enough to be used outdoors where it is affected by various noises, or in a store or a station where a large number of people gather.

Therefore, it is desired to effectively suppress non-directional background noise as well, but as will be described next, the spectral subtraction (SS) is performed.
A processing function may be added.

That is, the directional noise is suppressed by the beam former, and the non-directional background noise is suppressed by the spectral subtraction (SS) process. To that end, in the latter stage of the system having the configuration of FIG. 3 or FIG.
Further, the spectral subtraction (S
S) The processing unit 30 is connected.

The spectrum subtraction (SS) processing unit 30 includes a voice band power calculation unit 31, as shown in FIG.
It is composed of a noise band power calculation unit 32, a band weight calculation unit 33, and a spectrum subtraction unit 34.

Of these, the voice band power calculation unit 31
Is for dividing the voice frequency obtained by the beamformer 13 into frequency bands and calculating the voice power for each band. The noise band power calculator 32 is configured to calculate the noise obtained by the beamformer 16. The frequency components (or the noise frequency components respectively obtained by the beam formers 16 and 22 and selected by the effective noise determination unit 24 and output) are divided into frequency bands to calculate noise power for each band. .

The band weight calculation unit 33 uses, for each band k, the obtained average band power Pv (k) of voice and average band power Pn (k) of noise, and the band weight coefficient W (k) for each band. The corrected spectrum subtraction unit 34 calculates the voice signal based on the input band power calculated by the input band power calculation unit 31 and the voice band power calculated by the voice band power calculation unit 31. The background noise is suppressed by weighting each frequency band.

Both the voice frequency component used in the voice band power calculation unit 31 and the noise frequency component used in the noise band power calculation unit 32 are either in Example 0A1 or Example 0A.
The target speech component and the noise component, which are the two outputs of the two beam formers, are used. Then, in general, a noise suppression process known as spectrum subtraction (SS) suppresses a background noise component having no directivity.

Generally performed spectral subtraction (SS) is a one-channel microphone (that is,
Since one microphone) is used and the power of noise is estimated from the output of this microphone in the section where there is no voice, it is not possible to deal with the case where non-stationary noise is superimposed on the voice.

Also, when two-channel microphones (that is, two microphones) are used and one is used for noise collection and one is used for noise-superimposed voice collection, both microphones must be installed at different locations. As a result, the noise superimposed on the voice and the noise captured by the noise collecting microphone are out of phase, and the effect of improving noise suppression is not significantly improved even if spectral subtraction is performed.

In this embodiment, since a beamformer for extracting a noise component is prepared and the output of this beamformer is used, the phase shift is corrected as described in the configuration examples 1 and 2. Even in the case of non-stationary noise, highly accurate spectrum subtraction (SS) can be realized.

Further, since the output of the beamformer in the frequency domain is used, frequency analysis can be omitted and spectral subtraction can be performed, and non-stationary noise can be suppressed with a smaller amount of calculation than in the past.

A specific spectral subtraction (SS) method will be described below.

<Principle of Spectral Subtraction (SS)> First, the principle of spectrum subtraction will be described.

If the output of the target speech beamformer (first beamformer 13) is Pv and the output of the noise beamformer (second or third beamformer 16 or 22) is Pn, then Pv = V + B 'Pn = N + B ″, where V is the power of the voice component,
B ′ is the power of the background noise included in the voice output, and N ′
Is the power of the noise source component, and B ″ is the power of the background noise included in the noise output. Of these, the background noise component included in the voice output component is suppressed by the spectral subtraction process.

If B ′ in the voice output component is equal to B ″ in the noise output component and the power N of the noise source component is smaller than the power V of the voice component, consider B ′ = Pn. And the weighting coefficient W for spectral subtraction (SS) processing can be obtained as follows.
That is, W is W = (Pv-Pn) / Pv - V / (V + B '), and the voice component can be approximately obtained as V - Pv * W.

Such processing is performed by the spectrum subtraction processing unit 30. A specific example of the spectrum subtraction processing unit 30 will be described below.

<Example of Configuration of Spectrum Subtraction Processing Unit> FIG. 10 shows the configuration required for the spectrum subtraction (SS) process, and FIG. 11 shows the spectrum subtraction process procedure.

As shown in FIG. 10, the spectrum subtraction processing unit 30 is composed of a voice band power calculation unit 31, a noise band power calculation unit 32, a band weight calculation unit 33, a spectrum subtraction unit 34, and a control unit 35. There is.

Of these, the voice band power calculation unit 31
Calculates the voice band power using the voice frequency component that is the output from the first beamformer 13, averages the calculated power values in the time direction, and outputs the average power (the average band power of the voice) for each band. Pv (k))
The noise band power calculation unit 32 calculates the noise band power by using the noise frequency component output from the second beam former 16 (or the third beam former 22). At the same time, the calculated power values are averaged in the time direction, the average power (average band power Pn (k) of noise) is obtained for each band, and given to the band weight calculation unit 33.

Further, the band weight calculation unit 33 uses the obtained average band power Pv (k) of voice and the average band power Pn (k) of noise for each band k, and uses the band weight coefficient W for each band.
(K) is calculated, and the spectrum subtraction unit 34
Suppresses the noise component by weighting the voice frequency component Pv (k) input from the first beamformer 13 using the weighting coefficient W (k) for each band calculated by the band weight calculation unit 33. The calculated voice frequency component Pv (k) 'is obtained.

The control unit 35 receives the signal from the spectrum subtraction control unit 50 and, in accordance with the signal type,
It controls the band weight calculation unit 33, and when the signal from the spectrum subtraction control unit 50 is “0”, a control command is given to the band weight calculation unit 33 so as to generate the minimum weight, and the band weight calculation is performed. The output of the unit 33 is controlled to have the minimum weight, and when the signals from the spectrum subtraction control unit 50 are "1" and "2",
The control unit 35 controls the band weight calculation unit 33 so as to obtain a normal weight coefficient.

When the control section 35 receives a signal from the spectrum subtraction control section 50, the control section 35 controls the band weight calculation section 33 and the spectrum subtraction section 34 according to the signal type from the spectrum subtraction control section 50. When the signal from the spectrum subtraction control unit 50 is “0”, a control command is given to the band weight calculation unit 33 to generate the minimum weight, and the output of the band weight calculation unit 33 becomes the minimum weight. Then, the spectrum subtraction unit 34 is controlled to cut the output signal by causing the spectrum subtraction unit 34 to perform a calculation for applying the minimum weight to the audio frequency component output from the first beam former 13, and from the spectrum subtraction control unit 50. When the signal of “1” is “1”, the control unit 35 considers that the sudden noise is superposed on the voice section, and the second The spectrum subtraction unit 34 is controlled to perform a two-channel spectrum subtraction process in which the output from the mu-former 16 is treated as a noise component, whereby the spectrum subtraction unit 34 treats the output from the second beam former 16 as a noise component. The signal from the spectrum subtraction control unit 50 is controlled to "2" by controlling the spectrum subtraction processing of the channel.
In the case of, the control unit 35 regards the output of the first beamformer 13 as one-channel spectrum subtraction, assuming that it is a voice-only section.
4 to control the spectrum subtraction unit 34 to perform one-channel spectrum subtraction on the output of the first beam former 13.

The spectrum subtraction control unit 5
When the signal from 0 is "1" and "2", the band weight calculation unit 33 causes the band weight calculator 33 to calculate the average band power Pv (k) of the obtained voice and the average band power Pn ( k), a band weighting coefficient W (k) for each band is calculated,
Only when the signal from the spectrum subtraction control unit 50 is "0", the control unit 35 is configured to control so as to minimize the band weighting coefficient W (k).

An audio frequency component and a noise frequency component are obtained as outputs from the two beam formers 13 and 15 (or 22). The voice band power calculation is performed using the voice frequency component output from the first beam former 13 (step S51), and the noise band power is calculated using the noise frequency component output from the beam former 15 (or 22). Calculation is performed (step S52).

The power calculation here uses the voice frequency component and the noise frequency component of the above-described system of the present invention. Since these processes the beamformer in the frequency domain, there is no need for frequency analysis. As it is, the power can be calculated for each band of the frequency components of voice and noise.

Next, the calculated power values are averaged in the time direction to obtain the average power for each band (step S5).
3). The band weight calculator 33 uses the obtained average band power Pv (k) of voice and the average band power Pn (k) of noise for each band k, and uses the following formula to calculate the band weight coefficient W for each band.
Calculate (k).

W (k) = (Pv (k) -Pn (k)) / P
v (k) (when Pv (k)> Pn (k)) W (k) = Wmin (when Pv (k) <= Pn (k)) The band weight has the maximum value 1.0 and the minimum value Wmin. The value of Wmin is set to, for example, "0.01".

Next, the spectrum subtraction unit 34 uses the weighting coefficient W (k) for each band calculated by the band weight calculation unit 33 to weight the input voice frequency component Pv (k) and suppress the noise component. The voice frequency component Pv (k) 'is calculated (step S54).

Pv (k) '= Pv (k) * W (k) Thus, the background noise having no direction is suppressed by the spectral subtraction (SS) process, and the noise having a direction is suppressed by the beam former described above. As a result, highly accurate noise suppression becomes possible.

In the spectrum subtraction processing unit 30, the voice frequency component given from the first beam former 13 of the speaker tracking microphone array 10 and the second beam former 16 (or the third beam former 22) are used.
The noise frequency component given by the above is basically received, and the noise compression is basically performed by such a process. However, by performing the required noise compression process in accordance with the three kinds of signals output from the spectrum subtraction control unit 50, It also enables suppression of sudden noise.

That is, in the spectrum subtraction processing unit 30, the spectrum subtraction control unit 50.
When the signal is "0", it can be regarded as a noise zone. Therefore, the control unit 35 causes the band weight calculation unit 33 to output the minimum weight coefficient, and uses this to cause the spectrum subtraction unit 34 to output the voice frequency. The output signal from the spectral subtraction processing unit 30 is cut by performing the calculation process on the component.

Further, when the signal from the spectrum subtraction control unit 50 is "1", it is considered that sudden noise is superposed on the voice section, and the control unit 35 causes the band weight calculation unit 33 to calculate a normal weight coefficient. The obtained weighting coefficient is given to the spectrum subtraction unit 34, and the spectrum subtraction unit 34 is used to perform a two-channel spectrum subtraction process in which the output from the second beam former 16 is treated as a noise component. Let

Further, in the spectrum subtraction processing unit 30, when the signal from the spectrum subtraction control unit 50 is “2”, it is considered as a voice-only section, and the control unit 35 causes the band weight calculation unit 33 to use the normal weighting factor. The weighting factor thus obtained is calculated and given to the spectrum subtraction unit 34, and the spectrum subtraction unit 34 is controlled using this so as to perform one-channel spectrum subtraction on the output of the first beamformer 13. Accordingly, the spectrum subtraction unit 34 causes the first beam former 13 to
1-channel spectral subtraction is performed on the output of the above, and output as the output of the audio frequency component.

In the case of this configuration, the voice band power calculation means divides the obtained spectrum component of the voice frequency into frequency bands to calculate the voice power for each band, and the noise band power calculation means obtains the obtained voice power. The spectral component of the obtained noise frequency is divided for each frequency band, and the noise power for each band is calculated. Then, the spectrum subtracting means suppresses the background noise by weighting each frequency band of the voice signal based on the frequency band power of the voice and noise obtained from the voice band power calculating means and the noise band power calculating means.

According to this structure, the nondirectional noise (background noise) that cannot be suppressed by the beamformer uses the target voice component and noise component that can be obtained by the beamformer of the system of the present invention, and uses this for spectral subtraction. It is suppressed by processing. That is, this system is provided with two beam formers for extracting a target voice component and a noise component as beam formers. The target voice component and the noise component, which are outputs of these beam formers, are used to perform spectral subtraction. By processing, background noise components having no directionality are suppressed. Spectral subtraction (SS) processing is known as noise suppression processing, but generally performed spectral subtraction (SS) processing uses a one-channel microphone (that is, one microphone), and the output of this microphone is used to output audio. Since the noise power is estimated in a non-existing section, it cannot be dealt with when non-stationary noise is superimposed on the speech. In addition, when two-channel microphones (that is, two microphones) are used and one is used for noise collection and one is used for noise-superimposed voice collection, it is necessary to separate the installation locations of both microphones. ,
The noise superimposed on the voice and the noise captured by the noise collecting microphone are out of phase with each other, and the effect of improving noise suppression is not significantly improved even if the spectral subtraction process is performed.

However, in the present invention, since the beam former for extracting the noise component is prepared and the output of this beam former is used, the phase shift is corrected. Therefore, even in the case of non-stationary noise, the accuracy is high. It is possible to realize various spectrum subtraction processing. Furthermore, since the output of the beamformer in the frequency domain is used, frequency analysis can be omitted and spectral subtraction can be performed, and non-stationary noise can be suppressed with a smaller amount of calculation than before.

<Example of Configuration of Directionality Detection Unit 40> Next, an example of the configuration of the directionality detection unit 40, which is an important component in the present system, will be described.

FIG. 12 is a block diagram showing the basic structure of the directionality detecting section 40. In FIG. 12, reference numeral 1 is a counter-directional microphone, 2 is a frequency analysis unit, and these are components of the speaker tracking microphone array 10. 41
Is a phase difference equivalent amount calculation unit, 42 is an inter-channel power ratio calculation unit, and 43 is a directionality calculation unit. The phase difference equivalent amount calculation unit 41, the inter-channel power ratio calculation unit 42, and the directionality calculation unit 43 The sex detector 40 is configured.

The facing directional microphone 1 is a two-channel (2ch) microphone in which two directional microphones are arranged with their axes inclined as described above.
The frequency analysis unit 12 receives two channels of the audio signal from the facing directional microphone 1 and calculates frequency components for each channel by, for example, fast Fourier transform (FFT), and the phase difference equivalent amount. The calculation unit 41 calculates the phase difference equivalent amount T corresponding to the phase difference between the two channels from the frequency component of each channel calculated by the frequency analysis unit 12.

The inter-channel power ratio calculation unit 42 sums the data of the frequency components of each channel calculated by the frequency analysis unit 12 for each channel to obtain the total power of all frequency components for each channel, and obtains the magnitude. Using the value of the magnitude of the total power for each channel, two types of ratios of the values between the two channels (one type for the other type and the other type for the one type) are obtained, and the larger one of them is set as the power ratio R. It is a thing.

Further, the directivity calculating section 43 has a phase difference corresponding amount T calculated by the phase difference corresponding amount calculating section 41 and a power ratio calculating section 42.
This is for multiplying the two by using the power ratio R obtained in 1. and obtaining the result as the directionality index D.

Next, the operation of the directionality detecting section 40 having the above structure will be described.

In the present invention, a noise suppression processing technique using a microphone of two or more channels is realized. However, for simplification, the case of a two-channel microphone having the most basic configuration will be described as an example.

FIG. 13 shows the flow of processing. Explaining with reference to FIG. 13, first, the voice input from the counter-directional microphone is set by installing the voice input unit 11 by the counter-directional microphone in which two directional microphones are arranged with their axes inclined. , Frequency analysis unit 1
2, and the frequency component is calculated for each channel by, for example, fast Fourier transform (FFT).

In the directionality detecting section 40, the calculated 2
When the data of the frequency components of the channels (ch1, ch2) is received, the minute phase difference equivalent amount calculation unit 41 calculates the amount T corresponding to the phase difference between the two channels from the received frequency component data of each channel. To do.

The method of obtaining the phase difference equivalent amount T is, for example, first of all, a cross spectrum W between two channels (ch1, ch2).
xy is obtained, and the imaginary number component of Wxy is expressed as an absolute value, and the sum is calculated for all frequency components as shown in equation (1).

S = Σ {Real (Wxy), | Imag (Wxy) |} (1) Next, as shown in equation (2), the phase T of S is the arctangent function.
Is obtained and this is taken as the phase difference equivalent amount.

T = atan2 (Imag (S), Real (S)) (2) Note that although all frequency components are used here, it is not necessary to limit to this, and only frequency components larger than the average power are used, for example. May be used.

On the other hand, the inter-channel power ratio calculation unit 42
Then, using the data of the frequency components obtained by the frequency analysis unit 12, the total power of all frequency components for each channel is obtained by the equation (3).

Px = ΣX, Py = ΣY (3) (X and Y are power components of each frequency band of each channel) Then, the larger value of Px / Py and Py / Px is set as the power ratio R. This is given to the directionality calculation unit 43.

In the directionality calculation unit 43, for example, the phase difference equivalent amount T calculated by the phase difference equivalent amount calculation unit 41 and the power ratio R calculated by the inter-channel power ratio calculation unit 42 are used.
The directionality index D is obtained as in Expression (4).

D = T * R (4) Further, instead of the equation (4), it is possible to obtain the directionality index D by a weighted sum like the equation (5).

D = a * T + b * R (a and b are constants) (5) In short, the directionality detection unit 40 is not limited to the information of the phase difference equivalent amount T obtained based on the frequency analysis result, and Direction ratio D using the power ratio R of
Is the point of the present invention.

When the directional index D is obtained, the power ratio R is introduced to take the amplitude difference of the waveform into consideration, and when a signal arrives from the front and exactly the same signal is output between the two channels. The directionality index D has a value of "0"
If it is not, the value is large, and it is possible to judge whether the arrival direction is the front or not depending on the magnitude of the value of the directionality index D.

The directionality detecting section 40 performs this process at short fixed time intervals such as 8 [msec]. The fixed time will be referred to as a frame hereinafter.

Therefore, for example, by repeating such processing at a frame period of 8 [msec], it is possible to obtain the directionality index D of the incoming voice that changes from moment to moment.

<Another Example of Directionality Detection Unit 40> The directionality detection unit 40 can also be realized by the configuration as shown in FIG. In FIG. 14, 11 is the same counter-directional microphone as described above, 12 is a frequency analysis unit, 44 is a spectrum normalization unit, and 45 is an inter-channel spectrum difference calculation unit.

Of these, the facing directional microphone 1
Is the same as that described above, and is a two-channel (2c) in which two directional microphones are arranged with their axes inclined to each other.
h) A microphone. Further, the frequency analysis unit 2 receives two channels of the audio signal from the counter directional microphone 1, and, for example, a fast Fourier transform (FF) is provided for each channel.
T) or the like to calculate the frequency component, and the spectrum normalization unit 44 receives the frequency component data from the frequency analysis unit 12 and normalizes the frequency component data for each channel. Is also
The inter-channel spectrum difference calculation unit 45 calculates the inter-channel spectrum difference from the normalized frequency component.

That is, this is an example in which the spectrum normalization unit 44 and the inter-channel spectrum difference calculation unit 45 constitute the direction detection unit 40.

In this example, in order to reduce the calculation amount, the phase difference equivalent amount T is not directly calculated, but the directionality index D indirectly considering the phase difference is calculated using only the amplitude difference.

FIG. 15 shows the flow of processing.

First, the sound of 2 channels input from the counter directional microphone 11 in which the two directional microphones are arranged with their axes inclined is sent to the frequency analysis unit 12 and, for example, a fast Fourier transform (FFT) is performed for each channel. ) Etc., the frequency component is calculated. And this frequency analysis unit 1
The frequency component for each channel obtained in 2 is input to the spectrum normalization unit 44.

Next, the spectrum normalizing section 44 finds the square root Rx (equation (6)) of the total power of the frequency components of one of the input frequency components of each channel, and obtains this. Rx normalizes the frequency components of both channels (equation (7)).

Rx = √ (ΣX) (6) X ′ (k) = X (k) / Rx Y ′ (k) = Y (k) / Rx (7) Next, this normalized channel The frequency component of is input to the inter-channel spectrum difference calculation unit 22, and the inter-channel spectrum difference calculation unit 45 calculates the power of the normalized difference between the spectra by the formula (8), and this is calculated as the directionality index D. And

D = Σ | X ′ (k) −Y ′ (k) | 2 (8) As a result, when a signal arrives from the front and the same signal is output between the two channels, “0” is output. Otherwise, a large value is obtained, and it can be determined from the magnitude of this value whether or not the arrival direction is the front.

The spectrum normalization is not limited to the above example, and for example, the sum of the absolute values of the spectra of two channels may be used, or the absolute value of the sum of the spectra of two channels may be used. Also, of the two channels,
You may use the larger absolute value. Further, each frequency component may be normalized by the absolute value of the larger total power of the frequency components of the two channels.

By repeating this processing, for example, in a frame cycle of 8 [msec], it is possible to obtain the directionality index D of the incoming voice which changes from moment to moment.

(Specific Configuration Example 1 of Noise Suppression Processing Apparatus of the Present Invention) The basic configuration example shown in FIG. 1 includes the speaker tracking microphone array 10, the spectrum subtraction processing unit 30, the direction detection unit 40, and the configuration described above. It is a noise suppression processing apparatus of the present invention as an example realized by using the spectrum subtraction control unit 50.

The operation of this noise suppression processing device will be described.
The basic configuration of the noise suppressing device according to the present invention shown in FIG.
As the speaker tracking microphone array 10, the configuration of FIG.
When the configuration of FIG. 10 is adopted as the spectrum subtraction processing unit 30 and the configuration of FIG. 12 or FIG. 14 is adopted as the direction detecting unit 40, it becomes as shown in FIG.

This system is characterized in that microphones for at least two channels having directivity are arranged as audio input sections with their axial directions inclined to each other, and audio signals obtained by these microphones are frequency-analyzed for each channel. By performing adaptive beamformer processing using the filter coefficient calculated so that sensitivity outside the desired direction becomes low,
By using a noise suppression processing device that suppresses speech from the speaker direction to obtain a noise component, and performs processing to suppress this noise component to obtain a speaker speech component with less noise,
A speaker voice component and a noise component in which noise is suppressed are obtained (FIG. 17).
Step S131) of the above, and a directionality detection unit based on short-time frequency analysis is added to this noise suppression processing device to prevent the arrival of sudden noise or high-speed moving sound source that cannot be suppressed by the beamformer processing. Spectral subtraction performed by the detection unit (step S132 in FIG. 17) and using the detection result and the speaker voice component and noise component obtained by the noise suppression processing device are controlled (FIG. 17).
By performing steps S133 and S134), it is possible to suppress a sudden noise, a high-speed moving sound source, and the like while securing a speaker tracking function capable of setting the allowable range in the speaker direction with high accuracy.

That is, the noise suppression processing device of FIG.
The voice input from the voice input unit 11 having two microphones is sent to the frequency analysis unit 12, and the frequency component is calculated by, for example, fast Fourier transform (FFT).
Then, the obtained data of the frequency component for each channel is given to the first and second beam formers 13 and 16 and the directionality detector 40.

In the first beam former 13, noise is suppressed from the frequency components corresponding to the two channels input from the frequency analysis unit 12 by the adaptive filter in the frequency domain, and the frequency component in the direction of the target sound is output (voice frequency Component output). Here, the operation of adjusting the phase by the first input direction correction unit 14 is performed using the output from the target sound direction estimation unit 18 so that the direction of the target sound is in front of the microphone.

In the second beam former 16, the target sound is suppressed from the frequency components corresponding to the two channels input from the frequency analysis unit 12 by the frequency domain adaptive filter, and the frequency component in the noise direction is output ( Noise frequency component output). Here, assuming that the direction of the noise is in front of the microphones, the second input direction correction unit 15 uses the output from the noise direction estimation unit 17 so that it can be considered that the noises have arrived at the two microphones at the same time. Perform the operation to adjust the phase (phasing).

Here, the noise direction estimation unit 17 estimates the noise direction from the adaptive filter of the first beam former 13, and the target sound direction estimation unit 18 estimates the noise direction from the adaptive filter of the second beam former 16. To estimate. These processes are performed at fixed time intervals such as 8 [msec].

Next, the direction detection section 40 and the spectrum subtraction control section 50, which are one of the important elements of the system of the present invention, will be described.

As described above, the direction detecting section 40 uses not only the phase difference T between the two microphones but also the power ratio R of the input signal of each channel, based on the frequency analysis such as the short-time FFT, to determine the directionality index. Calculate D. And
The obtained directionality index D is given to the spectrum subtraction control unit 50.

Spectral subtraction controller 50
Is the target sound direction (speaker direction) based on the obtained directionality index D and the target sound direction information output from the target sound direction estimation unit 18.
Based on and, three kinds of signals (“0”, “1”,
Either "2") is generated and sent to the spectrum subtraction processing unit 30.

Here, of the three types of signals, the signal "0"
Indicates that the section is almost only noise, the signal “1” indicates that a large sudden noise is superimposed on the speech section, and the signal “2” indicates that the section is almost only speech. It represents.

This is obtained as follows. First, the allowable range in the speaker direction is set from the center of the two microphones and the threshold value of the directionality index D is set to "some". For example, the allowable range in the speaker direction is set to ± 20 ° from the center of the two microphones, and the threshold value of the directionality index D is set to "1.0".

Then, it is determined whether or not the directionality index D sent from the directionality detection section 40 is less than or equal to the threshold value (1.0). If the result is less than or equal to the threshold value, then Based on the target sound direction information output from the target sound direction estimation unit 18, it is determined whether the target sound direction (speaker direction) is within the set range,
If it is within the set range, it means that it is a section of almost only voice, so a signal "2" is generated and sent to the spectrum subtraction processing unit 30. Further, if it is outside the set range, it means that it is a section including almost no noise, and therefore a signal “0” is generated and sent to the spectrum subtraction processing unit 30.

If the directionality index D is equal to or greater than the threshold value and the target sound direction is within the set range, it means that a large sudden noise is superposed on the voice section, and accordingly, , It is determined that sudden noise is superposed on the voice coming from the speaker direction, a signal "1" is generated and sent to the spectrum subtraction processing unit 30, and the directionality index D
Is greater than or equal to the threshold value and the target sound direction is outside the set range, it means that it is an interval of almost only noise, and therefore a signal “0” is generated and sent to the spectrum subtraction processing unit 30.

In this way, the spectral subtraction control unit 50 is based on the target sound direction (speaker direction) estimated by the target sound direction estimation unit 18 and the directionality index D calculated by the directionality detection unit 40. , Three kinds of signals (“0”,
Either “1” or “2”) is sent to the spectrum subtraction processing unit 30.

In the spectral subtraction processing unit 30, the voice frequency component given from the first beam former 13 of the speaker tracking microphone array 10 and the second beam former 16 (or the third beam former 22) are included.
And a noise frequency component given by the above, and performs necessary noise compression processing in accordance with the three types of signals output from the spectrum subtraction control unit 50.

The spectrum subtraction processing unit 30 receives the signal from the spectrum subtraction control unit 50, and when the signal is "0", it can be regarded almost as a noise section. Therefore, the output signal is cut by applying the minimum weight. In addition, the spectrum subtraction control unit 50
When the signal from 1 is “1”, it is considered that sudden noise is superposed on the voice section, and the two-channel spectral subtraction process is performed in which the output from the second beam former 16 is treated as a noise component.

That is, when the signal received from the spectrum subtraction control unit 50 is "0", it can be regarded almost as a noise section. Therefore, in the spectrum subtraction processing unit 30, the control unit 35 causes the band weight calculation unit 33 to generate the minimum weight. The band weight calculation unit 33 generates the minimum weight and gives it to the spectrum subtraction unit 34, so that the spectrum subtraction unit 3
Reference numeral 4 cuts the output signal by performing a calculation for applying the minimum weight to the output of the audio frequency component and outputting the result.

When the signal from the spectrum subtraction control unit 50 is "1", it can be considered that sudden noise is superposed on the voice section. A control command is given to the calculation unit 33 so as to calculate and output a normal weighting coefficient, whereby the band weight calculation unit 33 causes the voice band power calculation unit 31 to operate.
Based on the output of the noise band power calculator 32, the band weight coefficient is obtained.

Then, the band weight calculation unit 33 gives the obtained band weight coefficient to the spectrum subtraction unit 34, so that the spectrum subtraction unit 34 assigns the weight to the output of the audio frequency component from the first beam former 13. By performing the multiplication calculation and outputting the result, the two-channel spectral subtraction processing that treats the output from the second beamformer 16 as a noise component is performed, and the spectral subtraction processing result is subjected to the noise suppression processing. It can be output as a sound frequency component that has already been processed.

Further, when the signal from the spectrum subtraction control unit 50 is "2", it can be regarded as a voice only section. Therefore, in the spectrum subtraction processing unit 30, the control unit 35 receiving this signal receives the band weight. A control command is given to the calculation unit 33 to calculate and output a normal weighting coefficient, whereby the band weight calculation unit 33 determines the band weight based on the outputs of the voice band power calculation unit 31 and the noise band power calculation unit 32. Find the coefficient.

Then, the band weight calculation unit 33 gives the obtained band weight coefficient to the spectrum subtraction unit 34, so that the spectrum subtraction unit 34 assigns the weight to the output of the audio frequency component from the first beam former 13. By performing the multiplication calculation and outputting the result, the result of the state in which the 1-channel spectral subtraction is performed on the output of the first beamformer 13 is obtained, and this is output as the output of the audio frequency component.

As another control method, when the signal is "1", since a large sudden noise is superposed on the voice, it may be regarded as a noise section and may be processed in the same manner as the signal "0".

This apparatus can extract only a voice component with less distortion by suppressing a noise component having a direction and a noise component having no direction, and also for a sudden noise,
It is possible to extract a voice component with less distortion that suppresses a noise component.

(Specific Configuration Example 2 of Noise Suppression Processing Apparatus of the Present Invention) This specific configuration example is as shown in FIG. 18, and in this case, the speaker tracking microphone array 10 is shown in FIG.
10 is used as the spectrum subtraction processing unit 30, and the direction detection unit 4 is used.
This is an example in which the configuration of FIG. 12 or 14 is adopted as 0.

The operation of this system having such a configuration will be described.

First, the voice input unit 11 having a plurality of microphones, in this example, the voice input unit 11 having a total of two microphones m1 and m2, captures the voices of ch1 and ch2. The two channels of audio signals ch1 and ch2 input from the audio input unit 11 (that is, the first channel ch1 is the audio from the first microphone m1 and the second channel ch2 is the second microphone m).
(Corresponding to the voice from 2) is sent to the frequency analysis unit 12, where a frequency component (frequency spectrum) is obtained for each channel by performing processing such as fast Fourier transform (FFT).

The frequency components for each channel obtained by the frequency analysis unit 12 are respectively the first, second and third frequency components.
Beam formers 13, 16 and 22.

In the first beam former 13, the noise components are processed by adjusting the phases of the frequency component inputs for two channels according to the direction of the target sound and then processing the same as described above by the frequency domain adaptive filter. Suppress and output the frequency component in the direction of the target sound.

That is, the first input direction correction section 14 gives the following angle information (α) to the first beam former 13. That is, the first input direction correction unit 14
Uses the output from the voice direction estimation unit 18 or the voice direction estimation unit 23 given via the effective noise determination unit 24 so that the frequency components of the two channels are set so that the direction of the target sound is the front direction of the microphone. The angle information (α) necessary for adjusting the input phase is given to the first beam former 13 as an input direction correction amount.

As a result, the first beam former 13 corrects the target sound direction in accordance with the correction amount (α) and suppresses the voice coming from the direction other than the target sound direction, whereby the noise component Is suppressed and the target sound is extracted.

That is, in the case of the second and third beam formers 16 and 22, since the noise is the target sound, the phase is matched with the noise. As a result, the sound source of the speaker is treated as a noise source in the second and third beamformers 16 and 22,
Since the adaptive filter built in each beamformer performs the process of extracting the sound from the speaker sound source, the direction of the speaker sound source can be determined from the parameters of the adaptive filters of the second and third beamformers 16 and 22. Information that reflects is obtained.

Therefore, if the noise source direction is known by the first or second voice direction estimation unit 18 or 23 using the parameter of the adaptive filter in the second or third beam former 16 or 22, it is the target sound. It reflects the direction of a speaker's sound source. Therefore, the first or second voice direction estimation unit 18 or 23 outputs an output reflecting the parameter of the adaptive filter in the second or third beam former 16 or 22, and the first input direction correction unit 14 outputs the output. If an input direction correction amount (α) is generated in response to this output and the target sound direction in the first beam former 13 is corrected in accordance with this correction amount, the first beam former 13 will move from a direction other than the target sound direction. Since the incoming voice is suppressed, in this case, the component from the speaker sound source can be extracted.

On the other hand, since the parameters are controlled by the adaptive filter of the first beam former 13 so that the noise component is extracted, the noise direction estimation unit 17 estimates the noise direction from this parameter and outputs the information. Second and third
To the effective noise determining unit 24 and the input direction correcting units 15 and 21.

Then, the second input direction correction unit 15 receiving the output from the noise direction estimation unit 17 generates an input direction correction amount (α) corresponding to the output from the noise direction estimation unit 17, and If the target sound direction in the second beamformer 16 is corrected according to the correction amount, the second beamformer 16 suppresses the voice coming from directions other than the target sound direction. Therefore, the noise component that is the component of can be extracted.

At this time, since the parameters are controlled by the adaptive filter of the second beam former 16 so that the speaker voice component which is the target sound is extracted, the first voice direction estimation unit 18 uses this parameter from the parameters. , The speaker voice direction can be estimated. Then, the first voice direction estimating unit 18 gives the estimated information to the effective noise determining unit 24.

The output from the noise direction estimation unit 17 is also given to the third input direction correction unit 21, and the third input direction correction unit 21 that has received this output has the noise direction estimation unit 17 concerned. The input direction correction amount (α) is generated in response to the output from and is given to the third beam former 22. This allows
The third beam former 22 corrects its own target sound direction in accordance with the given correction amount.

As a result, the third beam former 22 suppresses the voice coming from a direction other than the target sound direction, and in this case, it is possible to extract a component other than the speaker sound source, that is, a noise component.

At this time, since the parameters are controlled by the adaptive filter of the third beam former 22 so as to extract the speaker voice component that is the target sound, the second voice direction estimation unit 23 uses the parameters from this parameter. , The speaker voice direction can be estimated. Then, this estimated information is given to the effective noise determination unit 24.

In the effective noise determining section 24, the first and second
Of the second beamformer 1 based on the speaker voice direction estimation information given from the voice direction estimation units 18 and 23 and the noise direction estimation information given from the noise direction estimation unit 17.
It is determined which of the sixth beamformer 22 and the third beamformer 22 is effectively tracking noise. Then, based on the result of this determination, the parameter of the adaptive filter in the beamformer that is determined to be effectively tracking is given to the first input direction correction unit 14.

Therefore, the first input direction correction unit 14 outputs an output reflecting the parameter, and the first input direction correction unit 14 generates the input direction correction amount (α) corresponding to this output. Since the target sound direction in the first beam former 13 is corrected according to the correction amount, the first beam former 13 is supposed to suppress the voice coming from a direction other than the target sound direction. In this case, the speaker It is possible to extract a component from a sound source, and when noise from a widely moving noise source is targeted, it is possible to reliably capture and remove the noise without losing the moving noise source.

However, since sudden noise cannot be dealt with, here, the configuration including the subtraction processing unit 30, the directionality detection unit 40, and the subtraction control unit 50 can be used.

That is, a directionality detector based on short-time frequency analysis is added, and the directionality detector 40 detects the arrival of sudden noise or high-speed moving sound source that cannot be suppressed by the beamformer processing. By controlling the spectral subtraction process performed using the speaker voice component and the noise component obtained by the noise suppression processing device, while securing the speaker tracking function that can set the allowable range in the speaker direction with high accuracy, Moreover, it is possible to suppress sudden noise, high-speed moving sound source, and the like.

That is, as described above, the direction detecting section 40 uses not only the phase difference T between the two microphones but also the power ratio R of the input signal of each channel based on the frequency analysis such as the short-time FFT. Calculate the sex index D. Then, the obtained directionality index D is given to the spectrum subtraction control unit 50.

Spectral subtraction controller 50
Is the target sound direction (speaker direction) based on the obtained directionality index D and the target sound direction information output from the target sound direction estimation unit 18.
Based on and, three kinds of signals (“0”, “1”,
Either "2") is generated and sent to the spectrum subtraction processing unit 30.

Here, of the three types of signals, the signal "0"
Indicates that the section is almost only noise, the signal “1” indicates that a large sudden noise is superimposed on the speech section, and the signal “2” indicates that the section is almost only speech. It represents.

This is obtained as follows. First, the allowable range in the speaker direction is set from the center of the two microphones and the threshold value of the directionality index D is set to "some". For example, the allowable range in the speaker direction is set to ± 20 ° from the center of the two microphones, and the threshold value of the directionality index D is set to "1.0".

Then, it is determined whether or not the directionality index D sent from the directionality detecting section 40 is less than or equal to the threshold value (1.0). If the result is less than or equal to the threshold value, then Based on the target sound direction information output from the target sound direction estimation unit 18, it is determined whether the target sound direction (speaker direction) is within the set range,
If it is within the set range, it means that it is a section of almost only voice, so a signal "2" is generated and sent to the spectrum subtraction processing unit 30. Further, if it is outside the set range, it means that it is a section including almost no noise, and therefore a signal “0” is generated and sent to the spectrum subtraction processing unit 30.

If the directionality index D is equal to or more than the threshold value and the target sound direction is within the set range, it means that a large sudden noise is superposed on the voice section, and accordingly, , It is determined that sudden noise is superposed on the voice coming from the speaker direction, a signal "1" is generated and sent to the spectrum subtraction processing unit 30, and the directionality index D
Is greater than or equal to the threshold value and the target sound direction is outside the set range, it means that it is an interval of almost only noise, and therefore a signal “0” is generated and sent to the spectrum subtraction processing unit 30.

In this way, the spectral subtraction control unit 50 is based on the target sound direction (speaker direction) estimated by the target sound direction estimation unit 18 and the directionality index D calculated by the directionality detection unit 40. , Three kinds of signals (“0”,
Either “1” or “2”) is sent to the spectrum subtraction processing unit 30.

In the spectral subtraction processing unit 30, the voice frequency component given from the first beam former 13 of the speaker tracking microphone array 10 and the second beam former 16 (or the third beam former 22).
And a noise frequency component given by the above, and performs necessary noise compression processing in accordance with the three types of signals output from the spectrum subtraction control unit 50.

The spectrum subtraction processing unit 30 receives the signal from the spectrum subtraction control unit 50, and when the signal is "0", it can be regarded almost as a noise section. Therefore, the output signal is cut by applying the minimum weight. In addition, the spectrum subtraction control unit 50
When the signal from 1 is “1”, it is considered that sudden noise is superposed on the voice section, and the two-channel spectral subtraction process is performed in which the output from the second beam former 16 is treated as a noise component.

That is, when the signal received from the spectrum subtraction control unit 50 is “0”, it can be regarded almost as a noise section. Therefore, in the spectrum subtraction processing unit 30, the control unit 35 causes the band weight calculation unit 33 to generate the minimum weight. The band weight calculation unit 33 generates the minimum weight and gives it to the spectrum subtraction unit 34, so that the spectrum subtraction unit 3
Reference numeral 4 cuts the output signal by performing a calculation for applying the minimum weight to the output of the audio frequency component and outputting the result.

Further, when the signal from the spectrum subtraction control unit 50 is "1", it can be considered that the sudden noise is superposed on the voice section, so that the control unit 35 receiving this signal receives the band weight. A control command is given to the calculation unit 33 so as to calculate and output a normal weighting coefficient, whereby the band weight calculation unit 33 causes the voice band power calculation unit 31 to operate.
Based on the output of the noise band power calculator 32, the band weight coefficient is obtained.

Then, the band weight calculation unit 33 gives the obtained band weight coefficient to the spectrum subtraction unit 34, so that the spectrum subtraction unit 34 assigns the weight to the output of the audio frequency component from the first beam former 13. By performing the multiplication calculation and outputting the result, the two-channel spectral subtraction processing that treats the output from the second beamformer 16 as a noise component is performed, and the spectral subtraction processing result is subjected to the noise suppression processing. It can be output as a sound frequency component that has already been processed.

Further, when the signal from the spectrum subtraction control section 50 is "2", it can be regarded as a voice-only section. Therefore, in the spectrum subtraction processing section 30, the control section 35 receiving this signal receives the band weight. A control command is given to the calculation unit 33 to calculate and output a normal weighting coefficient, whereby the band weight calculation unit 33 determines the band weight based on the outputs of the voice band power calculation unit 31 and the noise band power calculation unit 32. Find the coefficient.

Then, the band weight calculation unit 33 gives the obtained band weight coefficient to the spectrum subtraction unit 34, so that the spectrum subtraction unit 34 applies the weight to the output of the audio frequency component from the first beam former 13. By performing the multiplication calculation and outputting the result, the result of the state in which the 1-channel spectral subtraction is performed on the output of the first beamformer 13 is obtained, and this is output as the output of the audio frequency component.

As another control method similar to the above, when the signal is "1", a large sudden noise is superposed on the voice, so that it is regarded as a noise section and the same processing as the signal "0" is performed. Good.

With this apparatus, it is possible to extract only a voice component with little distortion in which a noise component having a direction and a noise component having no direction are suppressed, and also for a sudden noise,
It is possible to extract a voice component with less distortion that suppresses a noise component.

Next, an example of a noise suppression processing device capable of further improving the accuracy of the specific configuration example 1 will be described below as a specific configuration example 2.

(Another Example of Spectral Subtraction (SS) Processing) This specific configuration example is as shown in FIG. 18. In this case, the configuration of FIG. 7 is used as the speaker tracking microphone array 10, and As the subtraction processing unit 30, the configuration shown in FIG. 19 is applied.

In this embodiment, in the spectral subtraction (SS) process, the power of the noise component is corrected to enable the noise suppression with higher accuracy. That is, in the above example, it is assumed that the noise source power N is small. Therefore, if spectral subtraction (SS) processing is performed, there is a concern that distortion will increase in the portion where the noise source component is superimposed on the voice.

Therefore, here, the calculation of the band weight in the spectral subtraction process is corrected by using the power of the input signal (output of the frequency analysis unit 12).

First, the voice output power is Pv, the voice component power is V, the background noise power included in the voice output is B ′, the noise output power is Pn, the noise source component power is N, and the background included in the noise output. a noise component B ", and how the signal is also the power of the input signal without suppression and Px, Px = V + N + B Pv = V + B 'Pn = N + B" where where, B ~ B' ~ B "assuming, The power Pb of the true background noise component is Pb = Pv + Pn-Px = V + B '+ N + B "-(V + N + B) = B' + B" -B = B. The weight of the spectral subtraction (SS) using this noise power is , W = (Pv-Pb) / Pv = (Px-Pn) / Pv, and SS processing with little distortion can be performed even when background noise is non-stationary and N is large.

FIG. 19 shows a configuration example of the spectrum subtraction processing unit 30 used in this embodiment, and FIG. 20 shows a processing flow. In FIG. 19, 31 is a voice band power calculation unit, 32 is a noise band power calculation unit, 34 is a spectrum subtraction unit, 35 is a control unit, and 37 is an input signal band power calculation unit.

Of these, the voice band power calculation unit 31
Divides the audio frequency obtained by the first beamformer 13 into frequency bands to calculate the audio power for each band, averages the calculated power values in the time direction, and The average power (average band power Pv (k) of voice) is obtained and given to the band weight calculation unit 33. The noise band power calculation unit 32 uses the first beam former 16 or the (second beam former). 22)
The noise band power is calculated using the noise frequency component obtained by the effective noise determination unit 24 and output, and the calculated power values are averaged in the time direction to obtain the average power (noise) for each band. Average band power of Pn
(K)) is obtained and given to the band weight calculation unit 33.

Further, the input signal band power calculation unit 37 is
The frequency spectrum component of the input signal (either ch1 or ch2) obtained from the frequency analysis unit 12 is divided for each frequency band, and the input power for each band is calculated, and the calculated power value is calculated in the time direction. Averaged to
Average power for each band (average band power P (
k)) is obtained and given to the band weight calculation unit 33, and is given to the band weight calculation unit 33.

In addition, the band weight calculation unit 33 calculates the average band power Pv (k) of voice and the average band power Pn (k) of noise obtained by the input signal band power calculation unit 37 for each band k. The band weighting coefficient W (k) for each band is calculated based on the calculated average input band power P (k). The spectrum subtraction unit 34 includes the band weight calculation unit 33.
The background weight is suppressed by using the band weighting coefficient W (k) for each band obtained by the above, and applying the weighting coefficient for each frequency band of the audio signal.

The control unit 35 receives the signal from the spectrum subtraction control unit 50 and controls the band weight calculation unit 33 according to the signal type, and the signal from the spectrum subtraction control unit 50 is " When it is “0”, a control command is given to the band weight calculation unit 33 so as to generate the minimum weight, and the output of the band weight calculation unit 33 is controlled to have the minimum weight. When the signals are "1" and "2", the control unit 35 controls the band weight calculation unit 33 so as to obtain a normal weight coefficient.

The spectrum subtraction unit 34 is the band weight calculation unit 3
3 is used to multiply the output from the second beamformer 16 by the weighting coefficient and output the result as a signal of an audio frequency component in which a noise component is suppressed.

The structure of the spectrum subtraction (SS) processing unit 30 shown in FIG. 19 is different from the structure of the spectrum subtraction (SS) processing unit 30 of FIG. 10 in that no frequency component of the input signal (frequency analysis) is suppressed. The output of the unit 12) is further used.

Since there are two microphones for the output of the frequency analysis unit 12, there are two systems of ch1 and ch2, but either one may be used. With respect to the input signal frequency component from the frequency analysis unit 12, the input signal band power calculation unit 37 calculates the power for each band similarly to the voice frequency component or the noise frequency component from the beamformer (step S61).

As in the case of FIG. 10, the audio frequency component is output from the first beamformer 13, and
Since the noise frequency component is given as the output from the second beamformer 15 (or the third beamformer 22), the voice band power calculation unit 31 uses the voice frequency component output from the first beamformer 13. Voice band power calculation is performed (step S62), and the noise band power calculation unit 32 calculates the noise band power using the noise frequency component output from the second beam former 15 (or the third beam former 22). Is carried out (step S63).

Then, using these, the band weight calculating section 33 obtains the weight coefficient as described above (step S
64).

Then, the spectrum subtraction unit 34 uses the obtained weighting coefficient to perform a calculation for multiplying the output of the audio frequency component from the first beamformer 13 by the weighting, and outputs the result to obtain the spectrum. The subtraction processed voice frequency component is output (step S6).
5).

The calculation of the weighting coefficient by the band weight calculation unit 33 is carried out by the direction detection unit 40 and the spectrum subtraction control unit 50 under the same control as described above. That is, in the direction detection unit 40, as described above, the short-time FF
Based on the frequency analysis of T etc., not only the phase difference T of the two microphones but also the power ratio R of the input signal of each channel
Is used to calculate the directionality index D. Then, the obtained directionality index D is used as the spectrum subtraction control unit 5
Give to 0.

Spectral subtraction controller 50
Is the target sound direction (speaker direction) based on the obtained directionality index D and the target sound direction information output from the target sound direction estimation unit 18.
Based on and, three kinds of signals (“0”, “1”,
Either "2") is generated and sent to the spectrum subtraction processing unit 30.

Here, of the three types of signals, the signal "0"
Indicates that the section is almost only noise, the signal “1” indicates that a large sudden noise is superimposed on the speech section, and the signal “2” indicates that the section is almost only speech. It represents.

This is obtained as follows. First, the allowable range in the speaker direction is set from the center of the two microphones and the threshold value of the directionality index D is set to "some". For example, the allowable range in the speaker direction is set to ± 20 ° from the center of the two microphones, and the threshold value of the directionality index D is set to "1.0".

Then, it is determined whether or not the directionality index D sent from the directionality detecting section 40 is less than or equal to the threshold value (1.0). If the result is less than or equal to the threshold value, then Based on the target sound direction information output from the target sound direction estimation unit 18, it is determined whether the target sound direction (speaker direction) is within the set range,
If it is within the set range, it means that it is a section of almost only voice, so a signal "2" is generated and sent to the spectrum subtraction processing unit 30. Further, if it is outside the set range, it means that it is a section including almost no noise, and therefore a signal “0” is generated and sent to the spectrum subtraction processing unit 30.

If the directionality index D is equal to or more than the threshold value and the target sound direction is within the set range, it means that a large sudden noise is superposed on the voice section, and accordingly, , It is determined that sudden noise is superposed on the voice coming from the speaker direction, a signal "1" is generated and sent to the spectrum subtraction processing unit 30, and the directionality index D
Is greater than or equal to the threshold value and the target sound direction is outside the set range, it means that it is an interval of almost only noise, and therefore a signal “0” is generated and sent to the spectrum subtraction processing unit 30.

In this way, the spectral subtraction control unit 50 is based on the target sound direction (speaker direction) estimated by the target sound direction estimation unit 18 and the directionality index D calculated by the directionality detection unit 40. , Three kinds of signals (“0”,
Either “1” or “2”) is sent to the spectrum subtraction processing unit 30.

In the spectral subtraction processing section 30, the voice frequency component given from the first beam former 13 of the speaker tracking microphone array 10 and the second beam former 16 (or the third beam former 22) are used.
And a noise frequency component given by the above, and performs necessary noise compression processing in accordance with the three types of signals output from the spectrum subtraction control unit 50.

The spectrum subtraction processing unit 30 receives the signal from the spectrum subtraction control unit 50, and when the signal is "0", it can be regarded almost as a noise section. Therefore, the output signal is cut by applying the minimum weight. In addition, the spectrum subtraction control unit 50
When the signal from 1 is “1”, it is considered that sudden noise is superposed on the voice section, and the two-channel spectral subtraction process is performed in which the output from the second beam former 16 is treated as a noise component.

That is, when the signal received from the spectrum subtraction control unit 50 is "0", it can be regarded as almost a noise section. Therefore, in the spectrum subtraction processing unit 30, the control unit 35 causes the band weight calculation unit 33 to generate the minimum weight. The band weight calculation unit 33 generates the minimum weight and gives it to the spectrum subtraction unit 34, so that the spectrum subtraction unit 3
Reference numeral 4 cuts the output signal by performing a calculation for applying the minimum weight to the output of the audio frequency component and outputting the result.

Further, when the signal from the spectrum subtraction control unit 50 is "1", it can be considered that sudden noise is superposed on the voice section. A control command is given to the calculation unit 33 so as to calculate and output a normal weighting coefficient, whereby the band weight calculation unit 33 causes the voice band power calculation unit 31 to operate.
Based on the output of the noise band power calculator 32, the band weight coefficient is obtained.

Then, the band weight calculation section 33 gives the obtained band weight coefficient to the spectrum subtraction section 34, so that the spectrum subtraction section 34 assigns the weight to the output of the audio frequency component from the first beam former 13. By performing the multiplication calculation and outputting the result, the two-channel spectral subtraction processing that treats the output from the second beamformer 16 as a noise component is performed, and the spectral subtraction processing result is subjected to the noise suppression processing. It can be output as a sound frequency component that has already been processed.

Further, when the signal from the spectrum subtraction control unit 50 is "2", it can be regarded as a voice only section. Therefore, in the spectrum subtraction processing unit 30, the control unit 35 receiving this signal receives the band weight. A control command is given to the calculation unit 33 to calculate and output a normal weighting coefficient, whereby the band weight calculation unit 33 determines the band weight based on the outputs of the voice band power calculation unit 31 and the noise band power calculation unit 32. Find the coefficient.

Then, the band weight calculation unit 33 gives the obtained band weight coefficient to the spectrum subtraction unit 34, so that the spectrum subtraction unit 34 applies the weight to the output of the audio frequency component from the first beam former 13. By performing the multiplication calculation and outputting the result, the result of the state in which the 1-channel spectral subtraction is performed on the output of the first beamformer 13 is obtained, and this is output as the output of the audio frequency component.

As another control method similar to the above, when the signal is "1", a large sudden noise is superposed on the voice, so that it is regarded as a noise section and the same processing as the signal "0" is performed. Good.

The present apparatus can extract only a voice component with little distortion in which a noise component having a direction and a noise component having no direction are suppressed, and also with respect to a sudden noise,
It is possible to extract a voice component with less distortion that suppresses the noise component, and further, by correcting the power of the noise component in the spectral subtraction (SS) processing, it is possible to perform noise suppression with higher accuracy. It is possible to provide the following noise suppression processing device.

As a result, it becomes possible to extract only the voice component with a small distortion in which the noise component having the direction and the noise component having no direction are suppressed.

As described above, in this example, in the noise suppressor, the input signal obtained by frequency-analyzing the input signal obtained from the voice input means is divided into frequency components for each frequency band, and the input power for each band is calculated. A band power calculation means is provided to cause the spectrum subtraction means to carry out a process of weighting each frequency band of the voice signal and suppressing the background noise based on the input band power, the voice band power and the noise band power. It is characterized in that it is configured in.

In the case of this configuration, the voice band power calculation means divides the obtained spectrum component of the voice frequency into frequency bands to calculate the voice power for each band, and the noise band power calculation means obtains the obtained voice power. The spectral component of the obtained noise frequency is divided for each frequency band, and the noise power for each band is calculated. Further, there is an input band power calculation means, and this input band power calculation means receives the frequency spectrum component of the input voice obtained by frequency-analyzing the input signal obtained from the voice input means, and receives this for each frequency band. Divide and calculate the input power for each band. Then, the spectrum subtracting means suppresses the background noise by weighting each frequency band of the voice signal based on the frequency band power of the voice and noise obtained from the voice band power calculating means and the noise band power calculating means.

In this embodiment, in the spectral subtraction processing in the configuration example 1, the power of the noise component is further corrected, so that the noise can be suppressed with higher accuracy. That is, in the third invention, it is assumed that the power N of the noise source is small. Therefore, when the spectral subtraction process is performed, it is unavoidable that the distortion becomes large in the portion where the noise source component is superimposed on the voice. Here, the power of the input signal is used to correct the calculation of the band weighting coefficient in the spectral subtraction processing.

As a result, it becomes possible to extract only the voice component with a small distortion by suppressing the noise component having the direction and the noise component having no direction.

While various embodiments have been described above, the first aspect of the present invention is voice input means for receiving a voice of a speaker at two or more different positions and outputting the voice signals as voice signals. Frequency analysis means for performing frequency analysis for each channel of the audio signal corresponding to the sound receiving position and outputting frequency components for each channel, and adaptive filter processing using frequency components of a plurality of channels output by the frequency analysis means First beam former processing means for suppressing incoming noise other than the target voice and outputting a signal of the target voice component, and adaptive filter processing using the frequency components of a plurality of channels output by the frequency analysis means Is calculated by the second beamformer processing means for performing the suppression processing of the voice and outputting the signal of the noise component, and the first beamformer processing means. A noise direction estimating means for estimating a noise direction from a filter coefficient, a target sound direction estimating means for estimating a target sound direction from a filter coefficient calculated by the second beamformer processing means, and the first beamformer processing means. A first input direction correcting means for sequentially correcting a first input direction, which is a direction of arrival of a target sound to be input, on the basis of the target sound direction estimated by the target sound direction estimating means; Second input direction correction means for sequentially correcting the second input direction, which is the arrival direction of the noise to be input in the beamformer processing means, based on the noise direction estimated by the noise direction estimation means, First
Spectrum subtraction means for performing a non-linear noise suppression processing based on the output of the beam former processing means and the output of the second beam former processing means, and the incoming sound from the frequency component output from the frequency analyzing means. Of the spectral subtraction means based on the directionality index and the target sound direction output from the target sound direction estimating means And a spectral subtraction control means for controlling the subtraction process.

In the case of such a configuration, the voice input means receives the voice uttered by the speaker at two or more different positions, and the frequency analysis means receives the voice signal corresponding to the sound receiving position. The frequency analysis is performed for each channel and the frequency components of a plurality of channels are output. Then, the first beamformer processing means performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. The incoming beam noise suppressing process for suppressing a voice other than the voice from the speaker direction is performed to obtain a target voice component, and the second beamformer processing means
The frequency components of the plurality of channels obtained by the frequency analysis means are subjected to adaptive filter processing using filter coefficients calculated so that sensitivity outside the desired direction becomes low, thereby suppressing voice from the speaker direction. Then, the noise component is obtained. The noise direction estimating means estimates the noise direction from the filter coefficient calculated by the first beamformer processing means, and the target sound direction estimating means calculates the noise direction by the filter coefficient calculated by the second beamformer processing means. The target sound direction is estimated from. The target sound direction correcting means is the first
In the beam former, the first input direction, which is the arrival direction of the target sound to be input, is sequentially corrected on the basis of the target sound direction estimated by the target sound direction estimation means.
The first beam former suppresses noise components coming from other than the first input direction and extracts the speaker's voice component with low noise. Further, the noise direction correction means sequentially corrects the second input direction, which is the arrival direction of noise to be input in the second beamformer, based on the noise direction estimated by the noise direction estimation means. , The second beam former suppresses the components coming from other than the second input direction and extracts the remaining noise components in which the speaker's voice component is suppressed.

As described above, the present system can separately obtain the voice frequency component in which the noise component is suppressed and the noise frequency component in which the voice component is suppressed. The greatest feature of the present invention is that the first and second The beamformer operating in the frequency domain is used as the beamformer of the above, and in the present invention, the incoming sound comes from the target direction by using short-time data so as to deal with sudden noise. Incorporating directionality detection means to obtain a directionality index for deciding whether or not to be accurate, and suppressing spectral noise by controlling the spectral subtraction based on the directionality index and the speaker direction in conventional processing. It is in.

According to this, by determining the target sound / noise based on both the above-mentioned directionality index and the speaker direction obtained from the directivity of the beamformer filter,
It is possible to remove voices from areas other than the set speaker range, and it is also possible to remove signals with short durations, such as sudden noise, with high accuracy, making it possible to perform noise suppression processing in an actual environment with extremely high accuracy. Become.

Further, since the beam formers operating in the frequency domain are used as the first and second beam formers, the amount of calculation can be greatly reduced.

According to the present invention, the processing amount of the adaptive filter is greatly reduced, and frequency analysis processing other than the frequency analysis on the input voice can be omitted, and it is necessary for the filter calculation. The conversion processing from the time domain to the frequency domain is also unnecessary, and the overall calculation amount can be significantly reduced.

That is, in the prior art, the spectral subtraction (hereinafter abbreviated as SS) process is performed after the beamformer process in order to suppress the spreading noise that cannot be suppressed by the beamformer. Takes the frequency spectrum as input, so FFT
Frequency analysis such as (Fast Fourier Transform) has been conventionally required, but when a beamformer operating in the frequency domain is used, the frequency spectrum is output from the beamformer, and this can be diverted to SS. Conventional FFT processing steps that perform FFT for SS can be omitted. Therefore, the total calculation amount can be significantly reduced.

Further, the conversion processing from the time domain to the frequency domain, which is necessary for the direction estimation using the filter of the beamformer, becomes unnecessary, and the total amount of calculation can be greatly reduced.

Secondly, according to the present invention, a voice input means for receiving a voice of a speaker at two or more different positions and outputting the voice signals respectively, and a voice signal corresponding to the sound receiving position. Frequency analysis means for performing frequency analysis for each channel and outputting frequency components for each channel, and frequency components of the plurality of channels obtained by this frequency analysis means were calculated so that sensitivity outside the desired direction becomes low. First beamformer processing means for obtaining a target voice component by performing incoming noise suppression processing for suppressing voices other than the voice from the speaker direction by performing adaptive filter processing using a filter coefficient, and the frequency analysis. Adaptive filter using the filter coefficient calculated so that the sensitivity outside the desired direction becomes low for the frequency components of the plurality of channels obtained by the means A second beamformer processing unit for suppressing the voice from the speaker direction to obtain a first noise component by applying a logic, and the frequency components of the plurality of channels obtained by the frequency analysis unit in a desired direction. Second beamformer processing means for suppressing the voice from the speaker direction and obtaining a second noise component by performing adaptive filter processing using a filter coefficient calculated so that the outside sensitivity becomes low; A noise direction estimating means for estimating a noise direction from the filter coefficient calculated by the first beamformer processing means, and a first target sound direction is estimated from a filter coefficient calculated by the second beamformer processing means. A second target sound direction for estimating the second target sound direction from the first target sound direction estimation means and the filter coefficient calculated by the third adaptive beamformer processing means. The first input sound direction estimated by the first target sound direction estimation means is the first input direction which is the arrival direction of the target sound input by the estimation means and the first beamformer processing means. When,
First input direction correcting means for sequentially correcting based on one or both of the second target sound directions estimated by the second target sound direction estimating means, and the noise direction estimated by the noise direction correcting means Is within a predetermined first range, the second input for sequentially correcting the second input direction, which is the arrival direction of the noise to be input in the second beamformer processing means, based on the noise direction. When the noise direction estimated by the direction correcting means and the noise direction correcting means is within a predetermined second range, the third direction which is the arrival direction of the noise to be input in the third beamformer processing means Third input direction correcting means for sequentially correcting the input direction based on the noise direction, and whether the noise direction estimated by the noise direction estimating means has come from a predetermined first range or from a predetermined second range. Has it arrived Based on this, either one of the first and second output noises is determined as a true noise output, and either one of the noises is output, and at the same time, the first voice direction estimating means and the second voice direction estimating means Effective noise determining means for determining which estimation result is valid and outputting one of the speech direction estimation results to the first input direction correcting means, an output of the first beamformer processing means, and Based on the output of the second beamformer processing means, the spectrum subtraction means performs the spectrum subtraction processing which is a non-linear noise suppression processing, and the time difference and the amplitude difference of the incoming sound from the frequency component output from the frequency analysis means. Directionality detecting means for calculating a directionality index, and the spatial direction based on the directionality index and the target sound direction output from the target sound direction estimating means. It was constructed and a spectral subtraction control means for controlling the spectral subtraction process torr subtraction means.

In the case of this configuration, the voice input means receives the voice uttered by the speaker at two or more different positions, and the frequency analysis means receives this for each channel of the voice signal corresponding to the sound receiving position. Frequency analysis is performed and frequency components of a plurality of channels are output. Then, the first beamformer processing means performs adaptive filter processing on the frequency components of the plurality of channels obtained by the frequency analysis means using filter coefficients calculated so that the sensitivity outside the desired direction becomes low. Incoming noise suppression processing for suppressing voices other than the voice from the speaker direction is performed to obtain a target voice component, and the second beamformer processing means is for the plurality of channels of the plurality of channels obtained by the frequency analysis means. The frequency component is subjected to adaptive filter processing using a filter coefficient calculated so that sensitivity outside the desired direction becomes low, whereby the voice from the speaker direction is suppressed and a noise component is obtained. The noise direction estimating means estimates the noise direction from the filter coefficient calculated by the first beamformer processing means, and the target sound direction estimating means calculates the noise direction by the filter coefficient calculated by the second beamformer processing means. The target sound direction is estimated from.

The first target sound direction estimating means estimates the first target sound direction from the filter coefficient calculated by the second beamformer processing means, and the second target sound direction estimating means calculates The second target sound direction is estimated from the filter coefficient calculated by the third adaptive beam former processing means.

The first input direction correcting means estimates the first input direction, which is the arrival direction of the target sound to be input in the first beam former, by the first target sound direction estimating means. The first target sound direction and the second target sound direction estimated by the second target sound direction estimating means are sequentially corrected based on one or both of them. When the noise direction estimated by the noise direction correcting unit is within a predetermined first range, the second input direction correcting unit determines the arrival direction of noise to be input in the second beamformer. A certain second input direction is sequentially corrected on the basis of the noise direction, and the third input direction correction means, when the noise direction estimated by the noise direction correction means is within a predetermined second range, The third input direction, which is the arrival direction of noise to be input in the third beam former, is sequentially corrected based on the noise direction.

Therefore, the second beam former whose second input direction is corrected by the output of the second input direction correcting means suppresses components coming from other than the second input direction and extracts the remaining noise components. In addition, the third beam former whose third input direction is corrected by the output of the third input direction correcting means suppresses the components arriving from other than the third input direction and remaining noise components. Will be extracted.

Then, the effective noise determining means determines whether the noise direction estimated by the noise direction estimating means comes from the predetermined first range or the predetermined second range. Either one of the output noise and the second output noise is determined to be a true noise output, and either one of the noises is output, and at the same time, one of the first voice direction estimating means and the second voice direction estimating means is output. It is determined whether the estimation result is valid and the valid voice direction estimation result is output to the first input direction correcting means.

As a result, the target sound direction correction means obtains the first input direction, which is the arrival direction of the target sound to be input in the first beam former, by the determined target sound direction estimation means. The first beamformer suppresses the noise components coming from other than the first input direction and extracts the speaker's voice component with low noise, because the correction is sequentially performed based on the target sound direction.

As described above, this system can separately obtain the voice frequency component in which the noise component is suppressed and the noise frequency component in which the voice component is suppressed. The greatest feature of the present invention is that the first and second The beamformer operating in the frequency domain is used as the beamformer of the above, and in the present invention, the incoming sound comes from the target direction by using short-time data so as to deal with sudden noise. Incorporating directionality detection means to obtain a directionality index for deciding whether or not to be accurate, and suppressing spectral noise by controlling the spectral subtraction based on the directionality index and the speaker direction in conventional processing. It is in.

According to this, by determining the target sound / noise based on both the above-mentioned directionality index and the speaker direction obtained from the directivity of the filter of the beamformer,
It is possible to remove voices from areas other than the set speaker range, and it is also possible to remove signals with short durations, such as sudden noise, with high accuracy, making it possible to perform noise suppression processing in an actual environment with extremely high accuracy. Become.

Further, since the beam formers operating in the frequency domain are used as the first and second beam formers, the amount of calculation can be greatly reduced.

According to the present invention, the processing amount of the adaptive filter is significantly reduced, and frequency analysis processing other than the frequency analysis on the input voice can be omitted, which is necessary for the filter calculation. The conversion processing from the time domain to the frequency domain is also unnecessary, and the overall calculation amount can be significantly reduced.

Further, in the present invention, a noise tracking beamformer having completely different monitoring areas for noise tracking is provided to estimate the voice direction from each output and which is effective from each estimation result. It is determined whether or not the noise tracking is performed properly, and the result of estimation of the voice direction by the filter coefficient of the beamformer that is determined to be effective is given to the first target sound direction correction means to correct the first target sound direction. The means sequentially corrects the arrival direction of the target sound as the input target in the first beam former, based on the target sound direction estimated by the target sound direction estimation means. The beamformer No. 1 can suppress the noise component coming from other than the first input direction and extract the speaker's voice component with low noise, and even if the noise source moves, this is lost. In which it is possible to suppress and tracking without.

In the prior art, 2ch, that is,
In order to enable tracking of the target sound source with only two microphones, one beamformer for noise tracking is used separately from the beamformer for noise suppression. For example, the noise source has moved across the direction of the target sound. In such a case, noise tracking accuracy may decrease.

However, according to the present invention, since a plurality of beam formers for tracking noise are used to take charge of separate tracking ranges, it is possible to prevent the tracking accuracy from deteriorating even in the above case.

The present invention is not limited to the above-mentioned embodiments, but can be modified in various ways.

[0366]

As described above in detail, according to the present invention, it is possible to significantly reduce the total calculation amount, and
The conversion processing from the time domain to the frequency domain, which was necessary when performing the direction estimation using the filter of the beamformer, is not required, and the effect that the entire calculation amount can be significantly reduced can be obtained.

Further, in the present invention, since the beam former for extracting the noise component is prepared and the output of this beam former is used, the phase shift is corrected,
Therefore, highly accurate spectrum subtraction processing can be realized even in the case of non-stationary noise. Furthermore, since the output of the beamformer in the frequency domain is used, it is possible to omit frequency analysis and perform spectral subtraction.
It is possible to suppress non-stationary noise with a smaller amount of calculation than before, and to suppress not only directional noise components but also non-directional noise components (background noise) and extract speech components with less distortion. A so-called effect can be obtained.

Particularly, according to the present invention, the target sound / noise is determined based on both the directionality index and the speaker direction obtained from the directivity of the beam former filter.
It is possible to remove voices from areas other than the set speaker range, and it is also possible to remove signals with short durations, such as sudden noise, with high accuracy, making it possible to perform noise suppression processing in an actual environment with extremely high accuracy. Become.

[Brief description of drawings]

FIG. 1 is a diagram for explaining the present invention and is an overall configuration block diagram showing a basic configuration example of the present invention.

FIG. 2 is a diagram for explaining the present invention and is a flowchart showing a processing flow of a spectrum subtraction processing control unit used in the system of the present invention.

FIG. 3 is a diagram for explaining the present invention and is a block diagram showing an overall configuration of a speaker tracking microphone array 10 used in the system of the present invention.

FIG. 4 is a diagram for explaining the present invention, and is a diagram for explaining a configuration example and an operation example of a beam former used in the system of the present invention.

FIG. 5 is a diagram for explaining the present invention, and is a flowchart for explaining the operation of the direction estimating unit used in the system of the present invention.

FIG. 6 is a diagram for explaining the present invention, and is a flowchart for explaining the operation of the system of the present invention.

FIG. 7 is a diagram for explaining the present invention and is an overall configuration block diagram showing another configuration example of the system of the present invention.

FIG. 8 is a diagram for explaining the present invention and is a diagram for explaining the tracking range of the beam former in the configuration example 2 of the system of the present invention.

FIG. 9 is a diagram for explaining the present invention and is a flowchart for explaining the operation of the system in Configuration Example 2 of the system of the present invention.

FIG. 10 is a diagram for explaining the present invention, which is a spectrum subtraction (S
5 is a block diagram showing a configuration example of S) processing unit 30. FIG.

FIG. 11 is a diagram for explaining the present invention, which is a spectrum subtraction (S
S) A flowchart for explaining the operation of the processing unit 30.

FIG. 12 is a diagram for explaining the present invention and is a block diagram showing a configuration example of a directionality detection unit used in the system of the present invention.

FIG. 13 is a diagram for explaining the present invention and is a diagram showing a flow of processing of a directional index detection unit used in the system of the present invention.

FIG. 14 is a diagram for explaining the present invention and is a block diagram showing another configuration example of the directionality detection unit used in the system of the present invention.

FIG. 15 is a diagram for explaining the present invention and is a diagram showing a flow of processing of a directional index detection unit used in the system of the present invention.

FIG. 16 is a diagram for explaining the present invention and is a block diagram showing another specific configuration example of the system of the present invention.

FIG. 17 is a diagram for explaining the present invention and is a diagram showing the flow of the overall processing of the microphone array used in the system of the present invention.

FIG. 18 is a diagram for explaining the present invention and is a block diagram showing another specific configuration example of the system of the present invention.

FIG. 19 is a diagram for explaining the present invention, which is a spectrum subtraction (S
S) A block diagram showing another configuration example of the processing unit 30.

20 is a diagram for explaining the present invention and is a flowchart for explaining the operation of the spectrum subtraction (SS) processing unit 30 of the configuration of FIG. 19 used in the system of the present invention.

[Explanation of symbols]

10. Speaker tracking microphone array 11 ... Voice input section 12 ... Frequency analysis unit 13 ... First beam former 14 ... First input direction correction unit 15 ... Second input direction correction unit 16 ... Second beam former 17 ... Noise direction estimation unit 18 ... First voice direction estimation unit (target sound direction estimation unit) 21 ... Third input direction correction unit 22 ... Third beam former 23 ... Second voice direction estimating unit 24 ... Effective noise determination unit 30 ... Spectral subtraction (SS) processing unit 31 ... Voice band power calculator 32 ... Noise band power calculator 33 ... Band weight calculation unit 34 ... Spectrum subtraction unit 35 ... Input signal band power calculator 40 ... Direction detection unit 50 ... Spectral subtraction control unit.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-11-52977 (JP, A) JP-A-5-52923 (JP, A) JP-A-2000-47699 (JP, A) JP-A-4-245300 (JP, A) JP-A-10-207490 (JP, A) JP-A-11-41687 (JP, A) Hitoshi Nagata, Study on noise suppression by speaker tracking 2ch beamformer, IEICE technical research Report [Voice], July 18, 1997, SP97-36, p. 17-22 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G10L 21/02 G01S 3/802 G01S 7/523 JISC file (JOIS)

Claims (11)

(57) [Claims] 1. A voice input means for receiving a voice of a speaker at two or more different positions and outputting as a voice signal respectively, and performing frequency analysis for each channel of the voice signal corresponding to the sound receiving position. Frequency analysis means for outputting the frequency component of each channel, respectively, using the frequency components of the plurality of channels output by the frequency analysis means to suppress the incoming noise other than the target voice by adaptive filter processing, the target voice component A first beamformer processing means for outputting a signal; and a second beamformer processing means for suppressing a target voice by adaptive filter processing using frequency components of a plurality of channels output by the frequency analysis means and outputting a noise component signal. Beam former processing means, and a noise direction estimating means for estimating the noise direction from the filter coefficient calculated by the first beam former processing means. A target sound direction estimating means for estimating a target sound direction from a filter coefficient calculated by the second beamformer processing means, and an arrival direction of a target sound as an input target in the first beamformer processing means. First input direction correction means for sequentially correcting a certain first input direction based on the target sound direction estimated by the target sound direction estimation means; and noise to be input by the second beamformer processing means. Second input direction correcting means for sequentially correcting the second input direction, which is the arrival direction of the signal, based on the noise direction estimated by the noise direction estimating means, and the output of the first beamformer processing means and the first input direction correcting means. Spectrum subtraction means for performing spectrum subtraction processing which is nonlinear noise suppression processing based on the output of the second beamformer processing means; Directionality detecting means for calculating a directionality index based on the time difference and the amplitude difference of the incoming sound from the frequency component output from the analyzing means, and the directionality index and the target sound output from the target sound direction estimating means. And a spectrum subtraction control means for controlling the spectrum subtraction processing of the spectrum subtraction means on the basis of the direction and the direction, and outputs a target voice frequency component. 2. A plurality of microphones each having directivity are arranged such that axes of the directivity directions are inclined to each other,
Voice input means for obtaining the voice of the speaker as a voice signal, frequency analysis means for frequency-analyzing the voice signal for each of the microphones of the voice input means and outputting a frequency component for each channel, and output of the frequency analysis means The first beamformer processing means for performing the suppression processing of the incoming noise other than the target voice by the adaptive filter processing using the frequency components of the plurality of channels, and outputting the signal of the target voice component, and the output of the frequency analysis means. Second beamformer processing means for suppressing target voice by adaptive filter processing using frequency components of a plurality of channels and outputting a noise component signal; and a filter calculated by the first beamformer processing means. Noise direction estimation means for estimating the noise direction from the coefficient, and a filter calculated by the second beamformer processing means A target sound direction estimating means for estimating a target sound direction from a coefficient, and a first input direction which is a direction of arrival of a target sound as an input target in the first beamformer processing means is estimated by the target sound direction estimating means. A first input direction correcting means for sequentially correcting based on the generated target sound direction; and a second input direction, which is a direction of arrival of noise to be input in the second beamformer processing means, for estimating the noise direction. Second input direction correction means for sequentially correcting the noise direction estimated by the means, and nonlinear noise suppression processing based on the output of the first beamformer processing means and the output of the second beamformer processing means. The spectral subtraction means for performing the spectral subtraction process and the two channel components of the frequency components of each channel output from the frequency analysis means. Based on the phase difference between the partial frequency components,
Phase difference equivalent amount calculating means for calculating a phase difference equivalent amount which is an amount corresponding to a time difference between channel signals, inter-channel power ratio calculating means for calculating a power ratio between channels from frequency components of the two channels, Directionality detecting means including directionality calculating means for calculating a directionality index based on the phase equivalent amount calculated by the phase difference equivalent amount calculating means and the interchannel power ratio calculated by the interchannel power ratio calculating means; A target sound frequency, which comprises a spectrum subtraction control means for controlling the processing of the spectrum subtraction means based on the directionality index obtained by the directionality detection means and the target sound direction output from the target sound direction estimation means. A noise component suppressing device characterized by outputting a component. 3. A plurality of microphones, each of which has directivity, are arranged such that axes of the directivity directions are inclined to each other,
Voice input means for obtaining the voice of the speaker as a voice signal, frequency analysis means for frequency-analyzing the voice signal for each of the microphones of the voice input means and outputting a frequency component for each channel, and output of the frequency analysis means The first beamformer processing means for performing the suppression processing of the incoming noise other than the target voice by the adaptive filter processing using the frequency components of the plurality of channels, and outputting the signal of the target voice component, and the output of the frequency analysis means. Second beamformer processing means for suppressing target voice by adaptive filter processing using frequency components of a plurality of channels and outputting a noise component signal; and a filter calculated by the first beamformer processing means. Noise direction estimation means for estimating the noise direction from the coefficient, and a filter calculated by the second beamformer processing means A target sound direction estimating means for estimating a target sound direction from a coefficient, and a first input direction which is a direction of arrival of a target sound as an input target in the first beamformer processing means is estimated by the target sound direction estimating means. A first input direction correcting means for sequentially correcting based on the generated target sound direction; and a second input direction, which is a direction of arrival of noise to be input in the second beamformer processing means, for estimating the noise direction. Second input direction correction means for sequentially correcting the noise direction estimated by the means, and nonlinear noise suppression processing based on the output of the first beamformer processing means and the output of the second beamformer processing means. A spectral subtraction means for performing a spectral subtraction process, and two of the frequency components of the plurality of frequency components, The spectrum normalizing means for normalizing the magnitude of the frequency component and the power of the difference component between the two channel frequency components normalized by the spectrum normalizing means are calculated, and the power of the difference spectrum is used as a directional index. Directionality detecting means including inter-channel spectral difference calculating means for obtaining, and the spectral subtraction means based on the directionality index obtained by the directionality detecting means and the target sound direction output from the target sound direction estimating means. A noise subtraction apparatus comprising: a spectral subtraction control unit that controls processing, and outputting a target voice frequency component. 4. A voice input means for receiving a voice of a speaker at two or more different positions and outputting each as a voice signal, and performing frequency analysis for each channel of the voice signal corresponding to the sound receiving position. Frequency analysis means for outputting frequency components for each channel, and adaptive filters using filter coefficients calculated so that the sensitivity outside the desired direction is low for the frequency components of the plurality of channels obtained by the frequency analysis means. A first noise obtaining process for obtaining a target voice component by performing an incoming noise suppressing process for suppressing a voice other than the voice from the speaker direction by performing a process.
The beamformer processing means and the frequency component of the plurality of channels obtained by the frequency analysis means are subjected to adaptive filter processing using filter coefficients calculated so that sensitivity outside the desired direction becomes low. Second beamformer processing means for suppressing the voice from the other direction and obtaining the first noise component, and frequency components of the plurality of channels obtained by the frequency analyzing means, so that sensitivity outside the desired direction becomes low. Third beamformer processing means for suppressing the voice from the speaker direction to obtain a second noise component by performing adaptive filter processing using the filter coefficient calculated in step 1; and the first beamformer processing. Noise direction estimation means for estimating the noise direction from the filter coefficient calculated by the means, and a filter calculated by the second beamformer processing means. A first target sound direction estimating means for estimating a first target sound direction from a filter coefficient; and a second purpose for estimating a second target sound direction from a filter coefficient calculated by the third beam former processing means. Sound direction estimating means and a first input direction estimated by the first target sound direction estimating means, the first input direction being the arrival direction of the target sound to be input in the first beamformer processing means. A first input direction correcting means for sequentially correcting the sound direction and / or the second target sound direction estimated by the second target sound direction estimating means; and the noise direction correcting means for estimating A second arrival direction of noise to be input in the second beamformer processing means when the generated noise direction is within a predetermined first range.
A second input direction correcting means for sequentially correcting the input direction of the light source based on the noise direction, and the third beam when the noise direction estimated by the noise direction correcting means is within a predetermined second range. Third, which is the arrival direction of noise to be input in the former processing means
Third input direction correcting means for sequentially correcting the input direction of the noise direction based on the noise direction, and whether the noise direction estimated by the noise direction estimating means has come from a predetermined first range or a predetermined second range. From one of the first and second output noises is determined as a true noise output and one of the noises is output, and at the same time, the first voice direction estimating means and the second voice direction estimating means. Effective noise determination means for determining which estimation result of the speech direction estimation means is valid and outputting one of the speech direction estimation results to the first input direction correction means; and the first beamformer processing. Spectrum subtraction means for performing spectrum subtraction processing, which is nonlinear noise suppression processing, based on the output of the means and the output of the second beamformer processing means, and output from the frequency analysis means Based on the directionality index and the target sound direction output from the target sound direction estimation means, the directionality detection means calculating a directionality index based on the time difference and the amplitude difference of the incoming sound from the generated frequency component. And a spectral subtraction control means for controlling the spectral subtraction processing of the spectral subtraction means, and outputting a target audio frequency component. 5. A plurality of microphones each having directivity are arranged with their axes of the directivity direction inclined to each other,
A voice input means for obtaining the voice of the speaker as a voice signal, a frequency analysis means for frequency-analyzing the voice signal for each of the microphones of the voice input means, and outputting a frequency component for each channel, and the frequency analysis means. For the obtained frequency components of the plurality of channels, an incoming noise that suppresses voices other than the voice from the speaker direction by performing an adaptive filter process using a filter coefficient calculated so that the sensitivity outside the desired direction becomes low. First to obtain target speech component by performing suppression processing
The beamformer processing means and the frequency component of the plurality of channels obtained by the frequency analysis means are subjected to adaptive filter processing using filter coefficients calculated so that sensitivity outside the desired direction becomes low. Second beamformer processing means for suppressing the voice from the other direction and obtaining the first noise component, and frequency components of the plurality of channels obtained by the frequency analyzing means, so that sensitivity outside the desired direction becomes low. Third beamformer processing means for suppressing the voice from the speaker direction to obtain a second noise component by performing adaptive filter processing using the filter coefficient calculated in step 1; and the first beamformer processing. Noise direction estimation means for estimating the noise direction from the filter coefficient calculated by the means, and a filter calculated by the second beamformer processing means. A first target sound direction estimating means for estimating a first target sound direction from a filter coefficient; and a second purpose for estimating a second target sound direction from a filter coefficient calculated by the third beam former processing means. Sound direction estimating means and a first input direction estimated by the first target sound direction estimating means, the first input direction being the arrival direction of the target sound to be input in the first beamformer processing means. A first input direction correcting means for sequentially correcting the sound direction and / or the second target sound direction estimated by the second target sound direction estimating means; and the noise direction correcting means for estimating A second arrival direction of noise to be input in the second beamformer processing means when the generated noise direction is within a predetermined first range.
A second input direction correcting means for sequentially correcting the input direction of the light source based on the noise direction, and the third beam when the noise direction estimated by the noise direction correcting means is within a predetermined second range. Third, which is the arrival direction of noise to be input in the former processing means
Third input direction correcting means for sequentially correcting the input direction of the noise direction based on the noise direction, and whether the noise direction estimated by the noise direction estimating means has come from a predetermined first range or a predetermined second range. From one of the first and second output noises is determined as a true noise output and one of the noises is output, and at the same time, the first voice direction estimating means and the second voice direction estimating means. Effective noise determination means for determining which estimation result of the speech direction estimation means is valid and outputting one of the speech direction estimation results to the first input direction correction means; and the first beamformer processing. Spectrum subtraction means for performing spectrum subtraction processing, which is nonlinear noise suppression processing, based on the output of the means and the output of the second beamformer processing means, and the output of the frequency analysis means. Among the frequency components of each channel, based on the phase difference of the frequency components of the two channels,
Phase difference equivalent amount calculating means for calculating a phase difference equivalent amount which is an amount corresponding to a time difference between channel signals, inter-channel power ratio calculating means for calculating a power ratio between channels from frequency components of the two channels, Directionality detecting means including directionality calculating means for calculating a directionality index based on the phase equivalent amount calculated by the phase difference equivalent amount calculating means and the interchannel power ratio calculated by the interchannel power ratio calculating means; A direction subtraction control means for controlling the spectrum subtraction processing of the spectrum subtraction means based on the directionality index obtained by the directionality detection means and the target sound direction output from the target sound direction estimation means is provided. A noise component suppressing device characterized by outputting a voice frequency component. 6. A plurality of microphones, each of which has directivity, is arranged such that axes of the directivity directions are inclined with respect to each other.
A voice input means for obtaining the voice of the speaker as a voice signal, a frequency analysis means for frequency-analyzing the voice signal for each of the microphones of the voice input means, and outputting a frequency component for each channel, and the frequency analysis means. For the obtained frequency components of the plurality of channels, an incoming noise that suppresses voices other than the voice from the speaker direction by performing an adaptive filter process using a filter coefficient calculated so that the sensitivity outside the desired direction becomes low. First to obtain target speech component by performing suppression processing
The beamformer processing means and the frequency component of the plurality of channels obtained by the frequency analysis means are subjected to adaptive filter processing using filter coefficients calculated so that sensitivity outside the desired direction becomes low. Second beamformer processing means for suppressing the voice from the other direction and obtaining the first noise component, and frequency components of the plurality of channels obtained by the frequency analyzing means, so that sensitivity outside the desired direction becomes low. Third beamformer processing means for suppressing the voice from the speaker direction to obtain a second noise component by performing adaptive filter processing using the filter coefficient calculated in step 1; and the first beamformer processing. Noise direction estimation means for estimating the noise direction from the filter coefficient calculated by the means, and a filter calculated by the second beamformer processing means. A first target sound direction estimating means for estimating a first target sound direction from a filter coefficient; and a second purpose for estimating a second target sound direction from a filter coefficient calculated by the third beam former processing means. Sound direction estimating means and a first input direction estimated by the first target sound direction estimating means, the first input direction being the arrival direction of the target sound to be input in the first beamformer processing means. A first input direction correcting means for sequentially correcting the sound direction and / or the second target sound direction estimated by the second target sound direction estimating means; and the noise direction correcting means for estimating A second arrival direction of noise to be input in the second beamformer processing means when the generated noise direction is within a predetermined first range.
A second input direction correcting means for sequentially correcting the input direction of the light source based on the noise direction, and the third beam when the noise direction estimated by the noise direction correcting means is within a predetermined second range. Third, which is the arrival direction of noise to be input in the former processing means
Third input direction correcting means for sequentially correcting the input direction of the noise direction based on the noise direction, and whether the noise direction estimated by the noise direction estimating means has come from a predetermined first range or a predetermined second range. From one of the first and second output noises is determined as a true noise output and one of the noises is output, and at the same time, the first voice direction estimating means and the second voice direction estimating means. Effective noise determination means for determining which estimation result of the speech direction estimation means is valid and outputting one of the speech direction estimation results to the first input direction correction means; and the first beamformer processing. A spectrum subtraction means for performing a spectrum subtraction processing which is a non-linear noise suppression processing based on the output of the means and the output of the second beamformer processing means; Then, the power of the difference component between the frequency components of the two channels is calculated from the spectrum normalizing means for normalizing the magnitudes of the frequency components and the frequency components of the two channels normalized by the spectrum normalizing means. Directionality detection means including inter-channel spectrum difference calculation means for obtaining the power of the difference spectrum as a directionality index, directionality index obtained by the directionality detection means, and target sound output from the target sound direction estimation means. A spectrum subtraction control means for controlling the spectrum subtraction processing of the spectrum subtraction means based on the direction, and a noise component suppressing apparatus for outputting a target voice frequency component. 7. A voice input means for obtaining a voice of a speaker as a voice signal is constituted by arranging a plurality of microphones each having directivity, with axes of the directivity directions inclined to each other, and the frequency analyzing means 2. The audio signal of each microphone of the input means is frequency-analyzed to output a frequency component for each channel.
7. The noise component suppressing device according to any one of items 1 to 6. 8. The noise component suppressing apparatus according to claim 1, wherein the spectral subtraction unit divides the obtained voice frequency into frequency bands to calculate voice power for each band. A voice band power calculating means, a noise band power calculating means for calculating the noise power for each band by dividing the obtained noise frequency component for each frequency band, the voice band power calculating means and the noise band power calculating Based on the frequency band power of voice and noise obtained from the means and the output of the spectral subtraction control means,
A noise component suppressing device comprising: a band weight calculating means for obtaining a band weight coefficient; and a spectrum subtracting means for suppressing background noise by multiplying the voice signal by the band weight coefficient for each frequency band. 9. The noise component suppressing apparatus according to claim 1, wherein the spectral subtraction unit divides the obtained voice frequency into frequency bands to obtain voice power for each band. A voice band power calculation means for calculating, a noise band power calculation means for calculating the noise power for each band by dividing the obtained noise frequency component for each frequency band, and an input signal obtained from the voice input means. The frequency component of the input signal subjected to frequency analysis is divided for each frequency band, and input band power calculation means for calculating the input power for each band; and voice obtained from the voice band power calculation means and the noise band power calculation means. Noise frequency band power, input power for each band obtained by the input band power calculation means, and the spectral subtraction control A noise component characterized by comprising a band weight calculation means for obtaining a band weight coefficient based on the output of the means, and a modified spectrum subtraction means for suppressing the background noise by applying the band weight coefficient for each frequency band of the voice signal. Suppressor. 10. A step of receiving a voice of a speaker at two or more different positions to obtain respective voice signals, and a frequency analysis is performed for each channel of the voice signal corresponding to the sound receiving position for each channel. The frequency analysis step of outputting the frequency component of and the frequency component of the multiple channels obtained in this frequency analysis step is used to perform the adaptive filter processing to suppress the incoming noise other than the target voice, and to obtain the signal of the target voice component. And a first beamformer processing step for outputting a signal of a noise component by subjecting the target speech to suppression processing by adaptive filter processing using the frequency components of the plurality of channels obtained in the frequency analysis step. 2) beamformer processing step, and estimating the noise direction from the filter coefficient calculated in the first beamformer processing step Noise direction estimation step, a target sound direction estimation step of estimating a target sound direction from the filter coefficient calculated in the second beamformer processing step, and a target sound input target in the first beamformer processing step A first input direction that is a direction of arrival of the input sound direction based on the target sound direction estimated in the target sound direction estimation step, and an input in the second beamformer processing step. A second input direction correction step of sequentially correcting a second input direction, which is the arrival direction of the target noise, based on the noise direction estimated in the noise direction estimation step; and the first beamformer processing step. A spectrum that is a nonlinear noise suppression process based on the output obtained in step 1 and the output obtained in the second beamformer processing step. A spectral subtraction processing step of performing subtraction processing, a directionality detection step of calculating a directionality index based on a time difference and an amplitude difference of the incoming sound from the frequency components obtained in the frequency analysis step, and the directionality index And a spectral subtraction control step of controlling the spectral subtraction processing of the spectral subtraction processing step based on the target sound direction obtained in the target sound direction estimation step, and outputting a target audio frequency component. Characteristic noise component suppression method. 11. A voice input step of receiving a voice of a speaker at two or more different positions and outputting each as a voice signal, and performing a frequency analysis for each channel of the voice signal corresponding to the sound receiving position. A frequency analysis step of outputting frequency components for each channel, and an adaptive filter using filter coefficients calculated so that sensitivity outside the desired direction becomes low for the frequency components of the plurality of channels obtained in this frequency analysis step. A first beamformer processing step for obtaining a target speech component by performing incoming noise suppression processing for suppressing speech other than speech from the speaker direction by performing processing; and the plurality of channels obtained in the frequency analysis step. Adaptive processing using the filter coefficient calculated so that the sensitivity outside the desired direction becomes low for the frequency components of The second beamformer processing step of suppressing the voice from the speaker direction to obtain the first noise component by applying the frequency component of the plurality of channels obtained in the frequency analysis step is outside the desired direction. A third beamformer processing step of suppressing a voice from the speaker direction and obtaining a second noise component by performing adaptive filter processing using a filter coefficient calculated so as to have low sensitivity; A noise direction estimating step of estimating a noise direction from a filter coefficient calculated in the first beamformer processing step; and a first noise direction estimating step of estimating a first target sound direction from the filter coefficient calculated in the second beamformer processing step. estimation target sound direction estimation step, the second target sound direction from the filter coefficients calculated by the third beam former processing means A second target sound direction estimating step, and a first input direction, which is a direction of arrival of a target sound to be input in the first beamformer processing step, are estimated by the first target sound direction estimating means. A first input sound direction and a second input sound direction estimated in the second target sound direction estimation step, and a first input direction correction step for sequentially correcting the input target sound direction, and the noise. When the noise direction estimated in the direction correction step is within the predetermined first range, the second input direction, which is the arrival direction of the noise to be input in the second beamformer processing step, is set to the noise direction. A second input direction correcting step for sequentially correcting the noise direction, and the third beamformer processing step when the noise direction estimated in the noise direction correcting step is within a predetermined second range. In the input direction, the third input direction that is the arrival direction of the noise that is the input direction is sequentially corrected based on the noise direction, and the noise direction estimated in the noise direction estimation step is predetermined. Of the first and second output noises is determined as the true noise output based on whether the noise comes from the first range or the predetermined second range. At the same time as outputting, it is determined which estimation result of the first voice direction estimation step and the second voice direction estimation means is effective, and one of the voice direction estimation results is input to the first input direction correction means. An effective noise determination step to output, and a spectral subtraction that is a nonlinear noise suppression processing based on the outputs of the first beamformer processing step and the second beamformer processing step. A spectral subtraction processing step of performing a directional process, a directionality detection step of calculating a directionality index based on a time difference and an amplitude difference of the incoming sound from the frequency components obtained in the frequency analysis step, and the directionality index, A spectrum subtraction control step of controlling the spectrum subtraction processing in the spectrum subtraction processing step based on the target sound direction output from the target sound direction estimation step, and outputting a target voice frequency component. Noise component suppression method.