JP3435357B2 - Sound collection method, device thereof, and program recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress even such omnidirectional noise as an entire room is filled therewith. SOLUTION: Outputs of microphones 1, 2 are divided 4 into each frequency bands, and varied according to the positions of the microphones 1, 2. A difference between parameter values of each sound signal reaching the microphones is detected 3, and based on the detected difference, sound sources are separated by selecting a frequency component of each sound signal, and desired sound and undesired sound are discriminated 6 from each other from the difference between both frequency characteristics, and the undesired sound is suppressed on frequency base 7 and the output is synthesized with the sound source signal. The undesired signal is suppressed also on time base by paying attention to the difference in the periodicity of the waveforms between the desired and undesired sound.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound collecting method for separating at least one sound source from a plurality of sound sources by using a plurality of microphones, an apparatus therefor, and a program recording medium.

[0002]

2. Description of the Related Art Conventionally, for example, in order to collect voice for a call or to operate a recognition process such as voice recognition or speaker recognition, noise is not mixed in the input signal and the input signal is collected in a clean state. It had to be sounded. However, when we actually collect sound or use the recognition process, it is generally difficult to collect only the target sound clearly. Therefore, a method for improving the S / N of the picked-up signal and a technique for providing noise resistance in the recognition process have been developed. In particular, for non-stationary noise with unknown frequency characteristics or noise with frequency characteristics similar to the target speech (for example, when the noise is also speech), the frequency characteristics of each sound source should be set to the difference in the position of each sound source. A method for improving the S / N of a signal by calculating and utilizing it and separating and extracting the signals of each sound source has been developed (source separation method, device and recording medium: Japanese Patent Application No. 09-2
53212, September 1996, Proceedings of the Acoustical Society of Japan 48
Pp. 9-490 1-7-13 "Examination of two sound source separation methods focusing on information between channels"). Furthermore, in this conventional method, it is possible to separate and extract a plurality of sound sources only by selecting the frequency, by dividing each frequency component of the signal into bands so fine as to include only the frequency component of one sound source.

[0003]

However, in the conventional method, since the difference in the spatial arrangement of the sound sources is utilized, it is difficult to suppress the noise having no directivity, which fills the entire room, for example. Therefore, in the present invention, by adding a function of suppressing noise having no directionality to the conventional method, noise having directionality and noise having no directionality can be suppressed, and S / S
It is possible to enhance the N improvement effect.

[0004]

According to the present invention, a plurality of microphones provided apart from each other are used, and the difference in the parameter value of each acoustic signal reaching the microphones that changes due to the positions of the plurality of microphones. Based on, the frequency component of each acoustic signal is selected, and each sound source is separated and extracted. In addition, the difference in frequency characteristics between the target voice and noise is detected, and noise without directionality is suppressed on the frequency axis. Furthermore, focusing on the difference in the periodicity of the target sound and the waveform of the noise,
Unwanted signals are suppressed even on the time axis.

In contrast to the prior art of the present invention, conventionally, only the differences in the spatial arrangement of the sound sources and the differences in the frequency characteristics of the target sound and noise were used. In the present invention, the S / N of the signal is further improved by combining both processes.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, an embodiment of the invention of claim 1 will be described. FIG. 1 shows an example of the functional configuration of this embodiment. The microphones 1 and 2 are arranged at intervals, for example, about 20 cm, and the microphones 1 and 2 collect acoustic signals from the sound sources A and B and convert them into electric signals. The output of the microphone 1 is called an L channel signal and the output of the microphone 2 is called an R channel signal. The L-channel signal and the R-channel signal are the time difference between channels / level difference detection unit 3 in the inter-channel parameter value difference detection unit 3 for each band.
01 is supplied to the band division unit 4. In the band dividing unit 4, the L channel signal and the R channel signal are respectively divided into a plurality of frequency band signals and supplied to the band-based inter-channel time difference / level difference detecting unit 302 and the sound source determination signal selecting unit 5.

In accordance with the detection outputs of the detection units 301 and 302, the sound source determination signal selection unit 5 selects any channel signal for each band as an A component or a B component.
With respect to the selected A component signal and B component signal for each band, the unnecessary component identification unit 6 detects the difference in frequency characteristics between the target sound and noise, and identifies the noise component. The identified noise component is attenuated by the unnecessary component attenuator 7. Finally, the A component signal and the B component signal for each selected band are respectively synthesized by the sound source signal synthesizing unit 8 and separated and output as the sound source signal A and the sound source signal B.

As the parameter for obtaining the time difference between channels for each band, for example, the phase difference when the signal is frequency decomposed is used. Further, as the parameter for obtaining the level difference between channels for each band, for example, the difference of power spectrum of each channel is used. As a parameter for obtaining the time difference between channels, for example, cross-correlation between channels is used. The band splitting method of the band splitting unit 4 is performed by, for example, performing discrete Fourier transform to transform into a frequency domain signal, and then splitting into frequency bands. The L channel signal and the R channel signal R are respectively band signal L ( It is divided into f1) to L (fn) and R (f1) to R (fn).

The sound source determination signal selecting section 5 uses the values detected by the inter-band parameter value difference detecting section 3 for each band signal L (f1) to L (fn) and R (f1) to R (. f
The sound source signal determination unit 501 determines which one is selected for each corresponding to (n). For example, when the sound source A is near the L-side microphone 1 and the sound source B is near the R-side microphone 2, the band-based inter-channel time difference and band-based inter-channel level difference in the i-th band are positive values ( However, the level difference between channels and the time difference for each band are obtained by subtracting the value on the R side from the value on the L side.
In such a case, the sound source signal selection unit 502 attenuates R (fi).

The unnecessary component discriminating unit 6 discriminates between the frequency component of the target sound and the unnecessary sound frequency component in the signal whose frequency is decomposed by the band dividing unit, by utilizing the difference in the respective frequency characteristics. Next, the unnecessary component attenuator 7 attenuates the frequency component identified as unnecessary. In the sound source signal synthesizer 8, the output frequency components are resynthesized into an acoustic signal and subjected to, for example, inverse Fourier transform to be returned to a time waveform. By the above processing, even in an environment in which a plurality of sounds are uttered at the same time, the S / N of the signals can be improved by separating and extracting each signal. Detailed operation examples of the unnecessary component identification unit 6 and the unnecessary component attenuation unit 7 are shown below.

The unnecessary component identification section 6 and the unnecessary component attenuation section 7
It is desirable to install it in the latter stage as much as possible because the processing amount is lighter. (For example, as shown in Figure 1,
For example, it is inserted between the sound source signal selection unit 5 and the sound source synthesis unit 8. ) This is because the number of frequency components to be processed is smaller and lighter when the unnecessary sound identification processing and the unnecessary sound attenuation processing are added only to the components selected as the frequency components of each sound source instead of all the frequency components. This is because it can be realized by processing. However, since the same function can be realized at other installation locations, the unnecessary component identification unit 6 and the unnecessary component attenuation unit 7 are also provided.
Is between the band division unit 4 and the band-based inter-channel parameter value difference detection unit 3, or between the band-based inter-channel parameter value difference detection unit 3 and the sound source signal determination unit 5 , or the sound source signal determination Any number of units may be placed anywhere such as between the unit 501 and the sound source signal selection unit 502, or between the sound source signal selection unit 502 and the sound source synthesis unit 8.

In this embodiment, the inter-channel time difference / level difference between the L-channel signal and the R-channel signal is estimated by the inter-channel time difference / level-difference detecting section 301. The inter-channel time difference / level difference of the channel signal may be known, and the value may be given to the sound source determination signal selection section 5 as data. next,
The unnecessary component identification unit 6 and the unnecessary component attenuation unit 7 will be described.

Here, as an example, a method will be described in which the unnecessary component identifying unit 6 and the unnecessary component attenuating unit 7 are provided between the band dividing unit 4 and the band-based inter-channel parameter value difference detecting unit 3. First, the flow of the processing is shown in FIG. 2, and description will be given according to the flow. In this embodiment, noise suppression is performed by utilizing the difference in the statistical properties of the target sound and noise. Therefore, as an example, the signal to be suppressed will be described as a wideband signal. Here, it is necessary to determine the wideband component from the frequency components having a power equal to or higher than a certain value. First, the band division unit 4
The frequency component is received from (2-01). For each of the received frequency components, the power (Pow (k
Take a histogram for k)) (2-02). The histogram counts the number of bands within which the power falls within a certain range. An example of the frequency characteristic when the signal does not include wideband noise is shown in FIG. 3, and an example of the histogram is shown in FIG. 4A.
4B shows the power in that power section. further,
An example of frequency characteristics of a signal including wideband noise is shown in FIG.
FIG. 6 shows an example of the histogram.

If broadband noise is not included, FIG.
As shown in FIG. 4, the components other than the harmonic structure have small power, and therefore, in the histogram of FIG.
It is limited to a section where the power is very small (section 1 in FIG. 4A). However, when broadband noise is included,
As shown in, there are many bands having relatively large power other than the harmonic components of the signal. Therefore, FIG.
Also in the histogram, the frequency of a section having a relatively large power out of the sections having a power smaller than the power of the harmonic structure is prominent, for example, in the shaded area (section 3).

Therefore, when the number of bands within a predetermined range (power section) (section 3 in FIG. 6) exceeds a predetermined value (threshold value) x (2-03), an unnecessary component is included. It is determined that it is present (2-04), and the unnecessary component attenuation process is started (2-05). In the unnecessary component attenuation process (2-05), the band having the power of the section where the histogram is prominent (section 3 in the case of FIG. 6) is attenuated. As a method of attenuating, for example, a predetermined value α (0 <
= Α <1) is multiplied by the frequency component of the band to be attenuated.
Here, the value of α may be the same value for all bands or may be changed for each band. Further, the band to be attenuated may be not only the band included in the section in which the histogram is protruding, but also all the bands having the same or lower power as the section in which the histogram is protruding. With the above method, the unnecessary component identifying unit 6 can determine the presence or absence of wideband noise, and the unnecessary component attenuating unit 7 can attenuate the wideband noise component.

Here, as a caution, the unnecessary component identifying unit 6 and the unnecessary component attenuating unit 7 are provided between the band dividing unit 4 and the band-based inter-channel parameter value difference detecting unit 3, or the band-based inter-channel parameter value difference. Where between the detection unit 3 and the sound source signal determination unit 501, between the sound source signal determination unit 501 and the sound source signal selection unit 502, or between the sound source signal selection unit 502 and the sound source synthesis unit 8. , You can put any number. For example, comparison will be made between the case where it is placed between the sound source signal selection unit 502 and the sound source synthesis unit 8 and the case where it is placed between the band division unit 4 and the above-mentioned band-by-band parameter value difference detection unit 3. As an advantage of the former, there is an advantage that the number of bands to be identified is reduced because only the ones already selected as the frequency components of each sound source are identified and the processing is lightened.

Next, another embodiment will be described. In this embodiment, the target sound is distinguished from the target noise by utilizing the harmonic structure of the target sound to suppress the noise. Therefore, as an example, the target of noise is periodic noise or quasi-periodic noise (for example, a voice or a bark of an animal). In this case, the sound source signal determination unit 5
The harmonic structure of the sound source signal is estimated from the component determined in 01 (this mainly includes the frequency component of each sound source), and a component having a large power in frequency components other than the structure is determined to be noise. FIG. 7 shows the processing procedure. The procedure will be described below.

Here, the case where the unnecessary component identifying unit 6 and the unnecessary component attenuating unit 7 are placed between the sound source signal selecting unit 502 and the sound source signal synthesizing unit 8 will be described. First, the sound source signal selection unit 50
The frequency component selected from 2 is received (7-01).
With respect to the received frequency components, a band having the highest power (LL (f)) is extracted in descending order by a predetermined number (7-02). Next, the fundamental frequency is selected from these bands. That is, since the fundamental frequency of human voice exists within about 100 Hz to 250 Hz,
Of the bands selected in step 7-02, for example, 100H
Only those that enter 250 Hz from z are selected (7-0
3). Next, the band (LLs selected in step 7-03)
In order to estimate the fundamental frequency from (f)), the following processing is performed. That is, the power of the harmonic is added to each of the selected bands (LLs (f)). The added power is set to SumLLs (f) (7-04). LL where this added value SumLLs (f) is the largest
It is determined that s (f) is the fundamental frequency (7-05).

Next, it is determined whether or not a band mm having a large power exists in the components other than the determined fundamental frequency and its harmonics (7-06). In this determination, for example, if there is a band having a power exceeding a predetermined value x in a band other than the fundamental frequency and its harmonic component, the band is determined to be an unnecessary component, and if there is no unnecessary component, To determine. When it is determined that the unnecessary component is present, the unnecessary component attenuator 7 attenuates the power of the band mm determined to be unnecessary (7-07). This damping method is, for example,
The power in the band mm is multiplied by a certain value α (0 <= α <1). Here, with the same caution as in the case of the embodiment shown in FIG. 2, the unnecessary component identifying unit 6 and the unnecessary component attenuating unit 7 are provided between the band dividing unit 4 and the band-based inter-channel parameter value difference detecting unit 3. Alternatively, the band-by-band inter-channel parameter value difference detection unit 3
And the sound source signal determination unit 501, or between the sound source signal determination unit 501 and the sound source signal selection unit 502, or
Any number of the sound source signal selection units 502 and the sound source synthesis unit 8 may be placed anywhere. However, when placed only between the band division unit 4 and the band-based inter-channel parameter value difference detection unit 3 or between the band-based inter-channel parameter value difference detection unit 3 and the sound source signal determination unit 501, Since the components of the sound source have not yet been determined, the accuracy of estimating the harmonic structure of each sound source may deteriorate.

Finally, another embodiment will be described. Here, unnecessary sound is identified in the region of the time waveform, and the identified signal is suppressed. FIG. 8 shows the processing procedure. Here, a case where the unnecessary sound identification unit and the unnecessary sound attenuating unit are provided between the plurality of microphones 1 and 2 and the band dividing unit 4 will be described. First, the signal is read for a certain section length (8-01). In FIG. 8, as an example, a case of reading the frame length for the Fourier transform will be described. Next, it is determined whether the read signal is in the voice section. The method of the determination is, for example, calculating the autocorrelation function of the signal and obtaining the peak value (8-
02). Whether or not there is a peak is determined, for example, when the difference between the maximum value and the second largest value of the autocorrelation function exceeds a predetermined threshold value, the peak exists,
If it does not exceed the peak, the peak does not exist (8-03). If there is a peak, it is determined that the signal section is a voice section and is sent to the next band division unit 4 (8
-04). When the peak is not detected, the signal section is determined to be the unnecessary sound section, is sent to the unnecessary sound attenuator, and the amplitude of the signal is attenuated by a predetermined amount (8-0.
5). After being attenuated, it is sent to the band dividing unit 4, which is the next process. Further, the unnecessary sound identifying unit and the unnecessary sound attenuating unit may be placed after the sound source synthesizing unit 8 or both.

The signal whose noise is suppressed by the method described above can be used not only for the purpose of being directly heard by humans, such as for a call, but also as an input signal of a voice recognition device or a speaker recognition device. . According to the present invention, the signal S / N ratio can be improved and a clear signal can be obtained. It is not limited to the case where each unit described above is configured by hardware, and each function can be executed by reading and decoding and executing a program by a computer. Also, the method shown in FIG. 2 or 7 and the method shown in FIG. 8 can be used together.

[0022]

As described above, according to the present invention, the unnecessary sound discriminating unit discriminates unnecessary components such as voice and noise from the target signal, and the unnecessary sound components are attenuated by the unnecessary sound attenuating unit. By doing so, it is possible to suppress even non-directional noise that fills the entire room, improve the S / N of the signal, and obtain a clear signal.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of an embodiment of the present invention.

FIG. 2 is an unnecessary component identification unit 6 and an unnecessary component attenuation unit 7 in FIG.
6 is a flowchart showing an example of the processing procedure of FIG.

FIG. 3 is a diagram showing an example of frequency characteristics when there is no broadband noise.

4A is a diagram showing an example of a histogram when there is no broadband noise, and FIG. 4B is a diagram showing an example of values in each power section.

FIG. 5 is a diagram showing an example of frequency characteristics when there is wideband noise.

FIG. 6 is a diagram showing an example of a histogram when there is wideband noise.

7 is a flowchart showing another example of the processing procedure of the unnecessary component identification unit 6 and the unnecessary component attenuation unit 7. FIG.

FIG. 8 is a flowchart showing still another example of the processing procedure of the unnecessary sound identification unit 6 and the unnecessary sound attenuation unit 7.

─────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-4-245300 (JP, A) JP-A-9-153769 (JP, A) JP-A-7-295590 (JP, A) JP-A-60- 35797 (JP, A) European Patent Application Publication 831458 (EP, A 1) (58) Fields investigated (Int.Cl. ⁷ , DB name) G10L 21/02

Claims (5)

(57) [Claims] 1. A sound pickup method for separating at least one sound source from a plurality of sound sources by using a plurality of microphones provided apart from each other, wherein each output channel signal of each microphone is divided into a plurality of frequency bands. The band division process of dividing into the following, and for each same band of each output channel signal divided in the band division process, the parameter value of the acoustic signal reaching the microphone that changes due to the positions of the plurality of microphones The difference between the band-by-channel parameter value difference detection process for detecting the difference as a band-by-channel parameter value difference, and which of the band-divided output channel signals is based on the band-by-band parameter value difference between the bands. From the sound source signal determination process of determining whether to select, and from each output channel signal band-divided based on the determination result, It has a sound source signal selection process of selecting at least one signal input from one sound source, and a sound source synthesis process of synthesizing a plurality of band signals selected as signals from the same sound source in the sound source signal selection process as a sound source signal. In the method, between the band division process and the band-based inter-channel parameter value difference detection process, the band-based inter-channel parameter value difference detection process and the sound source signal determination process, the sound source signal determination process and the sound source During the signal selection process, or at least one of the sound source signal selection process and the sound source synthesis process, selection among frequency components of the output channel signals divided in the band division process is performed. The unnecessary component identification process that identifies unnecessary frequency components other than the signal from the sound source to be performed, and the unnecessary component that attenuates the frequency components identified in the unnecessary component identification process described above. Chi lifting the decay process, the unnecessary component identifying process, a signal from a sound source to be selected
By focusing on the statistical properties of unwanted signals,
A sound collection method characterized by identifying frequency components. 2. A plurality of micros provided separately from each other.
At least one sound source from multiple sound sources
This is a sound collection method that separates each output channel signal of each microphone above.
Band division process of dividing into wave number bands and each of the output channel signals divided in the above band division process
Due to the positions of the above microphones in the same band
Parameter of the acoustic signal that reaches the changing microphone
The difference between the values of the
Process of detecting parameter value difference between channels
Based on the parameter value difference between channels for each band above.
Which of the above-mentioned band-divided output channel signals
A sound source signal determination process for determining whether to select, and each output channel divided into the above-described bands based on the determination result.
Signal from the same sound source,
Select a signal from the same sound source in the sound source signal selection process to select one and the above sound source signal selection process
Sound source combination that synthesizes multiple generated band signals as sound source signals
And a parameter value between channels for each band,
During the difference detection process, inter-channel parameter values for each band
During the difference detection process and the sound source signal determination process, the sound source signal
Between the signal determination process and the sound source signal selection process,
At least one of the signal selection process and the sound source synthesis process
During one, the unnecessary component identification process is the frequency component selected by the sound source signal selection process.
, LLs having a large power within a predetermined frequency band
The process of selecting (f) and the power of each harmonic component of the selected component LLs (f).
, And the component LLs (f) that maximizes the addition power
No need for components other than the above fundamental frequency and its harmonic components
It is characterized by having the process of distinguishing from the frequency components of various signals
And sound collection method. 3. A sound pickup device for separating at least one sound source from a plurality of sound sources by using a plurality of microphones provided apart from each other, wherein each output channel signal of each of the microphones has a plurality of frequency bands. And a band dividing means for dividing into the same band of each output channel signal divided by the band dividing means, of the value of the parameter of the acoustic signal reaching the microphone that changes due to the positions of the plurality of microphones. The difference between the band-by-channel parameter value difference detecting means for detecting the difference as a band-by-channel parameter value difference, and based on the band-by-channel parameter value difference of each band, which of the band-divided output channel signals A sound source signal determining means for determining whether to select, and the same from the output channel signals divided into the above bands based on the determination result. A sound source signal selecting means for selecting at least one signal input from a sound source, and a sound source synthesizing means for synthesizing a plurality of band signals selected as signals from the same sound source by the sound source signal selecting means as a sound source signal. In the device, between the band dividing means and the band-based inter-channel parameter value difference detecting means, between the band-based inter-channel parameter value difference detecting means and the sound source signal determining means, the sound source signal determining means and the sound source Between the signal selecting means and at least one of the sound source signal selecting means and the sound source synthesizing means, one of the frequency components of each output channel signal divided by the band dividing means should be selected. Unnecessary component identification means for identifying unnecessary frequency components other than the signal from the sound source, and unnecessary component reduction for attenuating the frequency component identified by the unnecessary component identification means. And means, the unnecessary component identifying means, to each signal that is the band division
Next, create a histogram for the power of each band
Means and whether or not the frequency of a given power section is above a given value
And a means for identifying unnecessary components
Sound pickup device. 4. A plurality of micros provided separately from each other.
At least one sound source from multiple sound sources
A sound pickup device that separates each output channel signal of each microphone described above.
Band dividing means for dividing into wave number bands, and each of the output channel signals divided by the band dividing means
Due to the positions of the above microphones in the same band
Parameter of the acoustic signal that reaches the changing microphone
The difference between the values of the
Parameter value difference detection means for each band
Based on the parameter value difference between channels for each band above.
Which of the above-mentioned band-divided output channel signals
Sound source signal determining means for determining whether to select, and the output channels for which the band division is performed based on the determination result.
At least one signal input from the same sound source
Sound source signal selecting means for selecting one of the two, and the sound source signal selecting means selects as a signal from the same sound source
Sound source combination that synthesizes multiple generated band signals as sound source signals
An apparatus for and a forming means, said band division means and the band-by-band channel between the parameter values
Between the difference detection means, the inter-channel parameter value for each band
Between the difference detection means and the sound source signal determination means, the sound source signal
Between the sound source signal selecting means and the sound source signal selecting means.
At least one between the signal selection means and the sound source synthesis means.
In between, each of the output channel signals divided by the band dividing means
Of the frequency components, signals other than the signal from the sound source
An unnecessary component identifying means for identifying a necessary frequency component and an unnecessary component attenuating means for attenuating the frequency component identified by the unnecessary component identifying means are provided, and the unnecessary component identifying means is selected by the sound source signal selecting means. Means for selecting a component LLs (f) having a large power in a predetermined frequency band from the selected frequency components, a means for adding the power of each harmonic component of the selected component LLs (f), and the addition thereof. A sound collecting device comprising: a component LLs (f) other than the fundamental wave component LLs (f) having the maximum power; and a means for discriminating the harmonic component thereof from the frequency component of an unnecessary signal. 5. Each of the sound collecting methods according to claim 1 or 2.
Write a program to make the computer execute the process.
Recorded computer-readable recording medium.