JP3105589B2 - Sound extraction method and device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a plurality of sound sources simultaneously recorded by a plurality of sensors such as a plurality of microphones. The present invention relates to a sound extraction method for extracting a sound emitted from a sound source and a sound extraction device used for implementing the method.

[0002]

2. Description of the Related Art For example, only a specific person's voice is extracted from a large number of people's voices at a party venue or the like and recorded, or is emitted from a specific device in a factory having many noise sources around. There is a demand for extracting a sound emitted from a desired sound source among a plurality of sound sources, such as extracting abnormal sounds for use in failure diagnosis, and various attempts have been made for that purpose.

[0003] The following describes a conventional technique which is a premise of the present invention (Journal of the Acoustical Society of Japan, Vol. 46, No. 7, 1990).
See "Estimation of sound source position using multiple sensors" Hitoshi Nagata et al.). FIG. 11 is a diagram schematically illustrating a positional relationship between a plurality of sound sources and a plurality of sensors arranged in a free space. In this figure, a mark represents a point sound source, and a mark represents a sensor.

As shown in FIG. 11, if there are M sensors and N point sound sources in a free space, the output signal x _m (t) of the m-th sensor is expressed by the following equation.

[0005]

(Equation 1)

[0006] Here, signal when a _n is observed by position separated by unit length n-th sound source waveform from the sound source,
r _n , _m is the distance between the nth sound source and the mth sensor, and c is the speed of sound. When the equation (1) is Fourier transformed, the following equation is obtained.

[0007]

(Equation 2)

Is a Fourier transform of the n-th sound source a _n (t), and w (t) is a time window of length T. Here, a virtual point P _i is introduced into the space. In order to focus the sensor array on this virtual point, the phase and amplitude of the spectrum X _m (ω) in the above equation are corrected using the distance between the virtual point and all the sensors, and the weight function _{Wi, m} Corrected spectrum Y _m multiplied by
(Ω) becomes Y _m (ω) = W _{i, m} r _{i, m} X _m (ω) exp (jωri _{, m} / c) (4) Further, the following equation is obtained by averaging all the sensors.

[0009]

(Equation 3)

If W _{i, m} = 1, the spectrum X _m (ω) is multiplied by the distance r _{i, m} in equation (5), so that the weight for the sound source signal located far from the sensor is large. Become. Generally, a signal arriving from a distant place has a low S / N, so that such weighting is likely to be unstable. Therefore, here, a weighting function of Wi _{, m} = 1 / ri _{, m} is used. At this time, the average value of the corrected spectrum is represented by the following equation.

[0011]

(Equation 4)

If the power of the above equation is taken, the value is small even on average since r _{i, m} −r _{n, m} = 0 for all sensors when the virtual point coincides with the nth sound source. However, if they do not match, r _{i, m} −r _{n, m} takes a different value, so that it becomes smaller on average. Therefore, by distributing a large number of virtual points in the target space and calculating the power of the averaging of the above equation at each virtual point, the sound source position can be estimated because the sound source position has a peak. Further, Z (ω, i) at that time is an estimated value of the spectrum of the sound source at that position. Therefore, the sound emitted from this sound source can be obtained by performing an inverse Fourier transform on the spectrum of this sound source.

[0013]

When the above method is adopted, when a virtual point coincides with a predetermined sound source, a sound emitted from a sound source other than the predetermined sound source is at most 1 power.
/ M (M represents the total number of sensors). For example, even if 16 sensors are arranged, 1
Although the attenuation rate is only about 2 dB and the sound emitted from a predetermined sound source is to be extracted only, the sound emitted from a sound source other than the predetermined sound source enters as a considerably large noise, and There is a problem that the degree of separation is not sufficient.

An object of the present invention is to solve the above problems and to provide a sound extraction method with good separability and an apparatus used for implementing the method.

[0015]

FIG. 1 is a flow chart showing each step of a sound extraction method according to the present invention. First, at each of a plurality of (M) sensors arranged at mutually different positions, a sound is emitted from a sound source other than a desired sound source (the n-th sound source) among the plurality (N) of sound sources, and the desired sound is generated. Each synthesized sound signal x _m recorded at each sensor by recording the sound at the time when the sound source is at rest
(T) (m = 1, 2,... M) is obtained (step (a)).

Here, as a method of detecting a point at which a sound is emitted from a sound source other than the desired sound source and at the same time the desired sound source is at rest, it is determined whether or not each sound source emits a sound. It is only necessary to be able to receive the information in some way, and thus, without being limited to a particular method, it is possible to monitor the peak occurring at each sound source position using the method described above with reference to FIG. The above “time point” can be determined by the fact that a peak exists at the position of each sound source other than the sound source to be performed and the peak at the position of the desired sound source has disappeared.

[0017] In this manner, the plurality of synthesized speech in the "time" signal _{x m (t) (m =} 1,2, ... M) has been obtained, then the plurality of the synthesized speech signal x _m (t ) (M
= 1, 2,... M) and a power spectrum are obtained (step (b)). The cross spectrum and the power spectrum correspond to the synthesized sound signals x _m
The spectrum X _m (f) of (t) is

[0018]

(Equation 5)

[0019] Here, T represents the length of the time window. And when

[0020]

(Equation 6)

Where * represents a complex conjugate, m1, m2
Represents an integer value of any of 1 to M, respectively. Where m1 ≠ m2 is called a cross spectrum and m1 = m2 is called a power spectrum. Here, when the length T of the time window is sufficiently long, the time window is divided into a plurality (I) and the spectrum of x _m (t) obtained in each of the divided time windows is represented by X _{i. , m} (f) (i =
1, 2,..., I), each of these spectra X
_{i, m} (f) (i = 1, 2,..., I) each cross spectrum and each power spectrum

[0022]

(Equation 7)

The average value W _{m1, m2} (f) of each cross spectrum and each power spectrum is calculated as

[0024]

(Equation 8)

The average value W _{m1, m2} (f) may be used instead of the cross spectrum and the power spectrum shown in the equation (9). In this case, the averaging process allows the final sound extraction Accuracy can be improved.
In the present invention, various ancillary processes for improving accuracy, such as the averaging process, may be performed simultaneously, and these cases are also included in the present invention.

Thus, a plurality of synthesized sound signals x _m
(T) Cross spectrum and power spectrum of m (1, 2,... M) (including those after averaging) W _{m1, m2}
(F) When (m1, m2 = 1, 2,..., M) is obtained, the obtained cross spectrum and each power spectrum W _{m1, m2} (f) (m1, m2 = 1, 2,.
An inverse matrix of a transfer function matrix having M) as an element and a phase delay of sound transmission between a desired sound source and each of the plurality of sensors is obtained (step c).

Here, assuming that the spectrum of the sound emitted from each sound source is An (n = 1, 2,..., N),

[0028]

(Equation 9)

The following holds. In the equation (12), H _mn is a transfer function of a sound from the n-th sound source to the m-th sensor. Therefore, H _mn (m = 1, 2,..., M) is an element of the equation (12). The matrix is called a transfer function matrix. Here, what you want to request now is An (n = 1,2, ..., N) because it is, regarded as the equation (12) and simultaneous equations and variables A _n, try looking for the solution.

The least squares solution, since obtained by multiplying the transfer function matrix by the QR decomposition on both sides of R ^-1 Q ^T,

[0031]

(Equation 10)

## EQU1 ## Where R ⁻¹ Q ^T = G
... (14)
Also

[0033]

[Equation 11]

Is an estimated value of the spectrum radiated from the nth sound source. As can be seen from equation (15), this calculation is performed for each frequency, and inverse Fourier transform of equation (15) yields a filter coefficient for estimating the n-th waveform. Here, in order to completely separate and extract the sounds emitted from the respective sound sources, it is necessary to accurately know the transfer function matrix ( H _mn ) of Expression (12).
However, in the case of actual application, it is common that the transfer function is unknown. Therefore, here, a desired sound source (here, this is set as the first sound source. The purpose of this invention is to extract only the sound emitted from (a), and a transfer function matrix replacing the equation (12) is estimated as follows.

Here, it is assumed that the position of each sensor and the desired sound source are known. If the position of the sound source is unknown, it is possible to estimate the position of the sound source using a large number of sensors as described above. The element of the n-th row and the first column of the transfer function matrix considers a phase delay and an attenuation until the sound emitted from the sound source (the first sound source) is transmitted to each sensor,
By ignoring the reverberation component generated by the sound being reflected by a wall or the like, exp (jωτ _1m ) / r
It can be approximately _{1 m} . Here, τ _1m and r _1m are the transmission time (τ _1m = r _1m / C (C represents the speed of sound)) and the distance from the sound source 1 to the sensor m ( m = 1, 2,... M), respectively. Each sensor output when the target sound source is at rest is expressed by the following equation.

[0036]

(Equation 12)

These sensor outputs m1, m2 (m1, m
The cross spectrum and power spectrum between 2 = 1, 2,... M) are

[0038]

(Equation 13)

## EQU1 ## However

[0040]

[Equation 14]

Is caused by the finite time window length.
To reduce the error, | Ap | ² to change the time multiple times
It is calculated and averaged. Here, the m1th (m1 =
1, 2,..., M ; M ≧ N) and the m2th sensor (m2 =
Considering a matrix having elements of m2 rows and m1 + 1 columns, the cross spectrum and the power spectrum between (1, 2,..., M) sensors are as follows.

[0042]

(Equation 15)

As a general rule of the matrix, (a) Even if the second and subsequent columns of the matrix are multiplied by an integer, the first row of the left inverse matrix does not change. (B) Even if the second and subsequent columns of the matrix are added, the first row of the left inverse matrix does not change. Therefore, the filter obtained by QR decomposition of this matrix is the same as the filter obtained by QR decomposition of the transfer function matrix as long as it extracts the sound emitted from the first sound source. Therefore, since the purpose here is to extract the sound emitted from the desired sound source (the first sound source in the above example), a matrix such as that shown in equation (18) for this purpose is also used in the present invention. "Transfer function matrix".

The elements in the first column of the equation (18) do not include the term of attenuation due to the distance between the first sound source and each sensor. As in the case, the weighting is performed to improve the S / N, and therefore, the term of attenuation due to distance may be included together with the time delay of sound transmission. In addition, as described above, although the reverberation component of the sound emitted from the desired sound source (here, the first sound source) is not considered in Expression (18), this reverberation component is virtually created by reflection from a wall or the like. Therefore, as described in the description of the related art, the sound (reverberation component) emitted from the virtual image source is suppressed to 1 / M, and the power of the reverberation component is originally considered. Is usually much lower than the power of the sound emitted from the first sound source, so equation (18) gives a fairly good approximation as a transfer function for the sound emitted from the first sound source.

Here, in the actual calculation, equation (9)
Cross spectrum and power spectrum W obtained by
_{m1 and m2} are substituted into Expression (18), and the least squares solution thereof, that is, an inverse matrix (G _ij ) shown in Expression (13) is obtained. Here, since the purpose is to extract only the sound emitted from the first sound source, it is not necessary to find all the elements of the inverse matrix (G _ij ), and the first row of the equation (13) Each element G
Only _1m (m = 1, 2,..., M) needs to be obtained. Therefore, "determining the inverse matrix" in the present invention refers to determining a necessary element in the inverse matrix as described above.

In step (c) shown in FIG. 1, when the inverse matrix is obtained as described above, sounds emitted from a plurality of sound sources including a desired sound source are then transmitted to a plurality of sensors (M). Each synthesized sound signal is recorded and obtained (step (d)), and each element G _1m (m = 1, 2,... M) obtained in step (c) is calculated for each synthesized sound signal.
Are applied to each of the transfer functions (step (e)), and the synthesized sound signals after the filtering are added to each other (step (f)). The signal added to each other is a signal from which a sound emitted from a desired sound source (here, the first sound source) is extracted.

In the present invention, it is not always necessary to record sounds emitted from a plurality of sound sources including a desired sound source after performing an inverse matrix operation (step (c)). For example, in step (a), the sound at the time when the sound is emitted from the desired sound source may be recorded together with the sound at the time when the desired sound source is at rest.

Next, the sound extraction device of the present invention will be described. FIG. 2 is a block diagram showing the configuration of the sound extraction device of the present invention. This sound extraction device is provided with a plurality (here, M) of sensors 1 (1), 1 (2),..., 1 (M). These sensors 1 (1), 1 (2),
.., 1 (M) are arranged at different positions to record sounds emitted from a plurality of sound sources. Each synthesized sound signal output from each of the sensors 1 (1), 1 (2),..., 1 (M) is input to the spectrum calculation means 2 and each of the filters 5 (1), 5 (2) ,..., 5 (M).

Here, it is assumed that the position of each sound source is known. However, if the position of each sound source is unknown, the position detecting means for calculating the position of each sound source is replaced by the spectrum calculating means. 2 may be arranged at the preceding stage. In the spectrum calculation means 2, a sound is emitted from a sound source other than the desired sound source among the plurality of sound sources, and the cross spectrum and the power spectrum W _{m1, m2} ( m1, m2 = 1,
, M), and the obtained cross spectrum and power spectrum W _{m1, m2} (m1, m2 = 1,
, M) are input to the inverse matrix calculation means 3. In the inverse matrix calculation means 3, the input cross spectrum and power spectrum W _{m1, m2} (m1, m2 = 1,2, ...,
M) and a transfer function matrix taking into account the delay time of sound transmission between the desired sound source of the plurality of sound sources and each of the sensors 1 (1), 1 (2),..., 1 (M) ( Equation (18)
) Is obtained, and its inverse matrix (see equation (13)) is obtained. Equation (18) is an equation when the first sound source is a desired sound source. Therefore, in this case, equation (1)
Each element G _1m (m = 1, ₁ ) in the first row of the inverse matrix (G _ij ) of 3)
,..., M) are obtained and input to the filtering control means 4.

Here, each filter5(1),5(2),
…,5(M) is configured so that each transfer function can be changed
And each element input to the filtering means 4
G_1m(M = 1, 2, ..., M) is each filter5(1),5
(2), ...,5(M)
Each filter by the filtering means 45(1),5
(2), ...,5(M) is controlled.

After the above preparations have been made, each sensor 1
In (1), 1 (2),..., 1 (M), sounds emitted from a plurality of sound sources including a desired sound source are recorded, and the filters 5 (1), 5 (2),. are added together by the adder 6 is filtering ring M). This added signal is a signal that carries only the sound emitted from the desired sound source.

Here, the spectrum calculation means 2, the inverse matrix calculation means 3, the filtering control means 4, the filters 5 (1), 5 (2),..., 5 (M), the adder 6, and the position (not shown) The detection means may be configured as a device having the respective functions in hardware, but is not limited thereto, and may be realized by realizing each function in software using a general-purpose computer or the like. Is also good.

[0053]

The sound extracting method and apparatus according to the present invention have the above-described configuration, and particularly, a transfer function until a sound emitted from a desired sound source is transmitted to each sensor is transmitted with a transmission time delay (and a transmission time delay). Theoretically, the sound other than the desired sound source can be completely canceled, and as the noise, only the reverberation component of the sound emitted from the desired sound source is the power. And remains only in a state of 1 / M attenuated, and a better S /
With N, a sound emitted from a desired sound source can be extracted.

[0054]

Embodiments of the present invention will be described below. Here, first, the number of data required for estimating the transfer function matrix,
That is, a simulation performed to obtain an indication of how many times the number of times of cross-spectrum averaging is necessary is described. The simulation conditions were: 14 sensors, 1 noise source, 10 kHz sampling frequency.
The arrangement of z, sensors and sound sources is as shown in FIG. The transfer function of this system is as shown in FIG. 4, and the reverberation time is about 0.1 second. The simulation was performed for each of the filter tap numbers. The result is shown in FIG. In this simulation, when calculating the time average of the cross spectrum, the time window is shifted by half of the total time window length and the next waveform is cut out.
Taking 24 as an example, the filter coefficients are 1024, 153
6 , 2048... The point will be updated. Regardless of the length of the number of taps, convergence is performed at an average number of about 3 to 5 times. Therefore, in the example in which the number of taps is 1024, if the pause time of the desired sound source is about 0.3 seconds, the learning of the filter ends.

Next, an experiment conducted to confirm the effectiveness of the present invention will be described. Here, about 4 [m]
As shown in FIG. 6, 16 microphones and two sound sources A and B for generating impulse and white noise were placed in a room having a reverberation time of × 4 [m] × 4 [m] of about 0.5 [s]. Placed. First, only the noise source B is driven,
After learning the filter coefficient for suppressing the disturbing sound, the sound source A was also driven at the same time to check the disturbing sound suppressing state. Figure 7 is observed waveform, the impulse generated from the sound source A with reference to FIG. 9 the process of the invention in front of the microphone 1 microphone output signal at a position just before 3 cm, 8 which processes the sound source A The result of having performed extraction is shown. As shown in FIG. 10, an S / N improvement of about 20 [dB] was observed. Note that this is the most S / S of each microphone before processing.
This is a comparison with the microphone 2 having a high N ratio.

As described above, a better S / N can be obtained as compared with the conventional method.

[0057]

As described in detail above, the sound extraction method and apparatus of the present invention have the above-mentioned configuration, and particularly, the transfer function between the desired sound source and each sensor is determined by the transmission time delay of the sound. And attenuation),
It is possible to extract a sound emitted from a desired sound source with a better S / N than in the related art.

[Brief description of the drawings]

FIG. 1 is a flowchart showing each step of a sound extraction method according to the present invention.

FIG. 2 is a block diagram illustrating a configuration of a sound extraction device according to the present invention.

FIG. 3 is a diagram showing an arrangement of sensors and sound sources in a room in a simulation of the present invention.

FIG. 4 is a diagram showing a transfer function of the room shown in FIG. 3;

FIG. 5 is a diagram showing a simulation result.

FIG. 6 is a diagram showing the arrangement of microphones (sensors) and sound sources in a room used in an experiment performed to confirm the effectiveness of the present invention.

FIG. 7 is a diagram showing a sound emitted from a sound source A shown in FIG. 6;

FIG. 8 is an observation waveform before processing observed by the microphone 1.

FIG. 9 is a diagram showing a result of extracting an impulse emitted from a sound source A using the method of the present invention.

FIG. 10 is a diagram showing the attenuation rate of a noise component.

FIG. 11 is a diagram schematically illustrating a positional relationship between a plurality of sound sources and a plurality of sensors arranged in a free space.

[Explanation of symbols]

 1 (1), 1 (2),... 1 (M) Sensor 2 Spectrum calculation means 3 Inverse matrix calculation means 4 Filtering control means 5 (1), 5 (2),... 5 (M) Filter 6 Adder

Continued on the front page (73) Patent holder 591212567 Tone Sone 4-9-5 Midorigaoka, Taishiro-ku, Sendai, Miyagi Prefecture (73) Patent holder 591212578 Kenichi Kido 543-1, Shinjicho, Midori-ku, Yokohama-shi, Kanagawa-ken (72) Invention Person Kiyoto Fujii 24-18 Yamate-cho, Aoba-ku, Sendai City Ark House 103 (72) Inventor Masato Abe 6-8-8, Takamori, Izumi-ku, Sendai City (72) Inventor Toshio Sone 4-chome Midorigaoka, Taishiro-ku, Sendai City No. 5 (72) Inventor Kenichi Kido 543-1, Shinji-cho, Midori-ku, Yokohama-shi (56) References JP-A-50-11475 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) ) G01H 3/00 G01H 17/00 G10L 15/20 G10L 21/02

Claims (2)

(57) [Claims] A sound is emitted from a sound source other than a desired sound source among a plurality of sound sources, and a sound at a time when the desired sound source is at rest is output by a plurality of sensors arranged at different positions from each other. A plurality of synthesized sound signals are obtained by recording, each cross spectrum and each power spectrum of the plurality of synthesized sound signals are obtained, and each of the obtained cross spectrum and each power spectrum is used as an element. Obtain an inverse matrix of a transfer function matrix having a function representing a time delay of sound transmission between a sound source and each of the plurality of sensors as elements, and determine sounds emitted from the plurality of sound sources including the desired sound source. Each synthesized sound signal obtained by recording with each of the plurality of sensors is added to the inverse matrix corresponding to the desired sound source and A sound extraction method, comprising: performing a filtering process using each element corresponding to each of the number sensors as a transfer function; and adding the filtered synthesized voice signals to each other. 2. A plurality of sensors arranged at different positions to record sounds emitted from a plurality of sound sources, and a desired one of the plurality of sound sources recorded by each of the plurality of sensors. A spectrum calculating means for obtaining a cross spectrum and a power spectrum of a plurality of synthesized sound signals carrying sounds at the time when the desired sound source is at rest while a sound is emitted from a sound source other than the sound source; The inverse of a transfer function matrix having, as elements, each cross spectrum and each power spectrum obtained in the above, and a function representing a time delay of sound transmission between the desired sound source and each of the plurality of sensors. Inverse matrix operation means for obtaining a matrix; and performing each filtering process on each synthesized sound signal output from each of the plurality of sensors. A plurality of transfer function variable filtering means provided in correspondence with each of the plurality of sensors; and each of the plurality of filtering means, corresponding to the desired sound source of the inverse matrix, of the plurality of sensors. Sound extraction comprising: filtering control means for setting each element corresponding to each as a transfer function; and addition means for adding each synthesized sound signal output from each of the plurality of filtering means to each other. apparatus.