JP7316614B2 - Sound source separation device, sound source separation method, and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act [1] Date of issue (publication date) September 3, 2019 (Online Proceedings full text download: September 3, 2019 13:00 to September 4, 9:00) ( URL: https://ac.rsj-web.org/2019/ ), Publications The 37th Annual Conference of the Robotics Society of Japan (RSJ 2019) Proceedings The Robotics Society of Japan <Reference> Research papers published in the proceedings (excerpts) [ 2] Date: September 3, 2019 to September 7, 2019 (Released date: September 4, 2019) Meeting name, venue: The 37th Annual Meeting of the Robotics Society of Japan Sponsor: The Robotics Society of Japan, Waseda University, Waseda Campus <Documents> Overview of the 37th Annual Meeting of the Robotics Society of Japan Web page printout <Documents> Program of the 37th Annual Meeting of the Robotics Society of Japan (excerpt) <Documents> Materials for the 37th Annual Meeting of the Robotics Society of Japan (slides) [3] Publication date (released) Sunday) January 12, 2020 (Honolulu, Hawaii local time and Japan time) Publications Proceedings of the 2020 IEEE/SICE International Symposium on System Integration (SII2020), IEEE <References> Published research papers (excerpts) [4] Held Date January 12, 2020 to January 15, 2020 (Honolulu, Hawaii local time) January 12, 2020 to January 16, 2020 (Japan time) Meeting name, Venue International Symposium: 2020 IEEE / SICE International Symposium on System Integration (SII2020) Sponsored by IEEE Robotics & Automation Society Hawaii Convention Center, Honolulu, Hawaii, USA Program release URL: h ttps://ras. papercept. net/conferences/conferences/SII20/program/SII20_ContentListWeb_3. html <Reference> International Symposium (SII2020) Overview Web page printout <Reference> International Symposium (SII2020) program and abstracts of published research (excerpts) <Reference> International Symposium (SII2020) research oral presentation materials (slides)

The present invention relates to a sound source separation device, a sound source separation method, and a program.

By performing processing such as beamforming on acoustic signals collected by a microphone array, it is possible to perform sound source separation that extracts only a specific sound source from an observed signal in which multiple sound sources are mixed (for example, Patent Document 1). reference).

In these sound source separation processes, the theory is built on the premise that the sound source is a point sound source. Since a normal sound source is a surface sound source, conventionally, the surface sound source is treated as a point sound source and subjected to separation processing. Conventionally, methods of widening beams (directivity) such as delay-and-sum beamforming and echo cancellation have been used to pseudo-separate surface sound sources.

JP 2015-46759 A

However, with the conventional technology, it is difficult to finely control the directivity. For example, there are problems such as extracting a sound source in an area to be suppressed, and not being able to deal with a sound source having a square surface shape or a more complicated shape.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a sound source separation device, a sound source separation method, and a program capable of separating a surface sound source.

(1) In order to achieve the above object, a sound source separation device according to an aspect of the present invention has N (N is an integer equal to or greater than 2) microphones arranged at a first interval for picking up acoustic signals. A microphone array and a desired area are subdivided at a second interval, and for each of the subdivided areas, acoustic signals picked up by the microphone array are transmitted to sub-beams corresponding to the azimuth angles θ of the subdivided areas. and a sound source separation unit that separates and extracts by a beamforming method using , and separates the acoustic signal of the desired region by adding the extracted acoustic signals.

上式において、Ｄ（θ）は音源の開始方位角θ_ｂ以上源の終了方位角θ_ｅ以下の場合に１であり他の方位角で０である所望の領域の音響信号を分離するための理想的なフィルタであり、Ｐ_θｉはθ_ｉ方向にビームの指向性を向けた場合の点音源を対象としたビームフォーマの指向特性であり、θ_ｉは各ビームフォーマの指向性の方向であり、Ｈはエルミート共役を意味し、ａ（θ）は音源か前記マイクロホンへの伝達関数であり、Ｗ_θｉはビームフォーマＰ_θｉの係数ベクトルであり、この場合の面音源のビームフォーマの係数Ｗは次式であり、

Ｄ（θ）を次式としたとき、

θ_ｉ＝θ_ｂ＋（ｉ－１）θ_ｓｃａｎと定義でき、ｂはθ_ｉ＝θ_ｂとなるｉの値であり、ｅはθ_ｉ＝θ_ｅとなるｉの値を示す、ようにしてもよい。 (2) Further, in the sound source separation device according to one aspect of the present invention, the pattern of the sub-beams is represented by the following equation,

In the above equation, D(θ) is 1 when the starting azimuth angle _θb of the sound source is equal to _or less than the ending azimuth angle θe of the source, and is 0 at other azimuth angles. This is an ideal filter, P _θi is the directional characteristic of a beamformer targeting a point sound source when the directivity of the beam is directed in the _θi direction, and _θi is the direction of the directivity of each beamformer. , H means the Hermitian conjugate, a(θ) is the transfer function to the sound source or said microphone, W _θi is the coefficient vector of the beamformer P _θi , where the coefficients W of the beamformer for the surface sound source are and

When D(θ) is given by the following formula,

It can be defined as θ _i =θ _b +(i−1) θ _scan , where b is the value of i that satisfies θ _i =θ _b , and e is the value of i that satisfies θ _i =θ _e . good too.

(3) Further, in the sound source separation device according to an aspect of the present invention, an evaluation unit that calculates a relationship among a cost function J, the number of microphones, and the second interval; A selecting unit that selects the number of microphones that minimizes the cost function and the second spacing in the relationship between the cost function, the number of microphones, and the second spacing. good.

次式を用いて前記コスト関数Ｊを算出し、算出した前記コスト関数Ｊと、前記マイクロホンの数Ｎと、スキャン角度θ_ｓｃａｎに基づいて、前記所望の領域の音響信号を分離するための最適な前記マイクロホンの数Ｎと、スキャン角度θ_ｓｃａｎを求め、

上式において、αは所定値であり、λ_１、λ_２それぞれは調整パラメーターである、ようにしてもよい。 (4) In the sound source separation device according to an aspect of the present invention, the evaluation unit calculates the difference between the beam pattern and the ideal pattern using the logarithmic mean square error MSE of the following formula,

Calculate the cost function J using the following equation, and based on the calculated cost function J, the number N of the microphones, and the scan angle θ _scan , the optimum for separating the acoustic signal of the desired region Obtaining the number N of the microphones and the scan angle θ _scan ,

In the above formula, α may be a predetermined value, and λ ₁ and λ ₂ may be adjustment parameters.

(5) Further, in the sound source separation device according to the aspect of the present invention, the evaluation unit expresses the cost function J, the number N of microphones, and the scan angle θscan in a three-dimensional graph, and the cost function J is By selecting the minimum number N of microphones and the scan angle θscan, the optimum number N of microphones and the optimum scan angle θscan are obtained. It may be updated to the number N of microphones and the scan angle θscan.

(6) In order to achieve the above object, a sound source separation method according to an aspect of the present invention provides a microphone array having N (N is an integer equal to or greater than 2) microphones arranged at a first interval to generate an acoustic signal. Sound is picked up, a sound source separation unit subdivides a desired region at a second interval, and the sound source separation unit converts an acoustic signal to each of the subdivided regions into an azimuth angle θ of the subdivided region. are separated and extracted by a beamforming method using sub-beams corresponding to , and the sound source separation unit separates the acoustic signal of the desired region by adding the extracted acoustic signals.

(7) To achieve the above object, a program according to an aspect of the present invention causes a computer to transmit an acoustic signal to a microphone array having N (N is an integer equal to or greater than 2) microphones arranged at a first interval. picking up sound, subdividing a desired region at a second interval, and for each of the subdivided regions, using sub-beams corresponding to the azimuth angle θ of the subdivided region to perform the acoustic beam forming method By separating and extracting the signals and adding the extracted acoustic signals, the acoustic signal of the desired region is separated.

According to (1) to (7) described above, it is possible to appropriately suppress a sound source in a region to be suppressed, and to separate a plane sound source.
According to (2) above, the sub-beamformer is used for each subdivided region, and the acoustic signal of the subdivided region by the beamforming method can be extracted from the collected acoustic signal. A desired surface sound source can be separated by adding the obtained acoustic signals.
According to (3) to (5) described above, using the cost function J makes it possible to select the optimal number of microphones for extracting a surface sound source and the interval for dividing a desired region.

1 is a block diagram showing a configuration example of a sound source separation system according to an embodiment; FIG. FIG. 4 is a diagram showing an example of arrangement of microphones according to the embodiment; FIG. 3 is a diagram for explaining a beamformer used in an embodiment; FIG. FIG. 10 is a diagram showing the relationship between the setting of the microphone array used for evaluation and the surface sound source of the separation target; This is a setting example for extracting a plane sound source of a separation target and suppressing surrounding noise. FIG. 10 is a diagram illustrating an example beamformer for adaptive MVDR; FIG. 7 is an MSE surface when the adaptive MVDR beamformer of FIG. 6 is used. Fig. 2 shows a DS beamformer; 9 is an MSE curved surface when the DS beamformer of FIG. 8 is used. FIG. 5 is a diagram showing an example of setting a plane sound source as a separation target; FIG. 10 is a diagram showing the MSE surface of a DS scan-and-sum beamformer with a sub-beamformer; Fig. 3 shows the cost surface of the DS scan-and-sum beamformer with the sub-beamformer; FIG. 10 is a diagram showing pattern examples of winner beamforming and scan-and-sum beamformer; FIG. 10 is a diagram showing the MSE surface of the MVDR scan-and-sum beamformer with sub-beamformer; Fig. 3 shows the cost surface for the sub-beamformer MVDR scan-and-sum beamformer; FIG. 11 shows the pattern of the synthesized MVDR scan-and-sum beamformer with recommended settings; FIG. 10 is a diagram showing an example of a beam pattern when low-density scanning is selected; FIG. 10 is a diagram showing evaluation results for three different buffer sizes; It is a figure which shows the example of the separated surface sound source which concerns on embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In addition, in the drawings used for the following description, the scale of each member is appropriately changed so that each member has a recognizable size. Also, in the present embodiment, the acoustic signal in the specific area is called a surface sound source.

FIG. 1 is a block diagram showing a configuration example of a sound source separation system 1 according to this embodiment. A sound source separation system 1 includes a sound pickup unit 2 and a sound source separation device 3 .
The sound pickup unit 2 includes N (N is an integer equal to or greater than 2) microphones 21-1, . . . , and microphones 21-N. . . , and 21-N are referred to as microphones 21 unless otherwise specified.
The sound source separation device 3 includes an acquisition unit 31 , a transfer function storage unit 32 , a beam pattern storage unit 33 , a sound source separation unit 34 and an output unit 35 .
The sound source separation unit 34 includes a separation unit 341, an evaluation unit 342, and a selection unit 343 (evaluation unit).

The sound pickup unit 2 is a microphone array including n microphones 21 . The sound pickup unit 2 picks up an acoustic signal emitted by a sound source, and outputs the picked-up n-channel acoustic signal to the acquisition unit 31 . The arrangement of the microphones 21 will be described later.

The acquisition unit 31 acquires the analog n-channel acoustic signal output from the sound pickup unit 2 and converts the acquired analog acoustic signal into a digital acoustic signal. The acoustic signals output from the n microphones 21 of the sound pickup unit 2 are sampled using signals having the same sampling frequency. The acquisition unit 31 outputs the digitally converted acoustic signal to the sound source separation unit 34 .

The transfer function storage unit 32 stores, for each microphone 21 included in the sound pickup unit 2, a transfer function modeled as a function having the direction of arrival as an argument.

The beam pattern storage unit 33 may store sub-beam patterns. The sub-beam pattern will be described later.

The sound source separation unit 34 separates the acoustic signal of the desired region and outputs the separated acoustic signal of the desired region to the output unit 35 . The sound source separation unit 34 evaluates the beam pattern used for separation. The sound source separation unit 34 selects the number of microphones 21 and the intervals for dividing the desired region based on the evaluation result. Note that the desired region is a region including a region in which a surface sound source to be separated exists.

The separation unit 341 subdivides the desired area at equal intervals. The sound source separation unit 34 uses a sub-beamformer for each subdivided region to extract the acoustic signal of the subdivided region by the beamforming method from the collected acoustic signal. The sound source separation unit 34 separates a desired surface sound source by adding acoustic signals extracted for each subdivided area. Note that the separation unit 341 sets the number of microphones 21 and the interval for dividing the desired region to initial values stored therein. The separation unit 341 may update the number of microphones 21 and the interval for dividing the desired region based on the selection result output by the selection unit 343 .

The evaluator 342 evaluates the quality of the selected beam pattern using MSE (Mean Square Error) with a cost function J that balances performance and cost. The evaluation unit 342 outputs the evaluated evaluation result to the selection unit 343 . Note that the cost function J, MSE, and evaluation method will be described later.

The selection unit 343 selects the number of microphones 21 and the interval for dividing the desired region based on the evaluation result evaluated by the evaluation unit 342 . As will be described later, the selection unit 343 represents the cost function J, the number of microphones 21, and the intervals for dividing the desired area in a three-dimensional graph, and detects the minimum value in this graph to determine the number of microphones 21, the desired Select the interval to divide the regions. The selection unit 343 outputs the selected entrustment result to the separation unit 341 .

The output unit 35 is, for example, a speaker. The output unit 35 outputs the acoustic signal of the desired region separated by the sound source separation unit 34 .

<Example of placement of microphones>
Next, an arrangement example of the microphone 21 will be described.
FIG. 2 is a diagram showing an arrangement example of the microphones 21 according to this embodiment. The N microphones 21 are arranged at equal intervals (first intervals) every d, in the order of 21-1, 2-2, . It is arranged on the straight line g11. The microphone 21-1 at one end is used as a reference microphone. Note that the number N of the microphones 21 and the spacing d may be selected according to, for example, the cost to be described later.

All of the N microphones 21 are omnidirectional microphones. The distance between the sound source and the microphone 21 is desirably the distance at which the signal from the sound source is input to the microphone 21 as a plane wave.
A symbol g12 represents a plane wave input to the microphone 21 . The azimuth angle θ is the angle between the straight line g11 on which the microphone 21 is arranged and the input plane wave g12, and is assumed to increase counterclockwise. The sound speed of sound waves is represented by c (340.29 m/s).

A signal from a sound source is represented by a phase-shift vector represented by a(θ, f)εC ^N×1 (where C is a set of all complex numbers) represented by the following equation (1): is propagated to the sound pickup unit 2, which is a microphone array, through the transfer function. This transfer function is stored in the transfer function storage unit 32 .

In Equation (1), T represents a transposed matrix. τ is the delay time between two adjacent microphones 21 in the microphone array, and τ=(d·cos θ)/c. Also, f is the different frequency components of the input signal, where f=ω/2π. In the following description, f is omitted for simplification of description.

The i-th received point source is Z _i =a _i S _i . where a _i is the transfer function focused on the direction of arrival (DOA) of the i-th point source. Spatial white noise, such as microphone thermal noise, is represented as VεC ^N×1 .
Also, the mixed signal ZεC ^N×1 can be expressed by the following equation (2).

In equation (2), S=[S ₁ , S ₂ , . . . , S _Nsig ] ^T , where N _sig is the number of signals. A is a multi-channel audio signal for mixing individual point sources, and A=[a ₁ , a ₂ , . . . , a _Nsig ] is a mixing matrix.

<Basic formula of point source beamformer>
Next, the basic equations of the point sound source beamformer will be described.
The beamformer is represented by a coefficient vector WεC ^Nch×1 . The directional frequency response of the beamformer is a "pattern" defined as in Equation (3) below.

In equation (3), H means Hermitian conjugate, and θ _DOA represents the direction of arrival of the target sound source (focus DOA). Further, the subscripted DOA represents a design parameter of the direction of arrival of the focus. a(θ) is obtained by omitting f in the transfer function of equation (1). The variable represents the input direction of the signal. The selection of signal inputs from different directions is directivity in the beamformer.

<Scan-and-sum (SCAN-AND-SUM) beamformer>
Next, the scan-and-sum beamformer used in this embodiment will be described.
In this embodiment, instead of designing a planar beam pattern with a single beamformer, the sub-beamformer of the point sound source is changed in the focus arrival direction, and the area where the target planar sound source exists is scanned at an appropriate scan angle. In this embodiment, this sub-beamformer is integrated into a plane beamformer. In this embodiment, this method is called a scan-and-sum beamformer method (or scan-and-sum method).

FIG. 3 is a diagram for explaining the beamformer used in this embodiment. In FIG. 3, the horizontal axis is the direction of arrival θ [degrees], and the vertical axis is the gain (20log|P _θDOA |) [dB]. A pattern g21 is a beam pattern of MVDR (Minimum Variance Distortionless Response) scan of a comparative example. A pattern g22 is a beam pattern of the scan-and-sum method used in this embodiment. The pattern g23 is an ideal beam pattern for a plane sound source. In the case of MVDR, sound source separation is performed by obtaining a separation matrix that minimizes the output power under linear constraint conditions that do not distort the target sound source. Moreover, the beam pattern of the scan-and-sum method shown in FIG. 3 is an example, and is not limited to this.
Note that the scan-and-sum method is based on a model that a surface sound source can be decomposed into a combination of many point sound sources.

As an example of parameter settings, all pattern and analysis analyzes were performed with, for example, d=2 cm, N=20, f=2 kHz. Therefore, the length of the microphone array is 40 cm.

<Formula of scan-and-sum beamformer>
Next, the equations used in the scan-and-sum beamformer are described.
The ideal pattern D(θ) is in azimuth dimension as in Equation (4) below.

In equation (4), θ _b is the starting azimuth angle of the surface sound source, and θ _e is the ending azimuth angle of the surface sound source. The beamformer pattern of a known point source focused on a set of DOAs (θ _i ) is denoted as P _θi . where θ _i is the directivity direction of each beamformer and θ _i =θ _b +(i−1) θ _scan . θ _scan is the scan angle (second interval). Also, b is the value of i that satisfies θ _i =θ _b , and e indicates the value of i that satisfies θ _i =θ _e .
The formula of the scan-and-sum beamformer used in this embodiment is represented by the following formula (5).

In equation (5), P _θi is the directional characteristic of the beamformer at θ _i . P _θ is the directional characteristic of a beamformer intended for a point sound source when the directivity of the beam is directed in the θ direction. Note that the maximum response was normalized to 0 dB in the embodiment.
From equation (5), since P(θ) is a function obtained by adding eb+1 P _θi (θ), it is a function of _{θ scan} . θ), the scan-and-sum beamformer can also be expressed by the following equation (6).

In Equation (6), P _θi (θ) (i=1, ₂ , . As shown in Equation (5) or Equation (6), the scan-and-sum beamformer achieves a beam pattern that is close to the ideal pattern by synthesizing patterns of multiple sub-beams. Note that the sub-beam pattern is a pattern for each predetermined azimuth angle, as indicated by the dashed line g211 in FIG. Also, _NS is eb+1.
Further, from the equations (3) and (5), the coefficient vector W of the scan-and-sum beamformer is represented by the following equation (7).

Equation (6) may be replaced by the following equation (8) in consideration of the weight B=[b ₁ , b ₂ , _. . . , _bi , .

In this case, by solving the problem of minimizing the mean squared error (MSE) of P(θ) and D(θ), it can be uniquely determined as the following equation (9).

However, Q(θ)= _[ p _φ1 (θ), p _φ2 (θ), . . . , p _φi (θ), .

<Error analysis criteria>
Next, the criteria for error analysis will be described.
Evaluate the difference between the scan-and-sum beam pattern and the ideal pattern to see if the scan-and-sum pattern accurately approximates the desired pattern. From equation (6), P _θi (θ) is a function of the _number of microphones N, and the interval of φ _i is θ _scan , which is the same as the interval of θ _i . is a function of To show this explicitly, we redefine P([theta]) as PN _,[theta]scan ([theta]). At this time, MSE (Mean Square Error), which is a reference for evaluation, can be formulated as in the following equation (10).

The MSE is determined using two variables, the number of microphones N and the scan angle θ _scan .

FIG. 4 is a diagram showing the relationship between the setting of the microphone array used for evaluation and the surface sound source of the separation target. The azimuth of about 75 to 105 degrees is the separation target surface sound source (g31), the azimuth of about 45 to 75 degrees is the first interference wave (g32), and the azimuth of about 105 to 135 degrees is the second interference. Wave (g33). An interference wave is, for example, noise emitted from around a planar sound source to be extracted. Evaluation conditions were that the distance between the microphones 21 was 2 cm, f=2 kHz, the number of microphones was 20, and the scan angle θ _scan was 0.92 degrees. The sound source used for the evaluation is a far-field, plane sound source far from the microphone.

Also, let F be the set of slices of frequency in the FFT (Fast Fourier Transform) and |F| Emphasizing the error in the region where the sound source exists, the expression (10) is expressed as the following expression (11).

In equation (9), Θ _itf is the angle of the existence range of the first and second interference waves in FIG. 4 as shown in FIG. 5, and Θ _tar is the angle of the existence range of the target sound source in FIG. FIG. 5 is a setting example for extracting a surface sound source to be separated and suppressing surrounding noise. FIG. 6 is a diagram showing an example of a beamformer for adaptive MVDR. FIG. 7 is an MSE surface when the adaptive MVDR beamformer of FIG. 6 is used. 5 and 6, the horizontal axis is the angle (degrees) and the vertical axis is the gain (dB). In FIG. 7, the horizontal axis of the paper is the number of microphones N (pieces), the depth axis is the scan angle θ _scan (degrees), and the vertical axis is the logarithmic MSE.

As shown in FIG. 6, the adaptive MVDR beamformer pattern has a gain difference of about 10 dB between the separation target surface sound source and the interference wave. As shown in FIG. 7, the MSE was about -20 even when the number of microphones N was 100 and the scan angle was close to 0 degrees. Also, even when the number of microphones N was 100, the MSE was about -5 when the scan angle was widened.

FIG. 8 is a diagram showing a DS (Delay and Sum) beamformer. FIG. 9 is an MSE curved surface when the DS beamformer of FIG. 8 is used. In FIG. 8, the horizontal axis is the angle (degrees) and the vertical axis is the gain (dB). In FIG. 9, the horizontal axis of the paper is the number of microphones N (pieces), the depth axis is the scan angle θ _scan (degrees), and the vertical axis is the logarithmic MSE.

As shown in FIG. 8, in the pattern of the DS beamformer, the gain difference between the surface sound source of the separation target and the interference wave is about 6 dB even if the angle is separated by 30 degrees. As shown in FIG. 9, the MSE was about -10 even when the number of microphones N was 100 and the scan angle was close to 0 degrees. Also, even when the number of microphones N was set to 100, the MSE was about -7 when the scan angle was widened.

<Cost used for evaluation>
Here, the number of microphones (N) can be regarded as a physical cost. The reason for this is that more microphones require larger equipment and more hardware to accommodate them.
In equation (5), the scan angle (θ _scan ) can be viewed as a computational cost because the number of sub-beamformers (N _S ) is inversely proportional to the scan angle. We also need to calculate P _θi .

In general, increasing cost improves performance. Although not completely monotonic, the MSE improves with increasing microphone number N or decreasing scan angle θ _scan . In general, minimizing the MSE incurs unrealistic and unacceptable costs, so to uniformly measure both performance and cost, we introduce the cost function J of Equation (12).

(12), α is a predetermined value and λ _i (i=1, 2) is an adjustment parameter. In the following evaluation, λ ₁ =0.0159, λ ₂ =0.000159, α=0.2, and MSEmax=50. Also, in equation (12) we treat both the slice of the MSE surface that contains the expected performance and the region of N, θ _scan . Since the values of MSE and cost vary over a wide range, we need λ _i (i=1,2) to act as weights.

The third and fourth terms of equation (10) are normalized costs. where λ ₂ is chosen to normalize the maximum number of practical microphones (eg 1/100 = 0.01). Similarly, λ ₃ is chosen as the smallest possible scan angle (eg, 0.01°).
The MSE inter-slice weightings (first two items) and cost regions (last two items) can be adjusted by changing the values of λ _i (i=1,2,3).
The trade-off between performance and cost can be expressed as the following equation (13).

Note that arg min J(x) is the set of x that minimizes J(x).
According to this embodiment, by introducing the cost function J, it is possible to select the optimum number of microphones for extracting a plane sound source and the interval for dividing a desired region.

<Optimization and parameter adjustment>
Guidelines on how to implement and optimize the scan-and-sum beamformer of this embodiment are as follows.

I. The type of microphone array is determined according to the space coverage that needs to be covered. For example, if you want to cover 360 degrees of azimuth, choose a circular microphone array. The ideal pattern is determined by equation (4) according to the prerequisite localization information.

II. The user has the flexibility to choose different types of sub-beamformers. However, the user should choose carefully according to the application scenario. For example, in strong noise environments, the DS sub-beamformer may be suitable.

III. After synthesizing the scan-and-sum beam patterns, the MSE surface can be computed on a set of grids, eg, grids in the range Nε[6,100], θ _scan ε[0.01,10] on a logarithmic scale.

IV. MSE, number of microphones N, scan angle θ Expand into a cost function J so that the minimum point in _{the scan} space can be determined by a grid search algorithm.

V. The scan-and-beamformer settings determined by the cost function are preferably compared, eg, with the Wiener filter and the ideal pattern, and also numerically evaluated to confirm the results.

Although solving optimization problems analytically is difficult, it is desirable to find the best theoretical solution and then find a practical solution that approximates it. Wiener filters, known as optimal adaptive filters, require both spatial correlation information of the mixture and an ideal reference signal. Wiener filters are impractical because users typically do not have a target signal to serve as a reference. Comparing the scan-and-sum beamformer and the Wiener filter, the scan-and-sum beamformer of the present embodiment is a good approximation of the optimal solution.

In the adaptive design, the patterns of the adaptive sub-beamformers are slightly different for different surface source locations. FIG. 10 shows a surface sound source setting different from that in FIG. FIG. 10 is a diagram showing a setting example of a plane sound source as a separation target. In the example shown in FIG. 10, there are a separation target surface sound source (g31) and two interference waves (g32, g33). A buffer space (g35) occupying an azimuth angle of 30 degrees is provided between the surface sound source (g31) and the first interference wave (g32). A buffer space (g36) occupying an azimuth angle of 30 degrees is provided between the surface sound source (g31) and the second interference wave (g33).

In the following description, all MSE and cost surfaces are calculated without buffer space.

(first example)
In the first example, an example in which P _θi of the DS sub-beamformer is selected will be described.
DS sub-beamformers are the most basic of spatial filters and specialize in simplicity of implementation and robustness to strong noise, but DS designs are generally suboptimal because they are not adaptive to the signal function.

FIG. 11 is a diagram showing an MSE curved surface of a DS scan-and-sum beamformer as a sub-beamformer. The horizontal axis of the paper is the number of microphones M (pieces), the depth axis is the scan angle interval Δθ (degrees), and the vertical axis is the logarithmic MSE. Note that FIG. 11 is the result of calculation on a logarithmic scale within θ _scan ε[0.01, 10] every three Nε[2, 50]. The MSE curved surface in FIG. 11 changes monotonously with changes in the number of microphones N and the scan angle interval Δθ, and it is difficult to find the minimum value. Therefore, in the DS scan-and-sum beamformer, since it is difficult to manually select the parameter set of the number of microphones N and the scan angle interval Δθ, a cost curved surface is introduced.

FIG. 12 is a diagram showing the cost surface of the DS scan-and-sum beamformer with the sub-beamformer. The horizontal axis of the paper is the number of microphones N (pieces), the depth axis is the scan angle θ _scan (degrees), and the vertical axis is the cost function J. In the cost curved surface of FIG. 12, the parameter with the minimum J value was N=36 and θ _scan =0.64 degrees by the grid search process due to the change in the number of microphones N and the scan angle θ _scan .

FIG. 13 is a diagram showing pattern examples of the winner beamforming and the scan-and-sum beamformer. The horizontal axis is the arrival direction θ [degrees], and the vertical axis is the gain (20log|P _θDOA |) [dB]. The pattern g41 is a scan-and-sum beamformer pattern using delay-and-sum (DS). The pattern g42 is an ideal pattern for a surface sound source. A pattern g43 is a beamformer using a Wiener filter for comparison.
The value of MSE for this setting is 40.8, which was found to be lower than in FIG. 11, but the high level sidelobes of the beam pattern are poorly isolated.

(Second example)
In a second example, an example of selecting P _θi for the MVDR sub-beamformer will be described.
The MVDR sub-beamformer is an adaptive filter with high directivity and low distortion in both amplitude and phase, and exploits localization information and spatial correlation of the mixed signal that can be estimated by the instantaneous value of the mixture. Therefore, the MVDR design becomes practical in common scenarios.
FIG. 14 is a diagram showing the MSE curved surface of the MVDR scan-and-sum beamformer as the sub-beamformer. Each axis is the same as in FIG. FIG. 15 is a diagram showing the cost surface for the sub-beamformer MVDR scan-and-sum beamformer. Each axis is the same as in FIG.

The recommended parameter settings for the cost function C are N=18 and θ _scan =1.0°. FIG. 16 shows the pattern of the synthesized MVDR scan-and-sum beamformer with the recommended settings. In FIG. 16, each axis is the same as in FIG. In FIG. 16, symbol g52 is an ideal pattern for a planar sound source. Reference g53 is an MVDR scan-and-sum beamformer. 13 and 16, the MVDR scan-and-sum beamformer approximates the Wiener filter pattern. Thus, the MVDR scan-and-sum beamformer of the present embodiment is easy to implement and can implement patterns in Wiener filters.

In FIG. 11, the MSE surface of the DS scan-and-sum beamformer is not monotonic with a small number of microphones, but the MSE improves with an increase in the number of microphones. Basically, as the number of microphones N increases, the directivity of the sub-beamformer (P _θi ) improves. Insufficient phase synchronization of the outputs from each DS sub-beamformer can result in erratic performance along the scan axis.
From the DS scan-and-sum beamformer example, desirable properties include the ability to achieve high directivity of the sub-beamformer with a small number of microphones and the ability to avoid phase shifts. These functions lead to the MVDR filter.

Although the MVDR filter has some advantages, the design of the MVDR sub-beamformer assumes spatially independent point sources. However, there is basically no restriction on whether or not the surface sound source is generated as a uniform signal. do. This simplified simulation proves that the MVDR filter is theoretically more effective than the Wiener filter. However, it is difficult to keep the point sources independent which degrade the performance of each MVDR sub-beamformer.

FIG. 14 shows that the MSE surface of the MVDR scan-and-sum beamformer has a valley with a minimum value when viewed in the scan interval direction, and the scan interval with this minimum value tends to become a smaller value as the number of microphones N increases. There is This local minimum of the MSE surface of the MVDR scan-and-sum beamformer is characteristic compared to FIG. This phenomenon shows that as the physical cost increases, the computational cost also increases, otherwise the performance decreases. This is due to the high directivity of the MVDR.

12 and 15 are easier to detect the minimum value than in FIGS. 11 and 14. FIG. Therefore, it can be said that the cost curved surface is more suitable for detecting N and θ scan than the MSE curved surface.

FIG. 17 is a diagram showing an example of a beam pattern when low-density scanning is selected. Each axis is the same as in FIG. In FIG. 17, reference numeral 61 indicates the MVDR pattern. Reference g62 indicates a scan-and-sum pattern. Reference g63 indicates an ideal pattern.
In the example shown in FIG. 17, the number of microphones N is 100, and the angle of θ _scan is 5°. When the number of microphones is increased, the directivity of the microphones becomes so high that the main lobe of the beam pattern becomes narrower, but the scan angle becomes insufficient so that gaps appear in the connection regions of the sub-beamformers. As a result, a surface pattern is generated in which the information of the target area is lost.
Therefore, in the present embodiment, the sound source separation unit 34 (FIG. 1) performs evaluation using the cost function J to select an appropriate number of microphones and scan angles.

<Evaluation results>
Next, the evaluation results will be explained.
In the following evaluation, numerical simulations were performed to evaluate the performance of the scan-and-sum beamformer of this embodiment using the P _θi of the MVDR sub-beamformer. In the numerical simulation, the distance d between microphones was set to 2 cm, the number of microphones N was set to 20, and the angle of θ _scan was set to 0.5° according to design guidelines. In this case the cost is slightly higher than the recommended parameter settings for the cost function J.

The planar sound sources used for evaluation were set adjacent to each other in a small buffer space as shown in FIG. The separation target surface sound source is a male voice, the starting azimuth angle θ _b is 75°, the ending azimuth angle θ _e is 105°, the first interference wave is piano music, and the second interference wave is The waves are white noise, each distributed within a range of 30° with a density Δθ=0.1°.

In the evaluation, the mixing matrix A was used to synthesize the surface sound source as Equation (2). Spatially independent white noise was added to the mixture, so the simulation is under the condition of 20 dB SNR. The sampling rate Fs was set to 44:1 kHz to cover a typical wideband signal. Also, the duration of the audio file is about 6 seconds.

The evaluation used the Matlab® toolbox BSS EVAL version 2.1 to evaluate Signal-to-Interference Ratio (SIR), Sources to Artifacts Ratio (SAR), and Signal separation was analyzed for Signal to Distortion Ratio (SDR).

SIR can evaluate the separation rate of the results.
SAR can evaluate the sound quality of the algorithm. This is because good algorithms produce few artifacts.
SDR can evaluate distortions related to interference, disturbances, and noise.

FIG. 18 is a diagram showing evaluation results for three different buffer spaces.
For a practical spatial filter with a transition band (the band between the passband and the attenuation band), isolation with 0° buffer space is the most difficult, but the SIR is about 18 dB compared to the mixed state ( = 4.7 + 13.0) improved.
As shown in FIG. 18, increasing the angle of the buffer space further improved the SIR. In MVDR sub-beamformers, artifact noise is caused by uncertain estimation of spatial features. As a result, the narrower the buffer space, the lower the SAR, and the SDR was improved by about 9 dB.

FIG. 19 is a diagram showing an example of separated surface sound sources according to this embodiment. In FIG. 19, the horizontal axis of the paper is the azimuth angle (degrees), the depth axis is the number of frames, and the vertical axis is the spectrum (logarithmic expression). The arrangement of sound sources and microphones used for evaluation is the same as in FIG. As shown in FIG. 10, the plane sound source is arranged with an azimuth angle between 75 degrees and 105 degrees.
As a result of performing sound source separation using the scan-and-sum beamformer of this embodiment, surface sound sources could be appropriately separated as shown in FIG.

The advantage of surface source separation according to this embodiment is that various types of sub-beamformers can be selected according to the types of sound sources and microphone arrays, and that various ideal patterns can be defined for sound sources of various sizes and shapes. is. Thus, the scan-and-sum beamformer is very flexible for different scenarios.
Furthermore, in this embodiment, in order to evaluate the quality of the selected beam pattern, an MSE with a cost function J that balances performance and cost can be used for accurate evaluation. Further, according to the present embodiment, the optimum number of microphones and the optimum scan angle can be obtained from the evaluated MSE.

Also, as in the first and second examples above, according to this embodiment, guidelines on how to implement a scan-and-sum beamformer with tuned parameters and optimized patterns are useful.
Numerical simulations also show that the present embodiment improves the SIR of a mixture of three surface sources in difficult situations.

As described above, in this embodiment, the sub-beamformer is used for each subdivided area, and the acoustic signal of the subdivided area is extracted from the collected acoustic signal by the beamforming method. Then, in this embodiment, a desired planar sound source is separated by adding the acoustic signals extracted for each subdivided area.
As a result, according to the present embodiment, it is possible to appropriately suppress the sound source located in the region to be suppressed, and to appropriately separate the surface sound source.

The sound source separation system 1 described above can be applied to various devices such as robots, reception systems, in-vehicle voice recognition systems, smart speakers using voice recognition, and home appliances using voice recognition.
According to this embodiment, it is possible to appropriately suppress a sound source in a region to be suppressed, and to appropriately separate an area sound source. Therefore, it is possible to improve the robot's performance in applications such as human-robot interaction. can.

A program for realizing all or part of the functions of the sound source separation device 3 of the present invention is recorded in a computer-readable recording medium, and the program recorded in this recording medium is read into a computer system and executed. By doing so, all or part of the processing performed by the sound source separation device 3 may be performed. It should be noted that the "computer system" referred to here includes hardware such as an OS and peripheral devices. Also, the "computer system" includes a WWW system provided with a home page providing environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems. In addition, "computer-readable recording medium" means a volatile memory (RAM) inside a computer system that acts as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. , includes those that hold the program for a certain period of time.

Further, the above program may be transmitted from a computer system storing this program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in a transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing part of the functions described above. Further, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in the computer system.

As described above, the mode for carrying out the present invention has been described using the embodiments, but the present invention is not limited to such embodiments at all, and various modifications and replacements can be made without departing from the scope of the present invention. can be added.

Reference Signs List 1 sound source separation system 2 sound pickup unit 3 sound source separation device 21, 21-1, 21-N microphone 31 acquisition unit 32 transfer function storage unit 33 beam pattern storage Part 34... Sound source separation part 35... Output part 341... Separation part 342... Evaluation part 343... Selection part

Claims (7)

a microphone array having N (N is an integer equal to or greater than 2) microphones arranged at a first interval for picking up acoustic signals;
A desired area is subdivided at a second interval, and acoustic signals picked up by the microphone array are beamed to each of the subdivided areas using sub-beams corresponding to the azimuth angles θ of the subdivided areas. a sound source separation unit that separates and extracts by a forming method and adds the extracted acoustic signals to separate the acoustic signals of the desired region;
A sound source separation device. The pattern of the sub-beams is represented by the following equation,

In the above equation, D(θ) is 1 when the starting azimuth angle _θb of the sound source is equal to _or less than the ending azimuth angle θe of the source, and is 0 at other azimuth angles. This is an ideal filter, P _θi is the directional characteristic of a beamformer targeting a point sound source when the directivity of the beam is directed in the _θi direction, and _θi is the direction of the directivity of each beamformer. , H means the Hermitian conjugate, a(θ) is the transfer function to the sound source or said microphone, W _θi is the coefficient vector of the beamformer P _θi , where the coefficients W of the beamformer for the surface sound source are and

When D(θ) is given by the following formula,

can be defined as θ _i =θ _b +(i−1) θ _scan , where b is the value of i that satisfies θ _i =θ _b , and e is the value of i that satisfies θ _i =θ _e ;
The sound source separation device according to claim 1. an evaluation unit that calculates the relationship between the cost function J, the number of microphones, and the second spacing;
a selection unit that selects the number of microphones and the second interval that minimizes the cost function in the relationship between the cost function calculated by the evaluation unit, the number of microphones, and the second interval;
The sound source separation device according to claim 1 or 2, further comprising: The evaluation unit
Calculate the difference between the beam pattern and the ideal pattern using the logarithmic mean squared error MSE of

Calculate the cost function J using the following equation, and based on the calculated cost function J, the number N of the microphones, and the scan angle θ _scan , the optimum for separating the acoustic signal of the desired region Obtaining the number N of the microphones and the scan angle θ _scan ,

where α is a predetermined value and λ ₁ and λ ₂ are tuning parameters,
The sound source separation device according to claim 3. The evaluation unit
By representing the cost function J, the number N of microphones, and the scan angle θ _scan in a three-dimensional graph, and selecting the number N of microphones and the scan angle θ _scan that minimize the cost function J, the optimum the number N of microphones and the scan angle θ _scan ,
The sound source separation unit
update to the optimal number of microphones N selected by the evaluator and the scan angle θ _scan ;
The sound source separation device according to claim 4. A microphone array having N (N is an integer equal to or greater than 2) microphones arranged at a first interval picks up an acoustic signal,
The sound source separation unit subdivides the desired region at second intervals,
The sound source separation unit separates and extracts an acoustic signal for each of the subdivided regions by a beamforming method using a sub-beam corresponding to the azimuth angle θ of the subdivided region,
The sound source separation unit separates the acoustic signal of the desired region by adding the extracted acoustic signals.
sound source separation method. to the computer,
Acoustic signals are picked up by a microphone array having N (N is an integer equal to or greater than 2) microphones arranged at a first interval,
subdividing the desired area at a second interval;
For each of the subdivided regions, the acoustic signal is separated and extracted by a beamforming method using a sub-beam corresponding to the azimuth angle θ of the subdivided region;
separating the acoustic signal of the desired region by adding the extracted acoustic signals;
program.

Images